(Georgia Institute of Technology, 2018-06)
Tsuchiya, Takahiko; Freeman, Jason
Melodic sonification is one of the most common methods of
sonification: data modulates the pitch of an audio synthesizer
over time. This simple sonification, however, still raises
questions about how we listen to a melody and perceive the
motions and patterns characterized by the underlying data. We
argue that analytical listening to such melodies may focus on
different ranges of the melody at different times and discover
the pitch (and data) relationships gradually over time and after
repeated listening. To examine such behaviors in real-time
listening to a melodic sonification, we conducted a user study
employing interactive time and pitch resolution controls for
the user. The study also examines the relationships of these
changing time and pitch resolutions to perceived musicality.
The results indicate a stronger general relationship between
the time progression and the use of time-resolution control to
analyze data characteristics, while the pitch resolution controls
tend to have more correlation with subjective perceptions of
musicality.
(Georgia Institute of Technology, 2017-06)
Tsuchiya, Takahiko; Freeman, Jason
Auditory-display research has had a largely unsolved challenge of balancing functional and aesthetic considerations. While functional designs tend to reduce musical expressivity for the fidelity of data, aesthetic or musical sound organization arguably has a potential for representing multi-dimensional or hierarchical data structure with enhanced perceptibility.
Existing musical designs, however, generally employ
nonlinear or interpretive mappings that hinder the assessment of functionality. The authors propose a framework for
designing expressive and complex sonification using small
timescale musical hierarchies, such as the harmony and timbral
structures, while maintaining data integrity by ensuring a close-to-the-original recovery of the encoded data utilizing
descriptive analysis by a machine listener.