International Conference on Auditory Display (ICAD)

Series

International Conference on Auditory Display (ICAD)

Permanent Link

https://hdl.handle.net/1853/70916

Series Type

Event Series

Associated Organization(s)

Organizational Unit

Sonification Lab

Full item page

Publication Search Results

Now showing 1 - 10 of 1222

THE AUDIBLE UNIVERSE WORKSHOP: AN INTERDISCIPLINARY APPROACH TO THE DESIGN AND EVALUATION OF TOOLS FOR ASTRONOMICAL DATA SONIFICATION

(Georgia Institute of Technology, 2023-06) Misdariis, Nicolas ; Pauletto, Sandra ; Bonne, Nicolas ; Harrison, Chris ; Meredith, Kate ; Zanella, Anita

Even if images of astrophysical objects are used by professional astronomers for research and by the public for outreach, we are all basically blind to the Universe. Challenging the idea that we should always use visualisations, there has been a growing interest in converting astronomical phenomena into sound, motivated by: making astronomy more accessible to people who are blind or visually impaired (BVI); creating more engaging educational resources, and enabling a deeper understanding of complex astronomical data. The Audible Universe (AU) workshop focuses on consolidating what has been done in the field so far and identifying the areas where most effort is required to make progress over the coming years. The second edition ofthe AU workshop (AU2) took place in 2022, and brought together 50 experts, among whom astronomers interested in sonification, sound designers, experts in sound perception and educators. This community started a multi-disciplinary discussion about how to properly design and evaluate sonification tools. In this methodological and position paper, we present and discuss the main activities of the AU2 workshop, with a particular focus on activities concemed with the development of collaborative design processes, and the implementation of methods for evaluation. While this workshop was dedicated to fostering exchanges between the sonification community and astronomers, the structure and the methods used within the workshop are transferable to other application areas, and a contribution to the effort to develop interdisciplinary strategies for the development of the field of sonification.
AN AUDITORY DISPLAY FOR REMOTE ROAD VEHICLE OPERATION THAT INCREASES AWARENESS AND PRESENCE

(Georgia Institute of Technology, 2023-06) Larsson, Pontus ; de Souza, Johanna Bergfelt Ramos ; Begnert, Joel

The activities related to the development of automated road vehicles have been ever-increasing during the past few years. Automated Driving (AD) may bring positive societal effects in terms of reduced environmental impact, improved traffic safety, and more efficient mobility systems. Despite the rapid progress in this field, there are still many challenges to be solved before high level driving automation (SAE’s Levels 4-5) can be realized. Some of the most difficult challenges are related to the handling of scenarios that the AD system is unprepared for. Such scenarios could be approaching construction sites, public events, or traffic accidents. Other difficult AD-related challenges include e.g. sensor or system failure or when the vehicle approaches the limits of its Operational Design Domain (ODD) - e.g. conditions or situations that the AD system is not designed to handle. It has been proposed that remote operation enabling a human remote operator to support a connected and automated vehicle from a distance when unexpected events or failures occur could be a way to overcome these challenges. Furthermore, this type of technology could also allow a remote operator to act as a safety driver during the testing phases of AD vehicles that are not equipped with regular driver controls. Hence, remote operation has the potential to overcome some of the challenges that currently are slowing down the commercialization of automated vehicles.
DESIGN, IMPLEMENTATION AND EVALUATION OF AN AIR QUALITY SONIFICATION SYSTEM

(Georgia Institute of Technology, 2023-06) Terboven, Carla ; Steckhan, Nico ; Panourgia, Eleni-Ira ; Arnrich, Bert

Air pollution is the world’s deadliest environmental risk factor. Yet there is little effort to educate the public about personal exposure to pollutants such as particulate matter (PM). I would write it a bit different: This paper presents the design and implementation of a portable sensor box (PSB) to collect local, spatially highly resolved particulate matter data. To counteract common inaccuracies in mobile particulate matter measurements, data were aggregated and cleaned according to their location in a real-world investigation. We employed parameter mapping to develop a real-time, interactive and intuitive, yet scientifically accurate sonification of the data. The sonification was made accessible to listeners within a test area in Potsdam using a physical prototype. Through the implementation of a scripted exposure design, we investigated realistic, comparable statements and evaluation by the participants about the sonification device and the sonification.
SONIFYING GAIT KINEMATICS USING THE SOUND OF WADING: A STUDY ON ECOLOGICAL MOVEMENT REPRESENTATIONS

(Georgia Institute of Technology, 2023-06) Kantan, Prithvi ; Dahl, Sofia ; Serafin, Stefania ; Spaich, Erika G.

Interactive sonification of human movement can enhance motor learning by introducing an additional sensory feedback channel during rehabilitation. Successful past work on sonifying human walking has focused primarily on providing users with feedback on the stance (ground contact) phase of the gait cycle, whilst the equally important swing phase has been largely neglected. In this work, we developed and assessed a swing phase sonification algorithm that generates ecological feedback on limb swing in the form of wading sounds at two different depths. We evaluated the perceptual qualities of the algorithm output in a test with 16 respondents, as well as the user experience of 9 healthy participants in a reallife walking pilot test. Despite our simple approach of simulating wading acoustics using gyroscope-modulated liquid sounds, our results suggested that the simulation sounded perceptually natural, unintrusive during walking, and responded well to limb swing in real-time. Future work includes further improvements to the algorithm followed by rigorous user tests with both healthy and impaired individuals. We believe that this research can contribute to the development of meaningful auditory feedback schemes targeting limb swing.
FACIAL BEHAVIOR SONIFICATION WITH THE INTERACTIVE SONIFICATION FRAMEWORK PANSON

(Georgia Institute of Technology, 2023-06) Nalli, Michele ; Johnson, David ; Hermann, Thomas

Facial behavior occupies a central role in social interaction. Its auditory representation is useful for various applications such as for supporting the visually impaired, for actors to train emotional expression, and for supporting annotation of multi-modal behavioral corpora. In this paper we present a prototype system for interactive sonification of facial behavior that works both in realtime mode, using a webcam, and offline mode, analyzing a video file. The system is based on python and Jupyter notebooks, and relies on the python module sc3nb for sonification-related functionalities. Facial feature extraction is realized using OpenFace 2.0. Designing the system led to the development of a framework of reusable components to develop interactive sonification applications, called Panson, which can be used to easily design and adapt sonifications for different use cases. We present the main concepts behind the facial behavior sonification system and the Panson framework. Furthermore, we introduce and discuss novel sonifications developed using Panson, and demonstrate them with a set of sonified videos. The sonifications and Panson are Open Source reproducible research available on GitHub.
NEURODIVERGENCE IN SOUND: SONIFICATION AS A TOOL FOR MENTAL HEALTH AWARENESS

(Georgia Institute of Technology, 2023-06) Nadri, Chihab ; Al Mater, Hamza ; Morrison, Spencer ; Tiemann, Allison ; Song, Inuk ; Lee, Tae Ho ; Jeon, Myounghoon

The need to build greater mental health awareness as an important factor in decreasing stigma surrounding individuals with neurodivergent conditions has led to the development of programs and activities that seek to increase mental health awareness. Using a sonification approach with neural activity can effectively convey an individual’s psychological and mental characteristics in a simple and intuitive manner. In this study, we developed a sonification algorithm that alters existing music clips according to fMRI data corresponding to the salience network activity from neurotypical and neurodivergent individuals with schizophrenia. We conducted an evaluation of these sonifications with 24 participants. Results indicate that participants were able to differentiate between sound clips stemming from different neurological conditions and that participants gained increased awareness of schizophrenia through this brief intervention. Findings indicate sonification could be an effective tool in raising mental health awareness and relate neurodivergence to a neurotypical audience.
TWICE STEPPED IN STILL WATERS: SONIFICATION AND INTERDISCIPLINARITY AS ARTISTIC RESEARCH

(Georgia Institute of Technology, 2023-06) Boehringer, Jorge

Real-time EEG and sensor data sonification have been researched extensively. New implementations of these technologies in the fields of live electronic music performance and sonic art generally focus on instrumentalising brain waves as an interface, or are concerned with controlling brain waves through biofeedback. Twice Stepped in Still Waters is instead concerned with using bio and sensor feedback as a focused framework accommodating phenomenological listening for the performer. Furthermore, when presented before an audience, this activity results in circumstances for sound and listening (music). Currently available high-resolution consumer devices and open-source software have now rendered these technologies accessible to a wide pool of researchers from a variety of fields. The present work uses sonified real-time EEG and accelerometer data to situate phenomenological concerns of the performance situation within the field ofartistic research. Discussion of the technical development and artistic context for the piece reveals a broad interdisciplinary scope. Twice Stepped in Still Waters may be therefore read as a case study offering an implementation and a performance methodology for sonification and focused listening afforded by these sensor technologies. Variations on the approaches described may be applied by other investigators working within or outside of the domain of artistic research.
TOWARDS AN EX-SITU ECOLOGICALLY VALID SOUNDSCAPE EVALUATION METHODOLOGY USING IMMERSIVE AUDIO

(Georgia Institute of Technology, 2023-06) Dedousis, Giorgos ; Andreopoulou, Areti

The design of soundscape evaluation protocols, realized either in-situ (real-world) or in laboratory conditions, and the assessment of their ecological validity is gaining interest among researchers. This paper reports on an experiment that aims to simulate the listening conditions of interacting with a travel-guide application presenting users with recreations of historically-informed soundscapes, designed to be used in-situ, in a laboratory setting. This was achieved by simulating the real-world soundscapes, which would have surrounded users of the app when used outside, from a periphonic speaker arrangement in a sound-treated space using an 2D ambisonic reproduction of the corresponding data. The evaluation results showed that participants interacted with the travelguide content in a manner that resembled a real-life situation. That is, they were in general able to recognize and separate the auditory content of the travel-guide from that of the real-world simulated auditory space they where placed within, with some occasional confusions. This study adds to the work on the ecological validity of soundscape evaluations and tries to highlight the boundaries between the two simultaneous soundscape auditions.
PROOF OF CONCEPT OF A GENERIC TOOLKIT FOR SONIFICATION: THE SONIFICATION CELL IN OSSIA SCORE

(Georgia Institute of Technology, 2023-06) Poret, Maxine ; Desainte-Catherine, Myriam ; Celerier, Jean-Michaël ; Semal, Catherine

A popular topic in sonification research is the development of a complete, user-friendly tool for sonification creation. A study of existing attempts at such tools highlights what seems to be the main challenge of this endeavor: exhaustiveness with regards to the great diversity of approaches for auditory display of information. Most tools are designed to allow for selecting among a few typical modalities, but could not really be used to create any kind of sonification. In order to tackle this issue, we proposed a theoretical model of the sonification process, which is intended to take all of its properties as a data observation technique into account. The goal for this model is to be translated into a user interface and programming approach as part of a sonification toolkit. In the present paper, we report our work in creating a proof of concept of such a toolkit using the ossia musical sequencing environment, chosen for its proximity to our objectives in terms of user interaction and library of functionalities. This prototype was tested to recreate two of our previous data sonification works. Most of the specificities of these case studies could be recreated properly, though some of the planned features, notably for grain synthesis, are currently missing from the ossia environment. For our future works, we will consider that the current state of this proof of concept is sufficient to start studying the user experience of sonification designers interacting with the toolkit.
ASSESSING POWER IN PUNCHING SOUNDS FROM MAINSTREAM FILMS AND VIDEO GAMES

(Georgia Institute of Technology, 2023-06) Järveläinen, Hanna ; Hug, Daniel ; Papetti, Stefano

A preliminary study was conducted with the aim of contributing to a scientifically based sound design for interactive movement sonification, drawing on tacit sound design knowledge. Specifically, the perceived power in punching sounds from mainstream films and video games was assessed through subjective ratings. The same sound clips were analyzed to extract audio features, and correlations with power ratings were examined. Results showed that perceived power in punching sounds is especially influenced by decay time, spectral slope, and features related to Mel-frequency cepstral coefficients (MFCCs). These findings can inform sound design for interactive movement sonification, particularly in the context of action-oriented applications.