Sonic Shapes: Visualizing Vocal Expression
Author(s)
Pietrowicz, Mary
Karahalios, Karrie
Advisor(s)
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
Sound has been an overlooked modality in visualization. Why?
Because it is ephemeral. We experience it as it happens, often
in community with others. Then, the sound is gone.
Furthermore, sound in human communication is
multidimensional and includes the semantic meaning of words,
the meaning of expressive verbal gestures (paralingual and
prosodic components), the nonvocal gestures, and relational
gestures. Even though we are able to record sound and play it
back, we typically focus more on the semantic meaning of
words. In this paper, we describe a voice analytics toolkit and
present visualizations that focus on the relational and
expressive verbal gestures in speech. By making the
overlooked channels of human communication visible and
persistent, we make it possible to see beneath the surface of our
words. This insight will potentially enable the development of
new applications for speech therapy, the quantization and
visualization of vocal trends common to speakers with medical
conditions such as autism spectrum disorder (ASD), the
characterization and visualization of communication patterns
common in different kinds of relationships and cultures, and the
development of new kinds of creative, multimodal works.
Sponsor
Date
2013-07
Extent
Resource Type
Text
Resource Subtype
Proceedings