(Georgia Institute of Technology, 2016-04)
Tsuchiya, Takahiko; Freeman, Jason; Lerner, Lee W.
Creating interactive audio applications for web browsers
often involves challenges such as time synchronization
between non-audio and audio events within thread
constraints and format-dependent mapping of data to
synthesis parameters. In this paper, we describe a unique
approach for these issues with a data-driven symbolic
music application programming interface (API) for rapid
and interactive development. We introduce DataToMusic
(DTM) API, a data-sonification tool set for web browsers
that utilizes the Web Audio API1 as the primary means of
audio rendering. The paper demonstrates the possibility of
processing and sequencing audio events at the
audio-sample level by combining various features of the
Web Audio API, without relying on the
ScriptProcessorNode, which is currently under a redesign.
We implemented an audio event system in the clock and
synthesizer classes in the DTM API, in addition to a
modular audio effect structure and a
exible
data-to-parameter mapping interface. For complex
real-time configuration and sequencing, we also present a
model system for creating reusable functions with a
data-agnostic interface and symbolic musical
transformations. Using these tools, we aim to create a
seamless connection between high-level (musical structure)
and low-level (sample rate) processing in the context of
real-time data sonification.
(Georgia Institute of Technology, 2016-04)
Winters, R. Michael; Tsuchiya, Takahiko; Lerner, Lee W.; Freeman, Jason
This paper describes ongoing research in the presentation of
geo-located, real-time data using web-based audio and visualization technologies. Due to both the increase of devices
and diversity of information being accumulated in real-time,
there is a need for cohesive techniques to render this information in a useable and functional way for a variety of
audiences. We situate web-sonification|sonification of web-
based information using web-based technologies|as a particularly valuable avenue for display. When combined with
visualizations, it can increase engagement and allow users
to profit from the additional affordances of human hearing.
This theme is developed in the description of two multi-modal dashboards designed for data in the context of the
Internet of Things (IoT) and Smart Cities. In both cases,
Web Audio provided the back-end for sonification, but a new
API called DataToMusic (DTM) was used to make common
sonification operations easier to implement. DTM provides
a valuable framework for web-sonification and we highlight its use in the two dashboards. Following our description
of the implementations, the dashboards are compared and
evaluated, contributing to general conclusions on the use of
web-audio for sonification, and suggestions for future dashboards.