Person:
Freeman,
Jason
Publication Search Results
Data-Driven Live Coding with DataToMusic API
Live Coding With EarSketch
Person:
Freeman,
Jason
1 - 2 of 2
Item
Creating interactive audio applications for web browsers
often involves challenges such as time synchronization
between non-audio and audio events within thread
constraints and format-dependent mapping of data to
synthesis parameters. In this paper, we describe a unique
approach for these issues with a data-driven symbolic
music application programming interface (API) for rapid
and interactive development. We introduce DataToMusic
(DTM) API, a data-sonification tool set for web browsers
that utilizes the Web Audio API1 as the primary means of
audio rendering. The paper demonstrates the possibility of
processing and sequencing audio events at the
audio-sample level by combining various features of the
Web Audio API, without relying on the
ScriptProcessorNode, which is currently under a redesign.
We implemented an audio event system in the clock and
synthesizer classes in the DTM API, in addition to a
modular audio effect structure and a
exible
data-to-parameter mapping interface. For complex
real-time configuration and sequencing, we also present a
model system for creating reusable functions with a
data-agnostic interface and symbolic musical
transformations. Using these tools, we aim to create a
seamless connection between high-level (musical structure)
and low-level (sample rate) processing in the context of
real-time data sonification.
Item
EarSketch combines a Python / JavaScript API, a digital audio workstation (DAW) visualization, an audio loop library, and an educational
curriculum into a web-based music programming environment. While it was designed originally as a classroom educational tool for music technology and computer science, it has recently been expanded to support live coding in concert performance. This live coding
performance explores the artistic potential of algorithmic manipulations of audio loops in a multi-track DAW paradigm and explores the potential of DAW-driven visualizations to demystify live coding and algorithms for a concert audience.