Audio-Visual Panoramas and Spherical Audio Analysis using the Audio Camera
Author(s)
Duraiswami, Ramani
O'Donovan, Adam
Advisor(s)
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
Capturing a scene for later or contemporaneous display
needs to capture the complex interactions between the
source(s) in the scene and the environment. High order
spherical Ambisonics and plane-wave analysis are
powerful mathematical tools for such scene analysis. The
spherical microphone array (and its embodiment in the
Audio Camera) is a useful tool for capture and analysis of
scenes. Further information about the environment is
available from the visual scene.
We present the audio-visual panoramic camera as a
tool that greatly simplifies the task of processing audio
visual information by providing one common framework
for both modalities. Via the Audio Camera [1], we show
that microphone arrays can be viewed as a central
projection camera that can effectively image the audible
acoustic frequency spectrum. We demonstrate a new
device, the audio visual Panoramic camera that is
composed of a 64 channel spherical microphone array
combined with a 5 element video camera array. The
combined sensor is capable of real-time audio visual
panoramic image generation using state of the art NVidia
Graphics cards. It also provides an order-7 ambisonic
description of the scene.
Sponsor
Date
2010-06
Extent
Resource Type
Text
Resource Subtype
Proceedings