Title:
Multi-Cue Contingency Detection
Multi-Cue Contingency Detection
Files
Author(s)
Lee, Jinhan
Chao, Crystal
Bobick, Aaron F.
Thomaz, Andrea L.
Chao, Crystal
Bobick, Aaron F.
Thomaz, Andrea L.
Advisor(s)
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
The ability to detect a human's contingent response is an essential skill for a social robot attempting to engage new interaction partners or maintain ongoing turn-taking interactions. Prior work on contingency detection focuses on single cues from isolated channels, such as changes in gaze, motion, or sound.We propose a framework that integrates multiple cues for detecting contingency from multimodal sensor data in human-robot interaction scenarios. We describe three levels of integration and discuss our method for performing sensor fusion at each of these levels. We perform a Wizard-of-Oz data collection experiment in a turn-taking scenario in which our humanoid robot plays the turn-taking imitation game “Simon says" with human partners. Using this data set, which includes motion and body pose cues from a depth and color image and audio cues from a microphone, we evaluate our contingency detection module with the proposed integration mechanisms and show gains in accuracy of our multi-cue approach over single-cue contingency detection. We show the importance of selecting the appropriate level of cue integration as well as the implications of varying the referent event parameter.
Sponsor
Date Issued
2012-04
Extent
Resource Type
Text
Resource Subtype
Article
Pre-print
Pre-print