Person:

Dellaert, Frank

Permanent Link

https://hdl.handle.net/1853/71202

Associated Organization(s)

Organizational Unit

School of Interactive Computing

Full item page

Publication Search Results

Now showing 1 - 9 of 9

How A.I. and multi-robot systems research will accelerate our understanding of social animal behavior

(Georgia Institute of Technology, 2006-07) Balch, Tucker ; Dellaert, Frank ; Feldman, Adam ; Guillory, Andrew ; Isbell, Charles L. ; Khan, Zia ; Pratt, Stephen ; Stein, Andrew ; Wilde, Hank

Our understanding of social insect behavior has significantly influenced A.I. and multi-robot systems’ research (e.g. ant algorithms and swarm robotics). In this work, however, we focus on the opposite question, namely: “how can multi-robot systems research contribute to the understanding of social animal behavior?.” As we show, we are able to contribute at several levels: First, using algorithms that originated in the robotics community, we can track animals under observation to provide essential quantitative data for animal behavior research. Second, by developing and applying algorithms originating in speech recognition and computer vision, we can automatically label the behavior of animals under observation. Our ultimate goal, however, is to automatically create, from observation, executable models of behavior. An executable model is a control program for an agent that can run in simulation (or on a robot). The representation for these executable models is drawn from research in multi-robot systems programming. In this paper we present the algorithms we have developed for tracking, recognizing, and learning models of social animal behavior, details of their implementation, and quantitative experimental results using them to study social insects.
Learning and Inference in Parametric Switching Linear Dynamic Systems

(Georgia Institute of Technology, 2005-10) Oh, Sang Min ; Rehg, James M. ; Balch, Tucker ; Dellaert, Frank

We introduce parametric switching linear dynamic systems (P-SLDS) for learning and interpretation of parametrized motion, i.e., motion that exhibits systematic temporal and spatial variations. Our motivating example is the honeybee dance: bees communicate the orientation and distance to food sources through the dance angles and waggle lengths of their stylized dances. Switching linear dynamic systems (SLDS) are a compelling way to model such complex motions. However, SLDS does not provide a means to quantify systematic variations in the motion. Previously, Wilson & Bobick presented parametric HMMs [21], an extension to HMMs with which they successfully interpreted human gestures. Inspired by their work, we similarly extend the standard SLDS model to obtain parametric SLDS. We introduce additional global parameters that represent systematic variations in the motion, and present general expectation-maximization (EM) methods for learning and inference. In the learning phase, P-SLDS learns canonical SLDS model from data. In the inference phase, P-SLDS simultaneously quantifies the global parameters and labels the data. We apply these methods to the automatic interpretation of honey-bee dances, and present both qualitative and quantitative experimental results on actual bee-tracks collected from noisy video data.
Data-Driven MCMC for Learning and Inference in Switching Linear Dynamic Systems

(Georgia Institute of Technology, 2005-07) Oh, Sang Min ; Rehg, James M. ; Balch, Tucker ; Dellaert, Frank

Switching Linear Dynamic System (SLDS) models are a popular technique for modeling complex nonlinear dynamic systems. An SLDS has significantly more descriptive power than an HMM, but inference in SLDS models is computationally intractable. This paper describes a novel inference algorithm for SLDS models based on the Data- Driven MCMC paradigm. We describe a new proposal distribution which substantially increases the convergence speed. Comparisons to standard deterministic approximation methods demonstrate the improved accuracy of our new approach. We apply our approach to the problem of learning an SLDS model of the bee dance. Honeybees communicate the location and distance to food sources through a dance that takes place within the hive. We learn SLDS model parameters from tracking data which is automatically extracted from video. We then demonstrate the ability to successfully segment novel bee dances into their constituent parts, effectively decoding the dance of the bees.
Multitarget Tracking with Split and Merged Measurements

(Georgia Institute of Technology, 2005-06) Khan, Zia ; Balch, Tucker ; Dellaert, Frank

In many multitarget tracking applications in computer vision, a detection algorithm provides locations of potential targets. Subsequently, the measurements are associated with previously estimated target trajectories in a data association step. The output of the detector is often imperfect and the detection data may include multiple, split measurements from a single target or a single merged measurement from several targets. To address this problem, we introduce a multiple hypothesis tracker for interacting targets that generate split and merged measurements. The tracker is based on an efficient Markov chain Monte Carlo (MCMC) based auxiliary variable particle filter. The particle filter is Rao-Blackwellized such that the continuous target state parameters are estimated analytically, and an MCMC sampler generates samples from the large discrete space of data associations. In addition, we include experimental results in a scenario where we track several interacting targets that generate these split and merged measurements.
What Are the Ants Doing? Vision-Based Tracking and Reconstruction of Control Programs

(Georgia Institute of Technology, 2005-04) Balch, Tucker ; Dellaert, Frank ; Delmotte, Florent ; Khan, Zia ; Egerstedt, Magnus B.

In this paper, we study the problem of going from a real-world, multi-agent system to the generation of control programs in an automatic fashion. In particular, a computer vision system is presented, capable of simultaneously tracking multiple agents, such as social insects. Moreover, the data obtained from this system is fed into a mode-reconstruction module that generates low-complexity control programs, i.e. strings of symbolic descriptions of control-interrupt pairs, consistent with the empirical data. The result is a mechanism for going from the real system to an executable implementation that can be used for controlling multiple mobile robots.
What Are the Ants Doing? Vision-Based Tracking and Reconstruction of Control Programs

(Georgia Institute of Technology, 2005-04) Egerstedt, Magnus B. ; Balch, Tucker ; Dellaert, Frank ; Delmotte, Florent ; Khan, Z.

In this paper, we study the problem of going from a real-world, multi-agent system to the generation of control programs in an automatic fashion. In particular, a computer vision system is presented, capable of simultaneously tracking multiple agents, such as social insects. Moreover, the data obtained from this system is fed into a mode-reconstruction module that generates low-complexity control programs, i.e. strings of symbolic descriptions of control-interrupt pairs, consistent with the empirical data. The result is a mechanism for going from the real system to an executable implementation that can be used for controlling multiple mobile robots.
A Rao-Blackwellized Particle Filter for EigenTracking

(Georgia Institute of Technology, 2004-06) Khan, Zia ; Balch, Tucker ; Dellaert, Frank

Subspace representations have been a popular way to model appearance in computer vision. In Jepson and Black’s influential paper on EigenTracking, they were successfully applied in tracking. For noisy targets, optimization-based algorithms (including EigenTracking) often fail catastrophically after losing track. Particle filters have recently emerged as a robust method for tracking in the presence of multi-modal distributions. To use subspace representations in a particle filter, the number of samples increases exponentially as the state vector includes the subspace coefficients. We introduce an efficient method for using subspace representations in a particle filter by applying Rao-Blackwellization to integrate out the subspace coefficients in the state vector. Fewer samples are needed since part of the posterior over the state vector is analytically calculated. We use probabilistic principal component analysis to obtain analytically tractable integrals. We show experimental results in a scenario in which we track a target in clutter.
Efficient Particle Filter-Based Tracking of Multiple Interacting Targets Using an MRF-based Motion Model

(Georgia Institute of Technology, 2003-10) Khan, Zia ; Balch, Tucker ; Dellaert, Frank

We describe a multiple hypothesis particle filter for tracking targets that will be influenced by the proximity and/or behavior of other targets. Our contribution is to show how a Markov random field motion prior, built on the fly at each time step, can model these interactions to enable more accurate tracking. We present results for a social insect tracking application, where we model the domain knowledge that two targets cannot occupy the same space, and targets will actively avoid collisions. We show that using this model improves track quality and efficiency. Unfortunately, the joint particle tracker we propose suffers from exponential complexity in the number of tracked targets. An approximation to the joint filter, however, consisting of multiple nearly independent particle filters can provide similar track quality at substantially lower computational cost.
The Georgia Tech Yellow Jackets: A Marsupial Team for Urban Search and Rescue

(Georgia Institute of Technology, 2002) Dellaert, Frank ; Balch, Tucker ; Kaess, Michael ; Ravichandran, Ram ; Alegre, Fernando ; Berhault, Marc ; McGuire, Robert ; Merrill, Ernest ; Moshkina, Lilia ; Walker, Daniel

We describe our entry in the AAAI 2002 Urban Search and Rescue (USAR) competition, a marsupial team consisting of a larger wheeled robot and several small legged robots, carried around by the larger robot. This setup exploits complimentary strengths of each robot type in a challenging domain. We describe both the hardware and software architecture, and the on-board real-time mapping which forms the basis of accurate victim-localization crucial to the USAR domain. We also evaluate what challenges remain to be resolved in order to deploy search and rescue robots in realistic scenarios.