Musical abstractions for multi-robot coordination

Thumbnail Image
Santos Fernandez, Maria Teresa
Egerstedt, Magnus B.
Associated Organization(s)
Supplementary to
This work presents a new approach to human-swarm interactions, a discipline which addresses the problem of how a human operator can influence the behavior of large groups of robots, providing high-level information understandable by the team. While there exist potential advantages of introducing a human in the control loop of a robot swarm, how the human must be incorporated is not a simple problem. For the intervention of a human operator to be favorable to the performance of the team, the means and form of the information between the human and the robot swarm must be adequately defined: we need to design which device will be provided to the operator to interact with the swarm and how the information will be shaped so that both the human and the robot team understand it. Coordination of multi-robot systems involves the generation of involved motion patterns for the individual agents that result in an overall organized movement. We introduce in this thesis a new human-swarm interaction modality based on music theory, a discipline studied for centuries and capable of creating complex sound structures. In particular, we have focused on understanding how we can apply rules and structures from music theory to an operator's input so that each command both specifies the goal location to be visited and the geometry to be adopted by the swarm. We interpret the sequence of locations to be visited by the swarm as a musical melody, identifying each note with a certain location in the robots' workspace. Once the objective path is defined in the form of a melody, we can apply rules from harmony, a discipline of music theory, to create chords that harmonize the input melody. The interest in using these chords lies fundamentally in that they are structured combinations of pitches, heard simultaneously. These inherent structures will be used to determine the geometry that should be displayed by the team. The developed multi-robot control is applied to a team of differential drive mobile robots through an electronic piano.
Date Issued
Resource Type
Resource Subtype
Rights Statement
Rights URI