Title:
Machine Learning Driven Emotional Musical Prosody for Human-Robot Interaction

dc.contributor.advisor Weinberg, Gil
dc.contributor.author Savery, Richard
dc.contributor.committeeMember Arthur, Claire
dc.contributor.committeeMember Freeman, Jason
dc.contributor.committeeMember Howard, Ayanna
dc.contributor.department Music
dc.date.accessioned 2022-01-14T16:08:49Z
dc.date.available 2022-01-14T16:08:49Z
dc.date.created 2021-12
dc.date.issued 2021-11-18
dc.date.submitted December 2021
dc.date.updated 2022-01-14T16:08:50Z
dc.description.abstract This dissertation presents a method for non-anthropomorphic human-robot interaction using a newly developed concept entitled Emotional Musical Prosody (EMP). EMP consists of short expressive musical phrases capable of conveying emotions, which can be embedded in robots to accompany mechanical gestures. The main objective of EMP is to improve human engagement with, and trust in robots while avoiding the uncanny valley. We contend that music - one of the most emotionally meaningful human experiences - can serve as an effective medium to support human-robot engagement and trust. EMP allows for the development of personable, emotion-driven agents, capable of giving subtle cues to collaborators while presenting a sense of autonomy. We present four research areas aimed at developing and understanding the potential role of EMP in human-robot interaction. The first research area focuses on collecting and labeling a new EMP dataset from vocalists, and using this dataset to generate prosodic emotional phrases through deep learning methods. Through extensive listening tests, the collected dataset and generated phrases were validated with a high level of accuracy by a large subject pool. The second research effort focuses on understanding the effect of EMP in human-robot interaction with industrial and humanoid robots. Here, significant results were found for improved trust, perceived intelligence, and likeability of EMP enabled robotic arms, but not for humanoid robots. We also found significant results for improved trust in a social robot, as well as perceived intelligence, creativity and likeability in a robotic musician. The third and fourth research areas shift to broader use cases and potential methods to use EMP in HRI. The third research area explores the effect of robotic EMP on different personality types focusing on extraversion and neuroticism. For robots, personality traits offer a unique way to implement custom responses, individualized to human collaborators. We discovered that humans prefer robots with emotional responses based on high extraversion and low neuroticism, with some correlation between the humans collaborator’s own personality traits. The fourth and final research question focused on scaling up EMP to support interaction between groups of robots and humans. Here, we found that improvements in trust and likeability carried across from single robots to groups of industrial arms. Overall, the thesis suggests EMP is useful for improving trust and likeability for industrial, social and robot musicians but not in humanoid robots. The thesis bears future implications for HRI designers, showing the extensive potential of careful audio design, and the wide range of outcomes audio can have on HRI.
dc.description.degree Ph.D.
dc.format.mimetype application/pdf
dc.identifier.uri http://hdl.handle.net/1853/66096
dc.language.iso en_US
dc.publisher Georgia Institute of Technology
dc.subject music technology
dc.subject robotics
dc.subject artificial intelligence
dc.subject sound
dc.subject interaction
dc.title Machine Learning Driven Emotional Musical Prosody for Human-Robot Interaction
dc.type Text
dc.type.genre Dissertation
dspace.entity.type Publication
local.contributor.advisor Weinberg, Gil
local.contributor.corporatename College of Design
local.contributor.corporatename School of Music
local.relation.ispartofseries Doctor of Philosophy with a Major in Music Technology
relation.isAdvisorOfPublication f3feda3b-c805-4675-842e-01a40f8b40a4
relation.isOrgUnitOfPublication c997b6a0-7e87-4a6f-b6fc-932d776ba8d0
relation.isOrgUnitOfPublication 92d2daaa-80f2-4d99-b464-ab7c1125fc55
relation.isSeriesOfPublication d1fd9079-2e93-4803-98c8-8365a28e0761
thesis.degree.level Doctoral
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
SAVERY-DISSERTATION-2021.pdf
Size:
43.35 MB
Format:
Adobe Portable Document Format
Description:
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
LICENSE.txt
Size:
3.87 KB
Format:
Plain Text
Description: