Title:
Model Analysis Tool for Sign Language Video Recognition and other HAR Tasks
Model Analysis Tool for Sign Language Video Recognition and other HAR Tasks
Author(s)
Chadha, Ishan
Advisor(s)
Joseph, Duarette
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
Currently, over 95% of deaf children are born to hearing parents, many of whom do not learn sign language. Lack of communication between deaf children and their parents can lead to language deprivation syndrome (LDS), where the child does not develop important language skills such as short-term memory and basic communication abilities. The CopyCat project aims to address the learning gap that stems from LDS by designing a video game that accepts videos of deaf children signing simple phrases and employs computer vision models to recognize the phrases being signed and verify that they are correct. A key problem for the CopyCat project is selecting features that are representative of different patterns for similar words like “in” and “above,” where features are hand, face, and body pose elements such as fingertips or eyes that help to distinguish these signed words. This thesis focuses on creating a tool that helps to select features that make our American Sign Language recognition system more robust and extends a tool already widely used in the speech recognition community, ELAN. Model boundaries and Gaussian mixture parameters for Hidden Markov models were analyzed through the ELAN tool to show an example of how feature selection can be improved for the CopyCat project. The use of the ELAN tool resulted in a 6% decrease in confusion between the words “in” and “above” for the system, showing the application of this new tool to creating robust human activity recognition systems.
Sponsor
Date Issued
2022-05
Extent
Resource Type
Text
Resource Subtype
Undergraduate Thesis