Organizational Unit:

School of Interactive Computing

Permanent Link

https://hdl.handle.net/1853/70783

Parent Organization

Organizational Unit

College of Computing

ArchiveSpace Name Record

https://finding-aids.library.gatech.edu/agents/corporate_entities/1113

Full item page

Publication Search Results

Now showing 1 - 10 of 11

Leveraging Low-Dimensional Geometry for Search and Ranking

(Georgia Institute of Technology, 2023-12-06) Fenu, Stefano

There is a substantial body of work on search and ranking in computer science, but less attention has been paid to the question of how to learn geometric data representations that are amenable to search and ranking tasks. Index-based datastructures for search are commonplace, but these discard structural features of the data, often have large memory profiles, and scale poorly with data dimension. Geometric search techniques do exist, but few analogous search datastructures or preprocessing algorithms exist that leverage spatial structure in data to increase search performance. The aim of the research detailed here is to show that leveraging low-dimensional geometry can improve the performance of search and ranking over index-only methods, and that there are dimensionality reduction techniques that can make spatial search algorithms more effective without any additional memory overhead. This work accomplishes these aims by developing methods for: Learning low-dimensional coordinate embeddings explicitly for the purpose of search and ranking; and actively querying and constructing searchable embeddings to minimize user-labeling costs. This dissertation will further provide scalable versions of these algorithms and demonstrate their effectiveness across a broad range of problem domains including visual, text, and educational data. These performance improvements will allow human-in-the-loop search of larger datasets and enable new applications in preference search and ranking.
WIDGETs: Wireless Interactive Devices for Gauging and Evaluating Temperament for Service and Working Dogs

(Georgia Institute of Technology, 2021-08-27) Byrne, Ceara Ann

Both service and working dogs are significantly beneficial to society; however, a substantial number of dogs are released from time consuming and expensive training programs when their behavior is unsuitable for the role they are training to enter. Early predictions of which dogs will succeed in which programs would not only increase the availability of dogs, but also save time, training resources, and funding. This research explores whether aspects of canine temperament can be detected from interactions with sensors and develops machine learning models that use sensor data to predict the success of service and working dogs-in-training. In this dissertation, we show the potential of instrumented ball and tug toys for predicting, with 87.5% accuracy, the success (or failure) of dogs entering advanced training in the Canine Companions for Independence (CCI) Program. We also find that the toys can predict whether a working dog-in-training at Auburn University's Canine Performance Sciences center (CPS) is suitable for advanced detection training with 83% accuracy. Lastly, we provide an exploratory analysis of the relationship between independent interaction features and (1) a canine's suitability outcomes in service dog programs, (2) a canine's suitability outcomes in working dog programs, (3) a canine's reasons for being released from a working dog program, and (4) the differences between successful service and working dogs.
Wearable vibrotactile stimulation: How passive stimulation can train and rehabilitate

(Georgia Institute of Technology, 2019-03-27) Seim, Caitlyn

Haptic feedback from wearable devices is primarily used for alerts and virtual reality; however, wearable computing provides unique advantages in haptic interaction. Wearable devices can now provide tactile stimulation for extended periods of time and in the background of other tasks. Since repetition is key to practice, learning, and rehabilitation, stimulation for extended periods of time may enable intensive haptic training or mobile stimulation therapy. Training and rehabilitation require time, dedication and sometimes exertion. Stimulation in the background of other tasks can allow passive training and therapy, without requiring movement or attentional focus from the user. My work takes advantage of these unique considerations to develop wearable computing solutions to help address real-world applications, while informing what is possible using passive tactile stimulation and enabling others to apply these methods in the future. Ambient stimuli can enable passive learning: training while users are focused on other tasks. Most research on this topic has used audio or visual stimuli, and few have explored the use of haptic stimuli for passive learning. In this dissertation, I present evidence that wearable vibrotactile stimulation can help train a variety of skills including those involving rhythm, simultaneous actions, and various body parts. This work also provides essential guidelines on how to construct wearable computing systems that apply this technique to practical problems. Results suggest that this passive training method may allow users to recall dozens of motor actions with little practice and learn challenging skills with less difficulty. Wearable vibrotactile stimulation may also help re-train sensorimotor functions, for example, diminished arm function after a stroke. Stroke can lead to chronic physical disability in the limbs. In fact, stroke is the leading cause of adult disability in the US. Preliminary evidence suggests that peripheral tactile stimulation may facilitate limb rehabilitation, but current methods for applying this technique are limited to laboratory settings. Currently, there is no device available to administer and study therapeutic tactile stimulation for extended periods of time or outside the clinic environment. I present a low-cost, wireless wearable device to provide tactile stimulation therapy and an initial randomized controlled trial in stroke survivors over 8 weeks. Results suggest that wearable vibrotactile stimulation may also be a powerful tool to reduce disability after a stroke.
Lateral positioning of a monocular head-worn display

(Georgia Institute of Technology, 2017-08-25) Haynes, Malcolm Gibran

Head-worn displays (HWDs) such as Google Glass are becoming more common. However, the optimal location for the display of such devices is an open question. Existing devices have displays located above, below, and in line with the primary position of gaze. In addition to vertical displacement, an HWD can be displaced laterally. In fact, several studies point to potential advantages of lateral displacement. Yet, little research has examined the effects of laterally displacing a HWD. We evaluate lateral displacement of a monocular HWD across a representative set of activities for which a monocular HWD may be used. The selected activities include situations where interacting with the HWD is the primary or sole task and situations where focus is shifted back and forth between the HWD and the real world. Specifically, we conduct three studies. The first two examine effects of lateral displacement on a long duration reading task. The third considers effects of lateral displacement on a modification of a common industrial task, order picking. The general thesis proposed is that displacing a monocular head-worn display more than 10 degrees laterally negatively affects subjective perception of visual comfort. Across multiple studies, we found that a display offset at 0 degrees, 10 degrees or 20 degrees was rated more comfortable than a display offset at 30 degrees. Post hoc analysis of other measures such as preference, eye strain, and workload result in similar findings. Although there are also differences between measurements made at 0 degrees and 10 degrees compared to 20 degrees, they are less pronounced. Interestingly, there was no significant difference between conditions for most task performance measures such as speed or accuracy. Given the relative consistency of results across multiple studies and participant comments, we suggest that small field of view (FOV) displays should be mounted at lateral displacement angles of 20 degrees and less for sustained use. However, users may prefer offsets between 0 degrees and 10 degrees, with 10 degrees sometimes being more preferred because it keeps the display out of the way.
Facilitating American Sign Language learning for hearing parents of deaf children via mobile devices

(Georgia Institute of Technology, 2013-04-02) Xu, Kimberly A.

In the United States, between 90 and 95% of deaf children are born to hearing parents. In most circumstances, the birth of a deaf child is the first experience these parents have with American Sign Language (ASL) and the Deaf community. Parents learn ASL as a second language to provide their children with language models and to be able to communicate with their children more effectively, but they face significant challenges. To address these challenges, I have developed a mobile learning application, SMARTSign, to help parents of deaf children learn ASL vocabulary. I hypothesize that providing a method for parents to learn and practice ASL words associated with popular children's stories on their mobile phones would help improve their ASL vocabulary and abilities more than if words were grouped by theme. I posit that parents who learn vocabulary associated with children's stories will use the application more, which will lead to more exposure to ASL and more learned vocabulary. My dissertation consists of three studies. First I show that novices are able to reproduce signs presented on mobile devices with high accuracy regardless of source video resolution. Next, I interview hearing parents with deaf children to discover the difficulties they have with current methods for learning ASL. When asked which methods of presenting signs they preferred, participants were most interested in learning vocabulary associated with children's stories. Finally, I deploy SMARTSign to parents for four weeks. Participants learning story vocabulary used the application more often and had higher sign recognition scores than participants who learned vocabulary based on word types. The condition did not affect participants' ability to produce the signed vocabulary.
On-the-go text entry: evaluating and improving mobile text input on mini-qwerty keyboards

(Georgia Institute of Technology, 2012-11-13) Clawson, James

To date, hundreds of millions of mini-QWERTY keyboard equipped devices (miniaturized versions of a full desktop keyboard) have been sold. Accordingly, a large percentage of text messages originate from fixed-key, mini-QWERTY keyboard enabled mobile phones. Over a series of three longitudinal studies I quantify how quickly and accurately individuals can input text on mini-QWERTY keyboards. I evaluate performance in ideal laboratory conditions as well as in a variety of mobile contexts. My first study establishes baseline performance measures; my second study investigates the impact of limited visibility on text input performance; and my third study investigates the impact of mobility (sitting, standing, and walking) on text input performance. After approximately five hours of practice, participants achieved expertise typing almost 60 words per minute at almost 95% accuracy. Upon completion of these studies, I examine the types of errors that people make when typing on mini-QWERTY keyboards. Having discovered a common pattern in errors, I develop and refine an algorithm to automatically detect and correct errors in mini-QWERTY keyboard enabled text input. I both validate the algorithm through the analysis of pre-recorded typing data and then empirically evaluate the impacts of automatic error correction on live mini-QWERTY keyboard text input. Validating the algorithm over various datasets, I demonstrate the potential to correct approximately 25% of the total errors and correct up to 3% of the total keystrokes. Evaluating automatic error detection and correction on live typing results in successfully correcting 61% of the targeted errors committed by participants while increasing typing rates by almost two words per minute without introducing noticeable distraction.
Improving the efficacy of automated sign language practice tools

(Georgia Institute of Technology, 2010-07-07) Brashear, Helene Margaret

The CopyCat project is an interdisciplinary effort to create a set of computer-aided language learning tools for deaf children. The CopyCat games allow children to interact with characters using American Sign Language (ASL). Through Wizard of Oz pilot studies we have developed a set of games, shown their efficacy in improving young deaf children's language and memory skills, and collected a large corpus of signing examples. Our previous implementation of the automatic CopyCat games uses automatic sign language recognition and verification in the infrastructure of a memory repetition and phrase verification task. The goal of my research is to expand the automatic sign language system to transition the CopyCat games to include the flexibility of a dialogue system. I have created a labeling ontology from analysis of the CopyCat signing corpus, and I have used the ontology to describe the contents of the CopyCat data set. This ontology was used to change and improve the automatic sign language recognition system and to add flexibility to language use in the automatic game.
Child's play: activity recognition for monitoring children's developmental progress with augmented toys

(Georgia Institute of Technology, 2010-05-20) Westeyn, Tracy Lee

The way in which infants play with objects can be indicative of their developmental progress and may serve as an early indicator for developmental delays. However, the observation of children interacting with toys for the purpose of quantitative analysis can be a difficult task. To better quantify how play may serve as an early indicator, researchers have conducted retrospective studies examining the differences in object play behaviors among infants. However, such studies require that researchers repeatedly inspect videos of play often at speeds much slower than real-time to indicate points of interest. The research presented in this dissertation examines whether a combination of sensors embedded within toys and automatic pattern recognition of object play behaviors can help expedite this process. For my dissertation, I developed the Child'sPlay system which uses augmented toys and statistical models to automatically provide quantitative measures of object play interactions, as well as, provide the PlayView interface to view annotated play data for later analysis. In this dissertation, I examine the hypothesis that sensors embedded in objects can provide sufficient data for automatic recognition of certain exploratory, relational, and functional object play behaviors in semi-naturalistic environments and that a continuum of recognition accuracy exists which allows automatic indexing to be useful for retrospective review. I designed several augmented toys and used them to collect object play data from more than fifty play sessions. I conducted pattern recognition experiments over this data to produce statistical models that automatically classify children's object play behaviors. In addition, I conducted a user study with twenty participants to determine if annotations automatically generated from these models help improve performance in retrospective review tasks. My results indicate that these statistical models increase user performance and decrease perceived effort when combined with the PlayView interface during retrospective review. The presence of high quality annotations are preferred by users and promotes an increase in the effective retrieval rates of object play behaviors.
Segmental discriminative analysis for American Sign Language recognition and verification

(Georgia Institute of Technology, 2010-04-06) Yin, Pei

This dissertation presents segmental discriminative analysis techniques for American Sign Language (ASL) recognition and verification. ASL recognition is a sequence classification problem. One of the most successful techniques for recognizing ASL is the hidden Markov model (HMM) and its variants. This dissertation addresses two problems in sign recognition by HMMs. The first is discriminative feature selection for temporally-correlated data. Temporal correlation in sequences often causes difficulties in feature selection. To mitigate this problem, this dissertation proposes segmentally-boosted HMMs (SBHMMs), which construct the state-optimized features in a segmental and discriminative manner. The second problem is the decomposition of ASL signs for efficient and accurate recognition. For this problem, this dissertation proposes discriminative state-space clustering (DISC), a data-driven method of automatically extracting sub-sign units by state-tying from the results of feature selection. DISC and SBHMMs can jointly search for discriminative feature sets and representation units of ASL recognition. ASL verification, which determines whether an input signing sequence matches a pre-defined phrase, shares similarities with ASL recognition, but it has more prior knowledge and a higher expectation of accuracy. Therefore, ASL verification requires additional discriminative analysis not only in utilizing prior knowledge but also in actively selecting a set of phrases that have a high expectation of verification accuracy in the service of improving the experience of users. This dissertation describes ASL verification using CopyCat, an ASL game that helps deaf children acquire language abilities at an early age. It then presents the "probe" technique which automatically searches for an optimal threshold for verification using prior knowledge and BIG, a bi-gram error-ranking predictor which efficiently selects/creates phrases that, based on the previous performance of existing verification systems, should have high verification accuracy. This work demonstrates the utility of the described technologies in a series of experiments. SBHMMs are validated in ASL phrase recognition as well as various other applications such as lip reading and speech recognition. DISC-SBHMMs consistently produce fewer errors than traditional HMMs and SBHMMs in recognizing ASL phrases using an instrumented glove. Probe achieves verification efficacy comparable to the optimum obtained from manually exhaustive search. Finally, when verifying phrases in CopyCat, BIG predicts which CopyCat phrases, even unseen in training, will have the best verification accuracy with results comparable to much more computationally intensive methods.
Facilitating communication for deaf individuals with mobile technologies

(Georgia Institute of Technology, 2010-03-31) Summet, Valerie Henderson

Communication between deaf individuals and hearing individuals can be very difficult. For people who are born deaf, English is often a second language with the first language being American Sign Language (ASL). Very few hearing people in the United States sign or are aware of Deafness, Deaf culture, or how to appropriately communicate with people with hearing loss. In this thesis, I concentrate on the role that mobile technologies can play in ameliorating some of these issues. In formative work with Deaf teenagers in the metro-Atlanta area, I investigate the role that communication technologies play in the lives of many Deaf individuals and examine how these devices have effected their communication patterns and social circles. Specifically, the teens identified problems communicating with hearing individuals such as close friends and family in face-to-face situations. Having identified sign language use at home as one of the earliest interventions for Deaf children, I investigated the use of mobile phones for learning survival-level ASL. I created a prototype software application which presented short ASL lessons via either a mobile phone or desktop web-browser. The software presented the lessons via one of two different scheduling methods designed to take advantage of the spacing effect during learning. I designed and conducted a study of forty individuals with no prior ASL knowledge which compared the effects of both scheduling algorithm and platform. My results show that individuals who used a mobile phone platform and received a group of lessons at one time performed better on post-test receptive and generative ASL metrics than did participants in the three other conditions.