Person:
Abowd,
Gregory D.
Abowd,
Gregory D.
Permanent Link
Associated Organization(s)
Organizational Unit
ORCID
ArchiveSpace Name Record
Publication Search Results
Now showing
1 - 7 of 7
-
ItemLeveraging Context to Support Automated Food Recognition in Restaurants(Georgia Institute of Technology, 2015-01) Bettadapura, Vinay ; Thomaz, Edison ; Parnam, Aman ; Abowd, Gregory D. ; Essa, IrfanThe pervasiveness of mobile cameras has resulted in a dramatic increase in food photos, which are pictures re- flecting what people eat. In this paper, we study how tak- ing pictures of what we eat in restaurants can be used for the purpose of automating food journaling. We propose to leverage the context of where the picture was taken, with ad- ditional information about the restaurant, available online, coupled with state-of-the-art computer vision techniques to recognize the food being consumed. To this end, we demon- strate image-based recognition of foods eaten in restaurants by training a classifier with images from restaurant’s on- line menu databases. We evaluate the performance of our system in unconstrained, real-world settings with food im- ages taken in 10 restaurants across 5 different types of food (American, Indian, Italian, Mexican and Thai).
-
ItemA Practical Approach for Recognizing Eating Moments With Wrist-Mounted Inertial Sensing(Georgia Institute of Technology, 2015) Thomaz, Edison ; Essa, Irfan ; Abowd, Gregory D.Recognizing when eating activities take place is one of the key challenges in automated food intake monitoring. Despite progress over the years, most proposed approaches have been largely impractical for everyday usage, requiring multiple on-body sensors or specialized devices such as neck collars for swallow detection. In this paper, we describe the implementation and evaluation of an approach for inferring eating moments based on 3-axis accelerometry collected with a popular off-the-shelf smartwatch. Trained with data collected in a semi-controlled laboratory setting with 20 subjects, our system recognized eating moments in two free-living condition studies (7 participants, 1 day; 1 participant, 31 days), with F-scores of 76.1% (66.7% Precision, 88.8% Recall), and 71.3% (65.2% Precision, 78.6% Recall). This work represents a contribution towards the implementation of a practical, automated system for everyday food intake monitoring, with applicability in areas ranging from health research and food journaling.
-
ItemInferring Meal Eating Activities in Real World Settings from Ambient Sounds: A Feasibility Study(Georgia Institute of Technology, 2015) Thomaz, Edison ; Zhang, Cheng ; Essa, Irfan ; Abowd, Gregory D.Dietary self-monitoring has been shown to be an effective method for weight-loss, but it remains an onerous task despite recent advances in food journaling systems. Semi-automated food journaling can reduce the effort of logging, but often requires that eating activities be detected automatically. In this work we describe results from a feasibility study conducted in-the-wild where eating activities were inferred from ambient sounds captured with a wrist-mounted device; twenty participants wore the device during one day for an average of 5 hours while performing normal everyday activities. Our system was able to identify meal eating with an F-score of 79.8% in a person-dependent evaluation, and with 86.6% accuracy in a person-independent evaluation. Our approach is intended to be practical, leveraging off-the-shelf devices with audio sensing capabilities in contrast to systems for automated dietary assessment based on specialized sensors.
-
ItemPredicting Daily Activities From Egocentric Images Using Deep Learning(Georgia Institute of Technology, 2015) Castro, Daniel ; Hickson, Steven ; Bettadapura, Vinay ; Thomaz, Edison ; Abowd, Gregory D. ; Christensen, Henrik I. ; Essa, IrfanWe present a method to analyze images taken from a passive egocentric wearable camera along with the contextual information, such as time and day of week, to learn and predict everyday activities of an individual. We collected a dataset of 40,103 egocentric images over a 6 month period with 19 activity classes and demonstrate the benefit of state-of-the-art deep learning techniques for learning and predicting daily activities. Classification is conducted using a Convolutional Neural Network (CNN) with a classification method we introduce called a late fusion ensemble. This late fusion ensemble incorporates relevant contextual information and increases our classification accuracy. Our technique achieves an overall accuracy of 83.07% in predicting a person's activity across the 19 activity classes. We also demonstrate some promising results from two additional users by fine-tuning the classifier with one day of training data.
-
ItemFeasibility of Identifying Eating Moments from First-Person Images Leveraging Human Computation(Georgia Institute of Technology, 2013-11) Thomaz, Edison ; Parnami, Aman ; Essa, Irfan ; Abowd, Gregory D.There is widespread agreement in the medical research community that more effective mechanisms for dietary assessment and food journaling are needed to fight back against obesity and other nutrition-related diseases. However, it is presently not possible to automatically capture and objectively assess an individual’s eating behavior. Currently used dietary assessment and journaling approaches have several limitations; they pose a significant burden on individuals and are often not detailed or accurate enough. In this paper, we describe an approach where we leverage human computation to identify eating moments in first-person point-of-view images taken with wearable cameras. Recognizing eating moments is a key first step both in terms of automating dietary assessment and building systems that help individuals reflect on their diet. In a feasibility study with 5 participants over 3 days, where 17,575 images were collected in total, our method was able to recognize eating moments with 89.68% accuracy.
-
ItemDecoding Children’s Social Behavior(Georgia Institute of Technology, 2013-06) Rehg, James M. ; Abowd, Gregory D. ; Rozga, Agata ; Romero, Mario ; Clements, Mark A. ; Sclaroff, Stan ; Essa, Irfan ; Ousley, Opal Y. ; Li, Yin ; Kim, Chanho ; Rao, Hrishikesh ; Kim, Jonathan C. ; Presti, Liliana Lo ; Zhang, Jianming ; Lantsman, Denis ; Bidwell, Jonathan ; Ye, ZhefanWe introduce a new problem domain for activity recognition: the analysis of children’s social and communicative behaviors based on video and audio data. We specifically target interactions between children aged 1–2 years and an adult. Such interactions arise naturally in the diagnosis and treatment of developmental disorders such as autism. We introduce a new publicly-available dataset containing over 160 sessions of a 3–5 minute child-adult interaction. In each session, the adult examiner followed a semistructured play interaction protocol which was designed to elicit a broad range of social behaviors. We identify the key technical challenges in analyzing these behaviors, and describe methods for decoding the interactions. We present experimental results that demonstrate the potential of the dataset to drive interesting research questions, and show preliminary results for multi-modal activity recognition.
-
ItemRecognizing Water-Based Activities in the Home Through Infrastructure-Mediated Sensing(Georgia Institute of Technology, 2012-09) Thomaz, Edison ; Bettadapura, Vinay ; Reyes, Gabriel ; Sandesh, Megha ; Schindler, Grant ; Plötz, Thomas ; Abowd, Gregory D. ; Essa, IrfanActivity recognition in the home has been long recognized as the foundation for many desirable applications in fields such as home automation, sustainability, and healthcare. However, building a practical home activity monitoring system remains a challenge. Striking a balance between cost, privacy, ease of installation and scalability continues to be an elusive goal. In this paper, we explore infrastructure-mediated sensing combined with a vector space model learning approach as the basis of an activity recognition system for the home. We examine the performance of our single-sensor water-based system in recognizing eleven high-level activities in the kitchen and bathroom, such as cooking and shaving. Results from two studies show that our system can estimate activities with overall accuracy of 82.69% for one individual and 70.11% for a group of 23 participants. As far as we know, our work is the first to employ infrastructure-mediated sensing for inferring high-level human activities in a home setting.