Organizational Unit:

Rehabilitation Engineering Research Center on Technologies to Support Aging-in-Place for People with Long-Term Disabilities

Permanent Link

https://hdl.handle.net/1853/70667

Full item page

Publication Search Results

Now showing 1 - 10 of 13

Multidimensional Capacitive Sensing for Robot-Assisted Dressing and Bathing

(Georgia Institute of Technology, 2019-05-24) Erickson, Zackory ; Clever, Henry M. ; Gangaram, Vamsee ; Turk, Greg ; Liu, C. Karen ; Kemp, Charles C.

Robotic assistance presents an opportunity to beneﬁt the lives of many people with physical disabilities, yet accurately sensing the human body and tracking human motion remain difﬁcult for robots. We present a multidimensional capacitive sensing technique that estimates the local pose of a human limb in real time. A key beneﬁt of this sensing method is that it can sense the limb through opaque materials, including fabrics and wet cloth. Our method uses a multielectrode capacitive sensor mounted to a robot’s end effector. A neural network model estimates the position of the closest point on a person’s limb and the orientation of the limb’s central axis relative to the sensor’s frame of reference. These pose estimates enable the robot to move its end effector with respect to the limb using feedback control. We demonstrate that a PR2 robot can use this approach with a custom six electrode capacitive sensor to assist with two activities of daily living— dressing and bathing. The robot pulled the sleeve of a hospital gown onto able-bodied participants’ right arms, while tracking human motion. When assisting with bathing, the robot moved a soft wet washcloth to follow the contours of able-bodied participants’ limbs, cleaning their surfaces. Overall, we found that multidimensional capacitive sensing presents a promising approach for robots to sense and track the human body during assistive tasks that require physical human-robot interaction.
In-home and remote use of robotic body surrogates by people with profound motor deficits

(PLOS One, 2019-03) Grice, Phillip M. ; Kemp, Charles C.

By controlling robots comparable to the human body, people with profound motor deficits could potentially perform a variety of physical tasks for themselves, improving their quality of life. The extent to which this is achievable has been unclear due to the lack of suitable interfaces by which to control robotic body surrogates and a dearth of studies involving substantial numbers of people with profound motor deficits. We developed a novel, web-based augmented reality interface that enables people with profound motor deficits to remotely control a PR2 mobile manipulator from Willow Garage, which is a human-scale, wheeled robot with two arms. We then conducted two studies to investigate the use of robotic body surrogates. In the first study, 15 novice users with profound motor deficits from across the United States controlled a PR2 in Atlanta, GA to perform a modified Action Research Arm Test (ARAT) and a simulated self-care task. Participants achieved clinically meaningful improvements on the ARAT and 12 of 15 participants (80%) successfully completed the simulated self-care task. Participants agreed that the robotic system was easy to use, was useful, and would provide a meaningful improvement in their lives. In the second study, one expert user with profound motor deficits had free use of a PR2 in his home for seven days. He performed a variety of self-care and household tasks, and also used the robot in novel ways. Taking both studies together, our results suggest that people with profound motor deficits can improve their quality of life using robotic body surrogates, and that they can gain benefit with only low-level robot autonomy and without invasive interfaces. However, methods to reduce the rate of errors and increase operational speed merit further
3D Human Pose Estimation on a Configurable Bed from a Pressure Image

( 2018) Clever, Henry M. ; Kapusta, Ariel ; Park, Daehyung ; Erickson, Zackory ; Chitalia, Yash ; Kemp, Charles C.

Robots have the potential to assist people in bed, such as in healthcare settings, yet bedding materials like sheets and blankets can make observation of the human body difficult for robots. A pressure-sensing mat on a bed can provide pressure images that are relatively insensitive to bedding materials. However, prior work on estimating human pose from pressure images has been restricted to 2D pose estimates and flat beds. In this work, we present two convolutional neural networks to estimate the 3D joint positions of a person in a configurable bed from a single pressure image. The first network directly outputs 3D joint positions, while the second outputs a kinematic model that includes estimated joint angles and limb lengths. We evaluated our networks on data from 17 human participants with two bed configurations: supine and seated. Our networks achieved a mean joint position error of 77 mm when tested with data from people outside the training set, outperforming several baselines. We also present a simple mechanical model that provides insight into ambiguity associated with limbs raised off of the pressure mat, and demonstrate that Monte Carlo dropout can be used to estimate pose confidence in these situations. Finally, we provide a demonstration in which a mobile manipulator uses our network’s estimated kinematic model to reach a location on a person’s body in spite of the person being seated in a bed and covered by a blanket.
Haptic Simulation for Robot-Assisted Dressing

(Georgia Institute of Technology, 2017) Yu, Wenhao ; Kapusta, Ariel ; Tan, Jie ; Kemp, Charles C. ; Turk, Greg ; Liu, C. Karen

There is a considerable need for assistive dressing among people with disabilities, and robots have the potential to fulfill this need. However, training such a robot would require extensive trials in order to learn the skills of assistive dressing. Such training would be time-consuming and require considerable effort to recruit participants and conduct trials. In addition, for some cases that might cause injury to the person being dressed, it is impractical and unethical to perform such trials. In this work, we focus on a representative dressing task of pulling the sleeve of a hospital gown onto a person’s arm. We present a system that learns a haptic classifier for the outcome of the task given few (2-3) real-world trials with one person. Our system first optimizes the parameters of a physics simulator using real-world data. Using the optimized simulator, the system then simulates more haptic sensory data with noise models that account for randomness in the experiment. We then train hidden Markov Models (HMMs) on the simulated haptic data. The trained HMMs can then be used to classify and predict the outcome of the assistive dressing task based on haptic signals measured by a real robot’s end effector. This system achieves 92.83% accuracy in classifying the outcome of the robot-assisted dressing task with people not included in simulation optimization. We compare our classifiers to those trained on real-world data. We show that the classifiers from our system can categorize the dressing task outcomes more accurately than classifiers trained on ten times more real data.
Data-Driven Haptic Perception for Robot-Assisted Dressing

(Georgia Institute of Technology, 2016-08) Kapusta, Ariel ; Yu, Wenhao ; Bhattacharjee, Tapomayukh ; Liu, C. Karen ; Turk, Greg ; Kemp, Charles C.

Dressing is an important activity of daily living (ADL) with which many people require assistance due to impairments. Robots have the potential to provide dressing assistance, but physical interactions between clothing and the human body can be complex and difficult to visually observe. We provide evidence that data-driven haptic perception can be used to infer relationships between clothing and the human body during robot-assisted dressing. We conducted a carefully controlled experiment with 12 human participants during which a robot pulled a hospital gown along the length of each person’s forearm 30 times. This representative task resulted in one of the following three outcomes: the hand missed the opening to the sleeve; the hand or forearm became caught on the sleeve; or the full forearm successfully entered the sleeve. We found that hidden Markov models (HMMs) using only forces measured at the robot’s end effector classified these outcomes with high accuracy. The HMMs’ performance generalized well to participants (98.61% accuracy) and velocities (98.61% accuracy) outside of the training data. They also performed well when we limited the force applied by the robot (95.8% accuracy with a 2N threshold), and could predict the outcome early in the process. Despite the lightweight hospital gown, HMMs that used forces in the direction of gravity substantially outperformed those that did not. The best performing HMMs used forces in the direction of motion and the direction of gravity.
Collaboration Between a Robotic Bed and a Mobile Manipulator May Improve Physical Assistance for People with Disabilities

(Georgia Institute of Technology, 2016-08) Kapusta, Ariel ; Chitalia, Yash ; Park, Daehyung ; Kemp, Charles C.

We present a robotic system designed to provide physical assistance to a person in bed. The system consists of a robotic bed (Autobed) and a mobile manipulator (PR2) that work together. The 3 degree-of-freedom (DoF) robotic bed moves the person’s body and uses a pressure sensing mat to estimate the body’s position. The mobile manipulator positions itself with respect to the bed and compliantly moves a lightweight object with one of its 7-DoF arms. The system optimizes its motions with respect to a task model and a model of the human’s body. The user provides high-level supervision to the system via a web-based interface. We first evaluated the ability of the robotic bed to estimate the location of the head of a person in a supine configuration via a study with 7 able-bodied participants. This estimation was robust to bedding, including a pillow under the person’s head. We then evaluated the ability of the full system to autonomously reach task-relevant poses on a medical mannequin placed in a supine position on the bed. We found that the robotic bed’s motion and perception each improved the overall system’s performance. Our results suggest that this type of multi-robot system could more effectively bring objects to desired locations with respect to the user’s body than a mobile manipulator working alone. This may in turn lead to improved physical assistance for people with disabilities at home and in healthcare facilities, since many assistive tasks involve an object being moved with respect to a person’s body.
Optimization of Robot Configurations for Assistive Tasks

(Georgia Institute of Technology, 2016) Kapusta, Ariel ; Kemp, Charles C.

Robots can provide assistance with activities of daily living (ADLs) to humans with motor impairments. Specialized robots, such as desktop robotic feeding systems, have been successful for specific assistive tasks when placed in fixed and designated positions with respect to the user. General-purpose mobile manipulators could act as a more versatile form of assistive technology, able to perform many tasks, but selecting a configuration for the robots from which to perform a task can be challenging due to the high number of degrees of freedom of the robots and the complexity of the tasks. As with the specialized, fixed robots, once in a good configuration, another system or the user can provide the fine control to perform the details of the task. In this short paper, we present Task-centric Optimization of robot Configurations (TOC), a method for selecting configurations for a PR2 and a robotic bed to allow the PR2 to provide effective assistance with ADLs. TOC builds upon previous work, Task-centric initial Configuration Selection (TCS), addressing some of the limitations of TCS. Notable alterations are selecting configurations from the continuous configuration space using a Covariance Matrix Adaptation Evolution Strategy (CMA-ES) optimization, introducing a joint-limit-weighted manipulability term, and changing the framework to move all optimization offline and using function approximation at run-time. To evaluate TOC, we created models of 13 activities of daily living (ADLs) and compared TOC’s and TCS’s performance with these 13 assistive tasks in a computer simulation of a PR2, a robotic bed, and a model of a human body. TOC performed as well or better than TCS in most of our tests against state estimation error. We also implemented TOC on a real PR2 and a real robotic bed and found that from the TOC-selected configuration the PR2 could reach all task-relevant goals on a mannequin on the bed.
Autobed: Open Hardware for Accessible Web-based Control of an Electric Bed

(Georgia Institute of Technology, 2016) Grice, Phillip M. ; Chitalia, Yash ; Rich, Megan ; Clever, Henry M. ; Kemp, Charles C.

Individuals with severe motor impairments often have difficulty operating the standard controls of electric beds and so require a caregiver to adjust their position for utility, comfort, or to prevent pressure ulcers. Assistive human-computer interaction devices allow many such individuals to operate a computer and web browser. Here, we present the Autobed, a Wi-Fi-connected device that enables control of an Invacare Full-Electric Homecare Bed, a Medicare-approved device in the US, from any modern web browser, without modification of existing hardware. We detail the design and operation of the Autobed. We also examine its usage by one individual with severe motor impairments and his primary caregiver in their own home, including usage logs from a period of 102 days and detailed questionnaires. Finally, we make the entire system, including hardware design and components, software, and build instructions, available under permissive open-source licenses.
A Model that Predicts the Material Recognition Performance of Thermal Tactile Sensing

( 2016) Bhattacharjee, Tapomayukh ; Bai, Haoping ; Chen, Haofeng ; Kemp, Charles C.

Tactile sensing can enable a robot to infer properties of its surroundings, such as the material of an object. Heat transfer based sensing can be used for material recognition due to differences in the thermal properties of materials. While datadriven methods have shown promise for this recognition problem, many factors can influence performance, including sensor noise, the initial temperatures of the sensor and the object, the thermal effusivities of the materials, and the duration of contact. We present a physics-based mathematical model that predicts material recognition performance given these factors. Our model uses semi-infinite solids and a statistical method to calculate an F1 score for the binary material recognition. We evaluated our method using simulated contact with 69 materials and data collected by a real robot with 12 materials. Our model predicted the material recognition performance of support vector machine (SVM) with 96% accuracy for the simulated data, with 92% accuracy for real-world data with constant initial sensor temperatures, and with 91% accuracy for real-world data with varied initial sensor temperatures. Using our model, we also provide insight into the roles of various factors on recognition performance, such as the temperature difference between the sensor and the object. Overall, our results suggest that our model could be used to help design better thermal sensors for robots and enable robots to use them more effectively.
A Robotic System for Reaching in Dense Clutter that Integrates Model Predictive Control, Learning, Haptic Mapping, and Planning

(Georgia Institute of Technology, 2014-09) Bhattacharjee, Tapomayukh ; Grice, Phillip M. ; Kapusta, Ariel ; Killpack, Marc D. ; Park, Daehyung ; Kemp, Charles C.

We present a system that enables a robot to reach locations in dense clutter using only haptic sensing. Our system integrates model predictive control [1], learned initial conditions [2], tactile recognition of object types [3], haptic mapping, and geometric planning to efficiently reach locations using whole- arm tactile sensing [4]. We motivate our work, present a system architecture, summarize each component of the system, and present results from our evaluation of the system reaching to target locations in dense artificial foliage.