Learning Motion Policies for Dexterous Manipulation with Geometric Fabrics

Dexterous manipulation using multi-fingered robotic hands has been a long-standing challenge in robotics due to numerous factors, such as complex dynamics, high-dimensional action space, and difficulties involved in designing bespoken controllers. Recent advances suggest that learning-based techniques could hold the key to equipping robots with complex manipulation skills. However, existing learning methods either limit themselves to highly constrained problems and smaller models to enable impressive sample efficiency, or require substantial compute and data resources to train over-parameterized models that can solve more complex tasks. This dissertation makes contributions that combine the best of both high-capacity models and sample-efficient structures in order to help robots \textit{efficiently} learn and \textit{reliably} execute dexterous manipulation skills. First, we develop a new class of behavioral dynamical systems, we call {\em Geometric fabrics}, that generalizes classical mechanical systems to gain expressivity in encoding skills while retaining desirable theoretic properties. Second, to circumvent the need for painstaking human effort required to manually design these models, we introduce {\em Neural Geometric Fabrics} (NGFs) that can be used to sample-efficiently learn manipulation skills by leveraging structural inductive biases imposed by Geometric Fabrics. Third, we contribute a hierarchical learning framework that can leverage NGFs to encode skills shared across dexterous manipulation tasks and enable effective generalization and efficient skill transfer to novel tasks. We demonstrate the effectiveness of our methods in both simulation and on physical hardware. Results from comprehensive comparative and ablative experiments show our methods consistently outperform state-of-the-art baselines like Riemannian Motion Policies (RMPs), Neural Dynamics Policies (NDPs) and other commonly used policy architectures in terms of sample efficiency, task performance, and generalizability.

Date

2023-07-24

Resource Type

Text

Resource Subtype

Dissertation

Full item page