Title:
Unifying Semantic and Physical Intelligence for Generalist Humanoid Robots
Unifying Semantic and Physical Intelligence for Generalist Humanoid Robots
Author(s)
Shi, Guanya
Advisor(s)
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
Humanoid robots offer two unparalleled advantages in general-purpose embodied intelligence. First, humanoids are built as generalist robots that can potentially do all the tasks humans can do in complex environments. Second, the embodiment alignment between humans and humanoids allows for the seamless integration of human cognitive skills with versatile humanoid capabilities. To build generalist humanoids, there are three critical aspects of intelligence: (1) Semantic intelligence (how the robot understands the world and reasons); (2) Physical/Motion intelligence (locomotion and manipulation skills); and (3) Mechanical/Hardware intelligence (how the robot actuates and senses). In this talk, I will present some recent works (H2O, OmniH2O, WoCoCo, ABS) that aim to unify semantic and physical intelligence for humanoid robots. In particular, H2O and OmniH2O provide a universal and dexterous interface that enables diverse human control (e.g., VR, RGB) and autonomy (e.g., using imitation learning or VLMs) methods for humanoids, WoCoCo provides an efficient framework for loco-manipulation skill learning without motion priors, and ABS provides safety guarantees for agile vision-based locomotion control. Finally, I will briefly discuss how to combine learning-based control approaches and traditional model-based control approaches to get the best of two worlds.
Sponsor
Date Issued
2024-09-04
Extent
61:16 minutes
Resource Type
Moving Image
Resource Subtype
Lecture
Rights Statement
Unless otherwise noted, all materials are protected under U.S. Copyright Law and all rights are reserved