Toyota Research Institute makes generative AI breakthrough to teach robots new behaviors; Diffusion Policy
Toyota Research Institute (TRI) announced a breakthrough generative AI approach based on Diffusion Policy to teach robots new, dexterous skills quickly and confidently . This advancement significantly improves robot utility and is a step towards building “Large Behavior Models (LBMs)” for robots, analogous to the Large Language Models (LLMs) that have recently revolutionized conversational AI.
Our research in robotics is aimed at amplifying people rather than replacing them. This new teaching technique is both very efficient and produces very high performing behaviors, enabling robots to much more effectively amplify people in many ways.—Gill Pratt, CEO of TRI and Chief Scientist for Toyota Motor Corporation
Previous techniques to teach robots new behaviors were slow, inconsistent, inefficient, and often limited to narrowly defined tasks performed in highly constrained environments. Roboticists needed to spend many hours writing sophisticated code and/or using numerous trial and error cycles to program behaviors.
TRI has already taught robots more than 60 difficult, dexterous skills using the new approach, including pouring liquids, using tools, and manipulating deformable objects. These achievements were realized without writing a single line of new code; the only change was supplying the robot with new data. Building on this success, TRI has set an ambitious target of teaching hundreds of new skills by the end of the year and 1,000 by the end of 2024.
The news also highlights that robots can be taught to function in new scenarios and perform a wide range of behaviors. These skills are not limited to just “‘pick and place” or simply picking up objects and putting them down in new locations. TRI’s robots can now interact with the world in varied and rich ways—which will one day allow robots to support people in everyday situations and unpredictable, ever-changing environments, TRI says.
The tasks that I’m watching these robots perform are simply amazing—even one year ago, I would not have predicted that we were close to this level of diverse dexterity. What is so exciting about this new approach is the rate and reliability with which we can add new skills. Because these skills work directly from camera images and tactile sensing, using only learned representations, they are able to perform well even on tasks that involve deformable objects, cloth, and liquids—all of which have traditionally been extremely difficult for robots.—Russ Tedrake, Vice President of Robotics Research at TRI
Dr. Tedrake is also the Toyota Professor of Electrical Engineering and Computer Science, Aeronautics and Astronautics, and Mechanical Engineering at MIT.
TRI’s robot behavior model learns from haptic demonstrations from a teacher, combined with a language description of the goal. It then uses an AI-based Diffusion Policy to learn the demonstrated skill. This process allows a new behavior to be deployed autonomously from dozens of demonstrations. Not only does this approach produce consistent, repeatable, and performant results, but it does so with tremendous speed.
Diffusion Policy is a new way of generating robot behavior by representing a robot’s visuomotor policy as a conditional denoising diffusion process. Policy representations: a) Explicit policy with different types of action representations. b) Implicit policy learns an energy function conditioned on both action and observation and optimizes for actions that minimize the energy landscape c) Diffusion policy refines noise into actions via a learned gradient field. This formulation provides stable training, allows the learned policy to accurately model multimodal action distributions, and accommodates high-dimensional action sequences Chi et al..
Key achievements of TRI’s research for this novel development include:
Diffusion Policy: TRI and collaborators in Professor Song’s group at Columbia University developed a new, powerful generative-AI approach to behavior learning. This approach, called Diffusion Policy, enables easy and rapid behavior teaching from demonstration.
<Customized Robot Platform: TRI’s robot platform is custom-built for dexterous dual-arm manipulation tasks with a special focus on enabling haptic feedback and tactile sensing./p>
Pipeline: TRI robots have learned 60 dexterous skills already, with a target of hundreds by the end of the year and 1,000 by the end of 2024. /p>
Drake: Drake is a model-based design for robotics that provides a cutting-edge toolbox and simulation platform. Drake’s high degree of realism allows TRI researchers to develop in both simulation and in reality at a dramatically increased scale and velocity than would otherwise be possible. The internal robot stack is built using Drake’s optimization and systems frameworks. TRI has made Drake open source to catalyze work across the entire robotics community.
Safety: Safety is core to robotics efforts at TRI. The system is designed with strong safeguards, powered by Drake and a custom robot control stack, to ensure robots respect safety guarantees like not colliding with itself or its environment.
Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, Shuran Song (2023) “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion” arXiv:2303.04137v4 [cs.RO] doi: 10.48550/arXiv.2303.04137