The Toyota Research Institute (TRI) is helping to accelerate the field of soft robotics by sharing the design source files and full build instruction for its innovative domestic robot hands. Today, any research institution or aspiring roboticist can visit punyo.tech and build their own Punyo Soft Bubble Gripper.

TRI Soft Bubble Gripper

The soft robotics community is small, and the visuotactile sensing community is even smaller. By sharing the blueprints for this gripper with the world, we hope that our friends and colleagues can test our technology, improve upon it, and take us closer to building robotic assistants that help to provide independence, dignity and joy to those with disabilities or age-related challenges.

—Alex Alspach, TRI’s Robotics Tactile Team Manager and the lead developer of the Punyo Soft Bubble Gripper

Building on recently published work, the Punyo project is changing the mechanics of robot manipulation and contact with the world. To this day, most robots are hard to the touch and use rigid grippers, but TRI’s air-filled, elastic bubble design allows robots greater flexibility to hold objects better.

The Punyo bubbles employ state-of-the art visuotactile sensing techniques that allow a robot to recognize objects by shape, track their orientation in its grasp and sense forces as it interacts with the world. This feedback is critical as robots learn to push and pull on the world safely and robustly while assisting people by opening doors, putting things away, using household tools, and other domestic tasks.

When combined with cameras on the inside, this shape- and force-sensing gripper enables robots to respond to and control an object when it slips or moves. Lab-validated capabilities of the Soft Bubble Gripper include:

• Robust and Passively-Compliant Grasping. The innovation on the Soft Bubble Gripper begins with the use of the inherently grippy texture and durable elastic properties of latex. The latex is inflated to a degree of softness that optimizes compliance to the shapes of held objects—maximizing the grasp stability. Given the physical properties—its air-filled bubbles and the gummy, high-friction texture of latex—the Soft Bubble Gripper is very reliable when it comes to grabbing and holding onto objects.

• Recognizing Objects by Touch. Inside the bubble is a low-cost, off-the-shelf Time-of-Flight (ToF) depth sensor/IR camera that uses vision to “feel” what the gripper is holding. It enables the system to recognize objects by their shape and other physical properties and understand what to do with it within about a second.

  This capability lets the robot perform realistic, useful tasks typically found in a home, because it understands how the object is oriented in its “hand” and what it must do to complete the task.

  The process that the Soft Bubble Gripper uses to sort a sink full of objects is similar. It measures and recognizes geometric features, then either precisely sets mugs in the dishwasher or drops PET bottles into the recycling bin. It perceptively sorts the shapes, not visually, but through touch.

• Shear Force Detection and Interpretation. The Soft Bubble Gripper can also sense when some outside force is trying to take, twist, pull or push the object. It does this by using the camera inside the bubbles to measure how a dense dot pattern inside the latex membrane is moving and distorting, and inferring the magnitudes and directions of the forces causing this distortion. This lets it swiftly sense when the object it is lowering has landed on the counter, if it has accidentally bumped into something, or when an object it is handing over has been received. The gripper gives the robot tactile awareness of the outside world and its cohabitants.

• Operating Blind. Most robots rely heavily on cameras to create a sense of vision. This means that light is needed to help create this sense of perception. The technical term for this is “visual occlusion.” Traditionally, robots also have difficulty working around clutter or in confined spaces, where there’s no clear line of sight. Transparent, shiny or dark objects are also quite hard for robots to see. However, since a robot equipped with the Soft Bubble Gripper relies on sensors inside the bubble (and not an external camera) to recognize and manipulate objects, it works equally well in lit or darkened rooms. This also means the technology is ideally suited to situations in which it must reach into cluttered places (such as a sink full of miscellaneous objects and/or water) or has to manipulate in a way that would otherwise block its own view.

• Training through Self-Annotation. Traditionally, a robotic system guided by computer vision is trained to recognize objects by repeatedly showing it images of a thing in order to establish defined categories—a process known as supervised training. For example, somebody has to show the robot many times over various photos of coffee mugs and tell it, “these are all coffee mugs.” This process of annotation is incredibly time consuming. The team had a breakthrough when they learned how to set up the robot to self-annotate by filling a sink with one type of object and letting the robot repeatedly grasp and drop these objects, reducing the amount of time it takes the robot to “learn” a new object. This process is easily repeated for every new object the team wants the Soft Bubble Gripper to learn. It is important to build tactile sensors that can withstand the many cycles this type of training requires for minimal wear and damage.

• Use of Low-Cost Materials. Utilizing materials smartly with simple fabrication techniques, the team designed the Soft Bubble Gripper to be inexpensive to build, operate, maintain and repair.