In a briefing at CES 2019, Dr. Gill Pratt, Toyota Research Institute (TRI) CEO and Toyota Motor Corporation Fellow said that one of TRI’s most significant breakthroughs this year was the creation of blended envelope control for its Guardian driver assistance system. The control combines and coordinates the skills and strengths of the human and the machine.
TRI is following a two-track development approach to automated driving. Chauffeur development focuses on full autonomy, in which the human is essentially removed from the driving equation, either completely in all environments, or within a restricted operational design domain (ODD). Toyota Guardian, on the other hand, is being developed to amplify human control of the vehicle, not replace it.
With Toyota Guardian, the driver is meant to be in control of the car at all times, except in those cases where it anticipates or identifies a pending incident and employs a corrective response in coordination with driver input.
The blended envelope control system was inspired and informed by the way that modern fighter jets are flown, where you have a pilot that flies the stick, but doesn’t fly the plane directly. Instead, pilot intent is translated by the low-level flight control system thousands of times a second to stabilize the aircraft and stay within a specific safety envelope.
Toyota Guardian Will Amplify Human Control, Not Remove it Envelope-Control System Mimics Fighter Jet, Blending Best Input from Human and Machine Test Vehicle Crash on California Highway Reenacted with Actual Video and 3D Animation January 07, 2019 LAS VEGAS, Jan. 7, 2019 – Toyota is doubling-down on humans. That’s the answer to the question; What has the been doing in automated driving over the past 12 months? Using onboard cameras, sensors and 3D animation, TRI opened its portion of a press conference today at the annual CES show in Las Vegas with a vivid reenactment of a three-car crash on a California interstate, where no one was injured.
“We know what happened because we were there, in the thick of it,” said . “Our test vehicle was travelling at freeway speed in manual mode with its autonomy mode disabled as it gathered data at the many tunnels and bridges in the San Francisco bay area. As we downloaded data from the incident, we asked ourselves; Could this crash have been mitigated, or avoided altogether by a future Toyota Guardian automated safety system? We believe the answer is yes.”
TRI said that blended envelope control is much more difficult to create in a car than in a fighter jet because the control envelope for a car is not only defined by vehicle dynamics, but also by the vehicle’s perception and prediction ability of all things in its immediate environment.
This control envelope is not a discrete on-off switch between the human and the autonomy—it is intended to be a near-seamless blend of both, working as teammates to extract the best input from each.
Guardian is being developed as an automated safety system, capable of operating with either a human driver, or an autonomous driving system, provided by Toyota, or any other company. This is a key capability.
As Akio Toyoda announced at CES last year, Toyota plans to include Toyota Guardian as standard equipment on all Toyota e-Palette platforms (earlier post) that the company will build for the MaaS (Mobility as a Service) market. By doing so, MaaS fleet buyers can use any autonomous system they choose, with Toyota Guardian acting as a belt-and-suspender redundancy for any self-driving Chauffeur system.
Pratt stressed the importance of not underestimating the difficulty of developing an autonomous Chauffeur system, both technologically and sociologically. Technically, how do we train a machine about the social ballet required to navigate through an ever-changing environment, as well as, or better than, a human driver? Sociologically, public acceptance of the inevitable crashes, injuries, and deaths that will occur due to fully autonomous Chauffeur systems may take considerable time.
In the meantime, we have a moral obligation to apply automated vehicle technology to save as many lives as possible as soon as possible.—Gill Pratt
Hence, TRI’s primary focus over the last year has been to concentrate most of its effort on making Toyota Guardian a smarter machine. For Guardian to learn and get smarter, it must be subjected to difficult and demanding driving scenarios, “corner cases” that are simply too dangerous to perform on public roads.
On closed courses, Guardian’s intelligence and capabilities can be stretched and challenged. Through continuous refinement, Guardian learns how best to navigate and react to extremely dangerous scenarios, as they unfold.
TRI opened its CES session with a reenactment of a three-car crash on a California interstate, in which no one was injured.
We know what happened because we were there, in the thick of it. Our test vehicle was travelling at freeway speed in manual mode with its autonomy mode disabled as it gathered data at the many tunnels and bridges in the San Francisco bay area. As we downloaded data from the incident, we asked ourselves; Could this crash have been mitigated, or avoided altogether by a future Toyota Guardian automated safety system? We believe the answer is yes.—Gill Pratt
From that data, TRI developed an accurate simulation which was then translated into a learning tool for the car to figure out its options in a split-second. The scenario was then re-created on the test track, using real vehicles and a guided, soft-target, dummy-vehicle.
In this instance, Guardian’s best option was to quickly accelerate for daylight and away from encroaching vehicles. Here is a case where Guardian might avoid, or mitigate a collision for itself, while potentially doing the same for other nearby vehicles; in a way, an Altruistic Guardian.
Pratt addressed another key capability that makes Toyota Guardian even more evocative.
We humans have an inherent need for autonomy, which is much stronger than our desire for autonomous cars. It’s about the sheer delight of mobility when a child first learns to stand-up and giggle its way across a room without the help of mom or dad. And it is the joy of just going for a drive, behind the wheel of a car that can accelerate, brake and turn as if an extension of your body.—Gill Pratt
The joy of driving is an inherent and deliberate component of Guardian. Hands-on-the-wheel and eyes-on-the-road is first-and-foremost about safety. But it is not ALL about safety. It is also about how a driver feels behind the wheel; safe and secure to enjoy the drive, instead of the ride.