Toshiba Corporation has developed technology that can predict automobile and drone movement with high accuracy. Motion Estimation AI uses on-board cameras and inertial sensors that measure acceleration and angular velocity to deliver highly accurate estimates of the movement of a drone or a vehicle, while Motion Prediction AI predicts the movements of other vehicles on the road in a wide range of traffic conditions.

Tested against a public data set, they both achieve the world’s highest accuracy (Motion Estimation AI: Comparison between monocular cameras and inertial sensors without loop closure using public dataset of KITTI, EuRoC. Motion Prediction-AI: Comparison using public dataset of Argoverse). Details of the technologies were presented at International Conference on Robotics and Automation (ICRA), streamed online on 1 June.

A safe driving system must be able simultaneously to track the position of its vehicle and the surrounding environment—the real-time movement of other vehicles, pedestrians, potential hazards and obstacles. Technologies that use sensors such as LiDAR or GPS have been developed to do just that, but the former is expensive, and shadows from tall buildings can block the latter.

This has prompted growing interest in Simultaneous Localization and Mapping (SLAM) technology, which uses images from on-board camera to estimate a vehicle’s location and to make three-dimensional spatial maps around the location, and motion prediction technology, which uses past-to-present movements of other vehicles to dynamically estimate their position over time.

Technologies like these must meet the test of being usable in all kinds of road conditions and complex traffic flows, not only on expressways where vehicle movement is relatively simple and predictable.

Toshiba’s new AI solutions advance the accuracy of SLAM and motion prediction, and will contribute to the realization of autonomous driving.

Motion Estimation AI combines data from in-vehicle camera images and inertial measurement unit (IMUs), including angular velocity and acceleration sensors, to estimate vehicle movement, and performs robustly in all kinds of traffic conditions. However, there is a problem with this approach, in that the data from the inertial sensor can be affected by the velocity of the vehicle—its acceleration and deceleration—which adversely impacts on movement accuracy.

Toshiba has developed a method that constantly estimates vehicle motion by determining the usefulness of the data from the camera image and the IMUs and using only the relevant data.

Motion Estimation AI can estimate the motion of cars that travel at a relatively constant pace, with limited acceleration and deceleration. Tests using a public driving data set confirmed that the AI produces 40% fewer errors than the current method of estimates based on all of the data obtained from the camera and IMUs, and 82% fewer errors than when only the camera was used.

As the figure below shows, the estimated track of the Motion Estimation AI is almost perfectly aligned with the actual track.

Application of Motion Estimation AI is not confined to the road. The technology can also estimate the motion and much larger acceleration and deceleration of drones. This capability that advances the goal of realizing autonomous drones that are able to fly a predetermined route and respond to conditions around them at the same time.

Motion Prediction AI estimates the movements of other vehicles on the road in a wide range of traffic conditions. Conventionally, realizing highly accurate prediction of future movement requires predictive AI models for road configurations, including the number of lanes and road curvature. The large number of models needed to accommodate the many different possible configurations make it difficult to apply to a wide range of real-world conditions.

Toshiba’s Motion Prediction AI uses a deep neural network to input generalized geometric features, including road curvature, and realizes predictive AI that can accommodate a wide range of road configurations.

There are two stages in system operation: first the AI predicts the movement of other vehicles in each lane independently of road shape, and then predicts which lane each individual vehicle will select, enabling highly accurate prediction for many different road configurations. Tested against a public domain dataset, the Motion Prediction AI achieved the world’s highest accuracy, reducing the error in predicting the future position of other vehicle in four seconds’ time by more than 40%.

Going forward, Toshiba will evaluate this technology in real-world environments, such as public roads, aiming for practical application in fiscal 2023.