Mercedes-Benz S-Class INTELLIGENT DRIVE drives autonomously over 100km interurban and urban route with near-production technology
|Networked sensor systems of the S 500 INTELLIGENT DRIVE research vehicle. Click to enlarge.
Mercedes-Benz has demonstrated the feasibility of autonomous driving on both interurban and urban routes with its S 500 INTELLIGENT DRIVE research vehicle. The company set the vehicle to cover a 100 km (62-mile) route from Mannheim to Pforzheim—retracing that taken by motoring pioneer Bertha Benz exactly 125 years ago when she set off on the first long-distance drive.
The self-driving S-Class had to deal autonomously with a number of highly complex situations such as traffic lights, roundabouts, pedestrians, cyclists and trams. The successful drive was achieved with the aid of near-production-standard technology, very similar to that already found in the new E and S-Class. The project thus marks a milestone along the way that leads from the self-propelled (automobile) to the self-driving (autonomous) vehicle, the company said.
|In August 1888, Bertha Benz set off on the first long-distance automobile journey from Mannheim to Pforzheim. In doing so, the wife of Carl Benz demonstrated the suitability of the Benz patent motor car for everyday use and thus paved the way for the worldwide success of the automobile.
Autonomous driving with production-based sensors. The Mercedes-Benz S 500 INTELLIGENT DRIVE research vehicle was equipped with production-based sensors for the project. Based on a further development of the sensor technologies already in use in the new S-Class, the developers taught the technology platform to know where it is, what it sees and how to react autonomously. With the aid of its highly automated “Route Pilot”, the vehicle is able to negotiate its own way through dense urban and rural traffic.
We have demonstrated that highly automated driving is possible without the luxury of specially closed-off sections of road and relatively straightforward traffic situations. In line with the goal of the project, we have gained important insights into the direction in which we need to further develop our current systems in order to enable autonomous driving not just on motorways, but also in other traffic scenarios. Even we ourselves were quite surprised at just how far we got using our present-day sensor technology. But now we also know how much time and effort is needed to teach the vehicle how to react correctly in a host of traffic situations—because every part of the route was different.
With the new S-Class, we are the first to drive autonomously during traffic jams. We also want to be the first to bring other autonomous functions in series production vehicles. You can expect that we will reach this goal within this decade.—Professor Thomas Weber, member of the Board of Management of Daimler AG with responsibility for Group Research and Head of Mercedes-Benz Cars Development
|S-Class S 500 INTELLIGENT DRIVE. Click to enlarge.
There are three levels of autonomous driving, defined by a VDA working group in collaboration with the German Federal Highway Research Institute (BASt): partially, highly, and fully automated.
With partially automated driving, the driver must constantly monitor the automatic functions and must not pursue any non-driving-related activity.
In the case of highly automated driving, the driver need not permanently monitor the system. In this case, non-driving-related activities are conceivable on a limited scale. The system recognizes its limitations by itself and passes the driving function back to the driver with sufficient time to spare.
With fully automated driving, the system is capable of autonomously coping with every situation; the driver need not monitor the system and can pursue non-driving-related activities. Equally, driverless driving is possible at this level.
Partially automated driving is already available to drivers of new Mercedes-Benz E and S-Class models: the new DISTRONIC PLUS with Steering Assist and Stop&Go Pilot is capable of steering the vehicle mainly autonomously through traffic jams. This system thus forms the core of “Mercedes-Benz Intelligent Drive”, the intelligent networking of all safety and comfort systems on the way to accident-free and, ultimately, autonomous driving.
The now successfully conducted autonomous test drives allowed the Daimler researchers to gather important information on the challenges that remain to be addressed on the way to highly and fully automated driving and what, for example, still needs to be done to enable a car to navigate safely in highly complex situations involving traffic lights, roundabouts, pedestrians and trams.
Initial testing. Unnoticed by the public, yet authorized by the appropriate official exemptions and certificates from the TÜV (German Technical Inspection Authority), testing of the “Route Pilot” on the Bertha Benz route began in early 2012 with a total of three technology platforms based on the E- and S-Class, which are equipped with all available active and passive safety systems.
|Pinpointing vehicle and computation of driving route. Click to enlarge.
These test vehicles employed only those sensor technologies that are already today used in similar form in Mercedes-Benz standard-production vehicles. This is because those technologies are already affordable and suitable for everyday use, which facilitates a possible transfer to subsequent standard-production models. However, improvements were made to the number and arrangement of the sensors in order to achieve comprehensive coverage of the vehicle’s surroundings in every direction, and to obtain additional information on the area around the vehicle.
Based on these sensor data and determination of the vehicle’s own position with reference to information from a digital map, an autonomously driving vehicle analyses the available free area for driving and plans its own route. The required algorithms were developed by the Mercedes-Benz research team in collaboration with the Institute for Measuring and Control Technology at the Karlsruhe Institute of Technology (KIT).
The specific technical modifications compared with the standard-production version of a Mercedes-Benz S-Class are as follows:
The base width (distance between the eyes) of the stereo camera was increased to allow more-distant objects to be detected not only by the radar, but also by the camera.
Two additional long-range radars were installed at the sides of the front bumpers to provide early detection of vehicles coming from the left or right at junctions. A further long-range radar monitors the traffic to the rear.
Four short-range radars at the corners of the vehicle provide improved detection of the nearer surroundings and other road users.
Traffic lights are monitored by a color camera behind the windscreen with a 90-degree opening angle.
Another camera looks towards the back through the rear window to locate the vehicle with reference to known environment features. These environment features were previously entered on a digital map. By comparing what has just been seen by the camera with what is stored on the map, the vehicle is able to locate its position with significantly greater accuracy than would be possible with GPS alone.
|Autonomous turning. Click to enlarge.
For the demonstration drive, Mercedes-Benz collaborated with KIT and HERE, a division of Nokia specializing in the production of digital maps and location-specific services, to produce a 3D digital map of the route between Mannheim and Pforzheim that was specifically adapted to the requirements of an autonomous vehicle.
In addition to the road layout, this map—which must meet special requirements with regard to accuracy—includes information on the number and direction of traffic lanes and traffic signs as well as the positions of traffic lights. Digital maps of this kind are a key prerequisite for autonomous driving. Mercedes-Benz and HERE will therefore continue their collaboration in future with regard to the development of “intelligent” 3D digital maps for autonomous vehicles.
Route Pilot reacts to diverse traffic situations. The Route Pilot in the research vehicle is required to cope with a host of different challenges both on country roads and in urban traffic: roundabouts, obstructions in built-up areas with oncoming traffic, cyclists on the road, turn-off manoeuvres, variously parked vehicles, red traffic lights, “right before left” priority junctions, crossing pedestrians and trams.
|Autonomous navigation of roundabouts. Click to enlarge.
The autonomously driving S-Class was monitored during the tests by specially trained safety drivers who, whenever the system made an incorrect decision, were able to intervene immediately and take over control of the vehicle. As real traffic is unpredictable—i.e., no driving situation is exactly the same as an earlier one—a record was made each time it became necessary for the safety driver to take over control of the vehicle.
This information was then evaluated by the development team, thus making it possible to extend the vehicle’s repertoire of manoeuvres. This advances the development of the technology platform, enabling it to cope with more and more traffic situations.
The test drives along the 100-kilometer route deliver important information for further development of the technology and the product.
For example, it became apparent that the recognition of traffic light phases under different lighting conditions and the correct pairing of individual traffic lights with traffic lanes represents a major challenge. However, it is not our intention that the vehicle should master every situation on its own. If, for example, the road is blocked by a refuse collection vehicle, we certainly don’t want the vehicle to automatically overtake it, especially as the vision of the vehicle’s sensors is restricted in such a case. In such a situation, the vehicle passes control back to the driver.—Prof. Ralf Herrtwich, head of driver assistance and suspension systems at Daimler Group Research and Advance Development
Daimler says the success of the autonomous road tests lies in having identified those areas on which the development team needs to concentrate in future.
|Detecting and reacting to obstacles. Click to enlarge.
These include the situation-dependent control commands for steering, engine and brakes, such as how to autonomously negotiate a roundabout. A further challenge is to correctly locate the vehicle on the road, in order to determine, for example, precisely where a vehicle should stop at a junction while at the same time having a view of cross-traffic.
A particular challenge for autonomous vehicles is the way in which they communicate and interact with other road users. Coming to an agreement with an oncoming vehicle on who should proceed first around an obstruction is something that requires a great deal of situational analysis.
Where a human driver might boldly move forward into a gap, our autonomous vehicle tends to adopt a more cautious approach. This sometimes results in comical situations, such as when, having stopped at a zebra crossing, the vehicle gets waved through by the pedestrian—yet our car stoically continues to wait, because we failed to anticipate such politeness when we programmed the system.—Ralf Herrtwich
|Traffic light detection. Click to enlarge.
To enable the developers to reconstruct the decisions made by the autonomous research vehicle in individual driving situations, the car makes recordings of all its sensor data. Images from the stereo camera alone generate 300 gigabytes of data every hour. Also in later standard operation, some of these data will continue to be stored. If, for example, an autonomous vehicle is involved in an accident, this information will make it possible to establish what happened.
Challenges on the path to autonomous driving. Before the goal of highly and fully autonomous driving is achieved, the obstacles to be overcome will not be just of a technical nature. Many of the things that are already technically feasible are still not universally permitted.
For example, international UN/ECE Regulation R 79 (steering systems) allows only corrective steering functions, but not automatic steering at speeds above 10 km/h (6.2 mph). Under the Vienna Road Traffic Convention, which is relevant for EU law, the driver must be in constant control of their vehicle and be capable of intervening at all times. As autonomous vehicles were still out of the question at the time this convention was adopted, clarification is needed with regard to what this means for highly or fully automated vehicles. In some US states such as Nevada, there has already been such clarification, at least as far as the trial operation of autonomous vehicles is concerned.
Another prerequisite for the transition from partially to highly automated systems is their acceptance in society. Just as when the automobile was originally invented, it will first of all be necessary to build up confidence in the technical capabilities of the systems.
This is borne out by a recent study carried out by the Customer Research Centre at Mercedes-Benz involving around 100 test persons aged between 18 and 60. The initial skepticism of the study participants was almost entirely dispelled following an autonomous drive in the driving simulator. Even among those participants who were negatively disposed to begin with, there was a significant increase in acceptance after the drive in the simulator.
One way of ensuring that map data and route information is always kept up to date is to use "Car-to-X Communication". This could enable future vehicles to help each other to generate real-time maps, because, theoretically, every car is capable of recording the route it has driven and entering it in a database.
Information on a red traffic light could be relayed from a waiting car to other road users. Alternatively, the traffic light itself could send a signal to nearby vehicles. Mercedes-Benz has been working for several years on communication between vehicles and between vehicles and their environment. This year, it is set to become the first manufacturer to bring “Car-to-X functions” onto the market.
PROMETHEUS. Mercedes-Benz has already spent years of research in the field of autonomous driving. An earlier milestone was the Daimler-Benz-initiated research project EUREKA-PROMETHEUS (“Programme for European Traffic with Highest Efficiency and Unprecedented Safety”), which ran from 1986 and test vehicles of which made headlines when, in 1994, in normal traffic, they covered around 1,000 kilometers, mainly autonomously, on a multi-lane motorway in the Paris region and then, in 1995, drove from Munich to Copenhagen. Consequently, almost 20 years ago, Mercedes-Benz demonstrated that automated driving on motorways, including lane-changing, overtaking and keeping a safe distance, is technically feasible.
One of the outcomes of the Prometheus project was DISTRONIC adaptive cruise control, which went into production in the S-Class in 1998. Based on DISTRONIC, Mercedes-Benz has developed a succession of assistance systems capable of detecting hazardous situations, warning the driver and, ever more frequently, also automatically intervening.
The project also resulted in Speed Limit Assist, which went into series production in 2005. Continuous further advances in environment detection using stereo cameras, also first tested as part of PROMETHEUS, created the foundation for the “6D Vision” stereo camera, which has now been launched in the new E- and S-Class. Patented by Daimler, this technology makes it possible to anticipate the real-time movements of other nearby road users.
At a technical level, Prometheus and the Mercedes-Benz S 500 INTELLIGENT DRIVE are worlds apart.
Progress has been due above all to modern-day hardware and software, which have been the focus of targeted optimisation over the years. Technical components in those days were much too big and much too expensive for standard use in automobiles. Also, they were not powerful or reliable enough. The situation today is a quite different one. Our modern systems can be installed in compact control units that, while exceptionally powerful, are still affordable. Because that’s the only way in which the maximum possible number of customers can benefit from autonomous vehicle functions—and that’s our ultimate goal.—Thomas Weber