During current emissions standards tests, cars are placed on a chassis equipped with a dynamometer. The vehicle follows a precisely defined speed profile that tries to mimic real driving on an urban route with frequent stops. The conditions of the test are both standardized and public. This essentially makes it possible for manufacturers to intentionally alter the behavior of their vehicles during the test cycle. The code found in Volkswagen vehicles checks for a number of conditions associated with a driving test, such as distance, speed and even the position of the wheel. If the conditions are met, the code directs the onboard computer to activate emissions curbing mechanism when those conditions were met.

Electronic engine control has also made it easier to circumvent emissions testing by implementing a defeat device in software. The black box nature of emissions testing makes it nearly impossible to discover such a software-based defeat device during a test, forcing regulators to rely on heavy fines to discourage cheating. Unfortunately, as the Volkswagen case illustrates, it can take many years to discover such a defeat device. Given the ultimate limitations of testing, we are led to consider whether we can detect defeat devices using software verification techniques. Unfortunately, verifying complex software systems is a difficult problem in its own right, more so for a cyber-physical system like a modern automobile. In our case, the setting is also adversarial—rather than trying to find bugs, we are looking for intentional attempts to alter a system’s behavior under test conditions. This paper aims to be a first step in cyber-physical system verification in an adversarial setting with two case studies of automobile defeat devices and binary analysis techniques to identify verification- critical code elements across multiple software revisions.

—Contag et al.

Computer scientist Kirill Levchenko led the research effort at UC San Diego. The work, supported by the European Research Council and by the US National Science Foundation (NSF), started when computer scientists at Ruhr University, working with independent researcher Felix Domke, teamed up with Levchenko and the research group of computer science professor Stefan Savage at the Jacobs School of Engineering at UC San Diego.

Savage, Levchenko and their team have extensive experience analyzing embedded systems, such as cars’ onboard Engine Control Units for vulnerabilities. The team examined 900 versions of the code and found that 400 of those included information to circumvent emissions tests.

A specific piece of code was labeled as the “acoustic condition”—ostensibly, a way to control the sound the engine makes. But in reality, the label became a euphemism for conditions occurring during an emissions test. The code allowed for as many as 10 different profiles for potential tests. When the computer determined the car was undergoing a test, it activated emissions-curbing systems, which reduced the amount of nitrogen oxide emitted.

The Volkswagen defeat device is arguably the most complex in automotive history.

—Kirill Levchenko

Researchers found a less sophisticated circumventing ploy for the Fiat 500X. That car’s onboard computer simply allows its emissions-curbing system to run for the first 26 minutes and 40 seconds after the engine starts—roughly the duration of many emissions tests.

We implemented our approach in a tool called CURVEDIFF. Given that we perform an intra-procedural analysis, we might miss certain ways how a defeat device can be implemented and an inter-procedural analysis could enhance the soundness of our implementation. Furthermore, our analysis can be extended to take more primitive building blocks such as timers and multiplexers into account to deepen the knowledge about the relation of various components in the detection logic.

—Contag et al.

The study draws attention to the regulatory challenges of verifying software-controlled systems that may try to hide their behavior and calls for a new breed of techniques that work in an adversarial setting.

Resources

• Moritz Contag, Guo Li, Andre Pawlowski, Felix Domke, Kirill Levchenko, Thorsten Holz, and Stefan Savage (2017) “How They Did It: An Analysis of Emission Defeat Devices in Modern Automobiles”