Results from the inaugural testing round of driver-assist systems in highway applications were released this month. The Mercedes-Benz GLE earned a “very good” rating and scored 174 points, the highest of the 10 vehicles tested. Systems on the BMW 3-Series and the Audi Q8 also earned “very good” marks.
Beyond the particulars of the initial results, what’s most interesting may be how Thatcham and Euro NCAP devised the testing regimen.
Researchers measured performance across three categories, analyzing how well subsystems worked together to control speed and steering, how effective vehicles were in monitoring human drivers to ensure they remain engaged and how backup systems respond in emergencies or during malfunctions.
Establishing a healthy friction and collaboration between human and machine, rather than an implicit trust, is an underlying tenet of the ratings. Young describes this as a careful balancing act. If there’s not enough automation, drivers won’t appreciate the systems or find them helpful.
“If your keyboard only has half the letters in the alphabet, that doesn’t help you write an email,” he said. “However, with too much automation, users will come to rely on it even if it’s not competent. So there is a critical sweet spot with the person and car working together that provides the safest and most efficient operation.”
Without reaching that sweet spot, advanced driver-assist systems can lull drivers into a sense of automation complacency, a phenomenon that federal crash investigators have listed as a contributor to a range of transportation crashes, including fatal incidents involving Tesla’s Autopilot system, in the U.S.
Thatcham tested Autopilot in a Model 3 vehicle, and its findings illustrate the complexity involved in devising a robust system. Autopilot earned the highest marks of any of the evaluated technologies in two of the three categories. But it earned the lowest score in driver engagement.
While many other automakers use inward-facing cameras to ensure drivers are watching the road ahead, Tesla uses steering-wheel torque to monitor that. Findings from National Transportation Safety Board investigations have said that’s an inadequate measure of engagement.
Young finds it problematic for more mundane reasons.
“The driver engagement is really small,” he said. “During the driving task, the system provides very strong resistance on the steering wheel. If you want to make a minor course correction, the car actually fights you. With other systems, you can interact more and drive around a pothole if you want. Tesla fights this and disconnects, which is psychologically telling the driver, ‘If you fight me, I’m going to turn off.’ It’s the completely wrong ethos with assisted driving.”
Overall, Tesla earned a “moderate” grade on the tests.
The Thatcham and Euro NCAP tests come at a time when transportation officials in Britain and Germany are proposing regulations that would permit automakers to deploy more highly automated systems, those with SAE Level 3 automation capabilities, in which responsibility can be passed between human and machine. Systems currently on the road, including those tested by Thatcham and Euro NCAP, are Level 2, in which humans retain responsibility for vehicle operations at all times.
