How Should a Self-Driving Car Tell You to Take the Wheel?

Experts say self-driving cars should start with vibrations in your seat

Photo: William Howard/Getty Images
Advertisement

Someday, the word driver will connote something completely different than it does today. When cars are fully automated and don’t need us for anything more than letting them know where to go, the job of a “driver” will be like that of a patron in a bar selecting a song on a jukebox. But that scenario, where the passenger cabin is a rolling lounge in which all occupants are free to talk, entertain themselves with electronic gadgets, or sleep, is still decades away.

Until then, even the most sophisticated self-driving car will occasionally encounter a circumstance that overwhelms its computerized smarts. At those moments, it will look to a human to take charge. But how will it grab my attention from my phone call or my Netflix movie—or rouse me from sleep—and get me dialed into the potential danger in time to avoid a crash?

Because it’s possible to miss an audible or visual alert if you’re yakking away on a cellphone or engrossed in a car chase scene in a James Bond movie, researchers are looking to add another modality to take-over requests: vibrations in the car seat or seatbelt. A team of researchers at Technische Universität München, in Munich, Germany, and the Delft University of Technology, in the Netherlands, reviewed the spectrum of such alerts, known collectively as vibrotactile displays in the April 2016 edition of the IEEE Transactions on Intelligent Transportation Systems.

How effective is, say, a rumbling seat? “Vibrotactile displays have been shown to be effective as warnings and hard to ignore,” says Sebastiaan M. Petermeijer, a PhD candidate in the ergonomics department at Technische Universität München and the review’s lead author. “In my opinion [they are] a potentially valuable addition to visual and auditory displays in cars, especially when drivers will be engaged in other tasks during highly automated driving,” he says.

The transition from checking email to helming the vehicle doesn’t happen instantly. On average, it takes about 0.8 seconds for a driver to shift attention so that his eyes are on the road. But even after having receiving a tactile warning and turning his or her focus toward the situation, it still takes a moment before the driver has assessed the situation enough to make a helpful response. But tactile warnings, researchers have found, resulted in faster brake reaction times than did, say, flashing lights, beeps, or a computerized voice. Vibrations also spurred human drivers to take the wheel and begin steering much more quickly than did visual or auditory cues.

What’s more, say Petermeijer and colleagues in Germany and The Netherlands, they also found literature indicating that it’s possible to shorten the transition time by employing tactile warnings to direct the driver’s attention towards the danger. This, the researchers concluded, allowed the driver to make a visual assessment of the situation more quickly. In other words, the driver gets up to speed with, well, more speed.

How can the car tell you where to look? Previous research showed that, “a person wearing a tactile waist belt can sense directional feedback in increments as fine as 10 degrees in the horizontal plane.” They note that it is possible to map stimulus zones on the driver’s body to specific locations on the road. So, for example, a series of vibrations on the seat under the driver’s right hip might suggest the danger is coming from behind on the passenger side of the vehicle, while a similar buzzing under the left knee could be directing attention out of the front windshield toward a broken tree branch hanging precariously over the left side of the driving lane.

But just because vibrations can be used this way, does that mean they should? Though the researchers recommend that takeover requests be conveyed by auditory, visual, and vibrotactile displays, they say that each should have its own role in making the driver situationally aware.  “If the situation is truly urgent, you can't risk to rely on a single modality warning,” says Joost de Winter, a mechanical engineering professor at Delft University of Technology and one of the paper’s co-authors.

Asked whether it’s possible that multiple alarms, engaging several senses at once could be confusing, de Winter said:

“That is certainly a valid point. The issue of 'alarm floods' in process control rooms is well known; there is also some literature on overwhelming/startling alarms (in early aircraft, for example). These issues also apply to non-automated driving where increase of technology in cars may be both an opportunity and pose a risk because of mental overload.”

He notes that, although cars are becoming ever more complex, vibrotactile displays can be used in a way that’s unlikely to add to the complexity. “I would argue that vibrations can be intuitive as a warning,” de Winter says, pointing to the highly effective buzzing of mobile phones when they’re in silent mode.

To prevent cognitive overload, they suggest that the different warning modalities (tactile, auditory, and visual) be used at different stages of the take-over process: shift of attention, reposition (as in your hands moving to the steering wheel and your foot moving toward the brake pedal), cognitive processing, action selection, and action implementation.  They note that vibrotactile feedback, “could be used to shift the attention, after which a visual head-up display could show the danger areas that should be avoided by the driver. This way, one effectively exploits the benefits of the vibrotactile warnings (e.g., them being hard to ignore) and of visual feedback (e.g., them being suitable for conveying semantics).”

The Cars That Think Newsletter

Biweekly newsletter about the sensors, software, and systems that are making cars smarter.

About the Cars That Think blog

IEEE Spectrum’s blog about the sensors, software, and systems that are making cars smarter, more entertaining, and ultimately, autonomous.