Real Brainy: VR Experts Are Teaching Self-Driving Cars How Pedestrians Think

As self-driving cars have gone from science fiction to everyday reality, the dangers they can pose have led to efforts to get them to think like pedestrians.

Using virtual reality technology, University of Leeds researchers have applied their knowledge about how human brains make decisions to automated vehicles.

The researchers focused specifically on a decision-making model known as “drift diffusion.” It assumes that people reach decisions after the accumulation of sensory evidence up to a point at which the decision is made.

By teaching self-driving cars about this theory, the researchers hope vehicles will be able to predict whether a pedestrian intends to cross the road or wait for the car to pass.

If the autonomous vehicle could accurately make such a prediction, it would be able to communicate its intentions more effectively with pedestrians and decrease the chances of running them over, as happened in headline cases in the past few years.

In one incident, a cyclist in Arizona was killed by a self-driving Uber. In another, a self-driving minibus ran over a disabled pedestrian at the 2020 Tokyo Olympic Games.

“When making the decision to cross, pedestrians seem to be adding up lots of different sources of evidence, not only relating to the vehicle’s distance and speed but also using communicative cues from the vehicle in terms of deceleration and headlight flashes,” said Professor Gustav Markkula of the University of Leeds’ Institute for Transport Studies and the senior author of the self-driving car study.

“When a vehicle is giving way, pedestrians will often feel quite uncertain about whether the car is actually yielding and will often end up waiting until the car has almost come to a full stop before starting to cross,” Markkula said.

He explained that their model allows uncertainty to be limited by teaching automated vehicles about how pedestrians behave, which in turn improves safety and traffic flow.

“It is exciting to see that these theories from cognitive neuroscience can be brought into this type of real-world context and find an applied use,” Markkula said.

The research team placed participants in a variety of crossing scenarios at the University of Leeds’ HIKER (Highly Immersive Kinematic Experimental Research) simulator.

The participants were asked to walk freely around the simulated environment where a car was coming directly toward them. They were also asked to cross the road the moment that they felt it was safe.

Different scenarios were tested, with the approaching vehicle either maintaining the same speed or decelerating to let the pedestrian cross, sometimes also flashing the headlights, a signal commonly used in the U.K. to indicate intent to yield.

As the researchers predicted, pedestrians made their decision to cross based on several factors including the sensory data from vehicle distance, speed and acceleration, as well as communicative cues.

The drift-diffusion model was able to predict if and when the pedestrian was likely to cross the road. “These findings can help provide a better understanding of human behavior in traffic, which is needed both to improve traffic safety and to develop automated vehicles that can coexist with human road users,” Markkula said.

“Safe and human-acceptable interaction with pedestrians is a major challenge for developers of automated vehicles, and a better understanding of how pedestrians behave will be key to enable this.”

“Predicting pedestrian decisions and uncertainty can be used to optimize when and how, the vehicle should decelerate and signal to communicate that it’s safe to cross, saving time and effort for both,” said Jami Pekkanen, a cognitive scientist who was the lead author on the study, published in the journal Computational Brain & Behavior.

Edited by Richard Pretorius and Kristen Butler