Car Talk: LISA develops smart cars to make safer drivers

The behavioral research study is underway in UC San Diego’s Laboratory for Intelligent and Safe Automobiles (LISA), a sprawling set of spaces replete with cameras, microphones, computers and experimental facilities like the driving simulator. Led by Mohan Trivedi, a Jacobs School of Engineering professor and researcher in the California Institute for Telecommunications and Information Technology (Calit2), LISA is also bustling with more than a dozen graduate students and postdoctoral researchers in electrical and computer engineering, and cognitive science.

The test drives began in mid-July and remain open to volunteers. The findings will help Trivedi and his team fine-tune the mathematical algorithms that make it possible for an on-board computer to figure out whether the motorist intends to make a left turn—even before the driver starts to do it.

“Understanding driver behavior is critical to reducing fatalities on U.S. roads,” says Trivedi. “So in a very real sense, the students who volunteer to spend 30 minutes in the simulator could be saving lives down the road.”

The computer-based driving simulator is one of several novel testbeds that will make up a new Human Intention, Perception and Performance Observatory (HIPPO).

Enlarge

What makes this simulator different from regular videogames is the array of cameras in front of the driver: While the driver is watching the road, the cameras are watching the driver. In particular, Trivedi’s team uses a computer vision-based tracker to record the exact position of the driver’s head, and the precise direction of the driver’s gaze, facial expressions, even foot gestures.

“We have carried out similar experiments in our passenger-car testbeds,” explains Trivedi. “But it takes a whole team of people to make sure the technology is working when you need it, and it’s time-consuming. So bringing students into the simulator accelerates the experiment and allows us to base the experiment on many more drivers and driving behaviors.”

The objective of the lab’s inventions is to reduce the 35,000 fatalities on U.S. roadways each year. Understanding the driver can also lead to less congested roads, where American commuters spend 4.2 billion hours each year in traffic, contributing to three billion gallons of wasted fuel—and over 20 percent of the carbon dioxide released into the atmosphere.

Enlarge

To improve road safety, the National Academy of Sciences recently undertook a two-year study to collect the data necessary to influence future policies and standards. “We have some top priority areas, like driver distraction and driver fatigue,” says Charles Fay, who helps manage the initiative at the National Academy’s Transportation Research Board.

In the 1990s, many researchers took a Jetsons view of smart vehicles—banking that the future of car safety was a world where cars are autonomous and drive themselves. But the research pendulum appears to have swung back in the direction that engineers call “human-centered” driver assistance systems.

“Here at UCSD, we have worked for the past decade on a very different vision of the future,” says Trivedi, “one where drivers remain in charge, but with intelligent driver assistance technologies to help them make better, faster decisions. In other words, systems that support rather than replace the driver.”

LISA was among the first in the world to focus on driver-support systems, with financial and research support from international automakers including Mercedes, Nissan, Volkswagen and Audi. And it didn’t take long for automakers to begin implementing early versions of driver-assistance systems on their newest models. For example, Ford now touts its “Parking Assist” technology in primetime TV.

Enlarge

According to Trivedi, parking systems require the same basic technologies as many other assistance systems. “They require a full view of the area around the vehicle in order to gauge the location of the curb and where other cars may be parked in front and in back of the empty spot,” explains Trivedi. From there it’s a question of running all the information in real time through computer processors with mathematical algorithms that will send an exact set of instructions to actuators on the steering wheel, accelerator and brakes.

The chief of intelligent technology research at the National Highway Traffic Safety Administration (NHTSA) says computer vision experts are ushering in a new future for automotive applications. “You replace the need for one or more expensive sensors with relatively inexpensive video that can provide up to 360 degrees of awareness around the vehicle,” says NHTSA’s Raymond Resendes. “From that you get the capability to provide an entire suite of integrated vehicle-safety systems that may help the driver avoid a crash.”

While other researchers were starting to install cameras on the outside of the vehicle, Trivedi and his team at UCSD were also installing cameras and other sensors inside the car. An early study funded by VW aimed to determine if a camera could assess the size and position of the passenger in the front seat—and send that information quickly enough to the computer for it to be processed and acted upon in the three-hundredths of a second it takes to reduce the force of an airbag deploying after a collision. The reason: many of the worst injuries to passengers in frontal collisions are sustained by smaller adults, or by passengers leaning forward at the time of the crash.

Nearly a dozen Ph.D. dissertations in Trivedi’s group have now relied on experimental data from test drives in the Nissan, VW and Audi sedans that make up LISA’s fleet of sensor-rich vehicles. Located in an expanded garage behind the SERF building, they are tricked-out with multiple cameras, radar, laser-based LIDAR, GPS systems, and trunks full of computer processing equipment. But according to Trivedi, cameras are not sufficient to prevent most accidents.

“You may have your eyes on the road and your hands on the wheel where they are supposed to be, but your mind may be elsewhere,” warns Trivedi. “So the challenge isn’t to see the driver, but to figure out whether the driver is paying attention to a looming danger.”

Enlarge

The UCSD researchers eventually made a breakthrough when they began to explore driver intent in a truly interdisciplinary collaboration: Trivedi’s team partnered with UCSD psychology professor Hal Pashler; cognitive scientists including UCSD’s Jim Hollan and Michael Mozer M.A. ’82, Ph.D. ’87, now at the University of Colorado; and Scott Makeig, Ph.D. ’85, director of UCSD’s Swartz Center for Computational Neuroscience, who studied the EKG’s of drivers to see if brain signals can help monitor the state of the driver.

“Are they tired, are they angry, are they depressed?” asks Trivedi. “All of those states can play a role in being safe on the road or being dangerous. One of the most telling research studies carried out so far in the UCSD lab involved predicting which way a driver would turn—before the right or left turn signal is activated. The research—published in 2009 and 2010 by Trivedi and current postdoctoral researcher Anup Doshi—showed that head and eye movements begin before the driver decides to turn.

“We can come up with a very, very accurate prediction of whether the driver is intending to break, change lanes, overtake or turn,” explains Trivedi. “We can make that prediction as much as two or three seconds before it happens, and if we have situational awareness at an intersection or the speeds and locations of other cars in the vicinity, we can predict whether the driver’s next action could result in an accident.”

2010 was a banner year for Trivedi’s team. They hosted the IEEE Intelligent Transportation Systems (ITS) conference—the first time ever it was staged in the western U.S.—and postdoctoral researcher Brendan Morris, M.S. ’06, Ph.D. ’10, earned the ITS Society’s Best Ph.D. Dissertation Award (two years after another graduate student in the group, Shinko Cheng, Muir ’01, M.S. ’04, Ph.D. ’05, won the same award in 2008). In addition, the top academic journal in the field labeled the UCSD group as the most productive and influential of the previous decade—even as many of the group’s innovations were starting to make their way into showrooms. Some cars, for example, now sport force-sensitive accelerators that make it harder to speed up if sensors indicate that a car in front is braking—technology that Trivedi’s lab was investigating five years ago. LISA researchers also demonstrated novel ways to derive a panoramic view around the vehicle—a critical step that made “park assist” features possible and economical.

“The car is not your grandfather’s Oldsmobile anymore,” laughs Trivedi. “Today, there is so much electronics, so much computing, so much software in vehicles, and these things have become the key innovation elements in the industry. That means California, San Diego and UCSD, with our strengths in information technology and telecom and eco-friendly solutions—we represent the future of the automobile.”

As if to prove the point, Trivedi recites the list of LISA’s freshly minted Ph.D. graduates and the high-tech companies that scooped them up: Google hired two of the most recent grads, and other Ph.D. researchers went to Microsoft Research, Qualcomm, SAIC, IBM, HRL Labs (formerly Hughes Research), Digimarks, and even Yahoo!

Trivedi freely admits that the U.S. is not yet leading the world in intelligent driver-assistance systems. “I just hope that the U.S. will always stay engaged in this pursuit because it’s an important area for multidisciplinary intellectual inquiry,” he says. “And because it’s important for society.”

Provided by University of California - San Diego