Researchers work to develop a vehicle that can be driven by the blind

Feb 04, 2011 By Alan S. Brown
Blind driver Mark Riccobono behind the wheel at the Daytona International Speedway. Credit: Steven Mackay / Virginia Tech

Last Saturday, a blind driver dodged cardboard boxes thrown in front of him while driving a modified Ford Hybrid Escape around the Daytona International Speedway. He had only seconds to react to the obstacles.

"If we just put boxes on the track, people might think we planned the route," said Dennis Hong, whose robotics and mechanisms lab at Virginia Tech modified the cars.

Instead, Hong's team threw boxes from a van so they bounced around. “That shows everyone that their position is random, and that the drivers are really driving,” said Hong.

In addition to avoiding boxes and taking the raceway's turns, the driver, Mark Riccobono, also passed the van.

Fortunately, Riccobono and a second blind driver, Anil Lewis, had done it before.

"The other day, when we got to drive by ourselves, it surpassed any perception of thrilling," Riccobono wrote in an e-mail after a mid-January test run at Virginia International Speedway. Riccobono, who lost his sight at the age of five, is executive director of the National Federation of the Blind's research arm, the Jernigan Institute.

"It's scary and exciting," said Lewis, 46, who lost his sight 21 years ago and thought he would never drive again.

The demonstration at Daytona took place before blind supporters and fans gathered for the Rolex 24 sports race as part of the National Federation of the Blind's Blind Driver Challenge to develop a car blind people can drive independently.

"We're trying to change people's minds about what blind people can do, and driving is going to change minds," said Lewis, the Federation's communications director. "This is taking it to whole different level."

Riccobono reached speeds of more than 25 mph and did not hit a single box.

This video is not supported by your browser at this time.

Lewis was not worried about a small accident or two. "If it goes off too perfectly, people won't believe its credibility," Lewis said. After all, he explained, the car is a research project and people should expect some failures. Besides, "sighted" drivers have traffic accidents too.

Hong had other goals. "I want it to be perfect," he said. "This is a controversial project. I'm getting hundreds of e-mails from people saying, 'You won't believe how much hope this brings to us.' "But I'm also getting hate mail, saying 'Are you insane?'"

Hong understands. In fact, he had doubts until blind drivers tested a prototype.

Development

The National Federation of the Blind announced the Blind Driver Challenge in 2004. Only Virginia Tech responded, and it did not sign up until 2006.

At the time, Hong's lab was developing a driverless car for the DARPA Urban Challenge. Sponsored by the Defense Advanced Research Projects Agency, it called for cars to navigate through 60 miles of traffic, lights, stop signs, and obstacles at an old military base. Virginia Tech's upstart team surprised everyone by claiming the $500,000 third prize.

Hong thought he could use a similar approach to build a car for the blind. That was not the Federation's goal.

"They wanted the blind person to make active decisions, and actually drive and control the vehicle," Hong said.

Yet the two vehicles had much in common. Both use similar sensors. A laser light detection and ranging (LIDAR) system identified cars and other obstacles in the road. In addition, two forward-pointing cameras monitored the road as well as lights and stop signs. A GPS system located the car on a map, while an inertial measurement unit tracked the car's speed and direction in case it lost GPS contact.

The vehicles' computers automatically gathered all the sensory information and blended it together to create a model of the car's rapidly changing environment.

In the driverless car, the computer assessed the model, picked out the route, and used the drive-by-wire system to drive the car.

The Blind Driver Challenge vehicle was different. Instead of telling the car how to drive, it had to communicate this information to the driver, who then had to respond accordingly.

Hong's team faced two initial hurdles. First came money. Autonomous cars are expensive, but the Federation offered Hong only $5,000 to get started. He put together a team of 12 undergraduates and they bought a used gasoline-powered dune buggy on eBay for $2,000.

They bought whatever equipment was not donated. The LIDAR made by Hokuyo Automatic Co. was the most expensive component, and ordinarily cost about $8,000.

"Hokuyo originally donated it for another robotics project, but we used it on this," Hong related. "I was afraid to tell them, but when they saw what we did, they became big fans."

Driving By Touch

The second problem was more profound. How could Virginia Tech convey information fast enough to a driver who cannot see? After all, most computers communicate with humans through visual displays. For the blind, that wasn't possible.

Hong started by letting the computer pick the best route and communicate driving information -- fast, slow, right, left, stop -- to the driver.

For the dune buggy, the students built a vest from a massage chair vibrator. Different massage patterns told the driver when to speed up, slow down, or stop. In the Ford Escape, the expanded team of engineers built the vibrator, now branded SpeedStrip, into the driver's seat.

"We're experimenting with straight up-and-down and zigzag patterns, still figuring out which ones are most effective," Hong explained.

The dune buggy's original steering system used a steering wheel that made clicking noises when it turned. The car's computer told the driver how many clicks to turn the wheel. It was awkward, and forced blind drivers to listen to the computer rather than use their hearing to assess their environment.

The Ford Escape, by contrast, uses DriveGrip, a glove with a small vibrating motor on the knuckle of each finger. The more motors that vibrate, the sharper the driver needs to turn.

Finding the right vibration pattern proved a challenge. Hong considered which hand to signal for which turn; whether the motors should vibrate all at once or sequentially; and whether they should turn off suddenly or gradually as the driver completed the turn.

He also had to face the human-in-the-loop problem. An autonomous vehicle is easy for a computer to control because it responds to commands the same way every time.

Humans are another story. Different drivers may interpret the signal to turn hard differently. Even the same driver may respond differently on different days.

This showed up during testing. The computer would define the "road" as an 18 foot wide path on the 30 foot wide racetrack. "If I steered out of that lane, the car would shut down," Lewis recounted. This often happened when cornering.

"Sighted people get a chance to correct their errors," Lewis said. "We asked for the same chance, and when we got it. We got better at staying on the road."

In this case, the solution to the human-in-the-loop was to modify the car's controls so that the driver could learn to adapt to its signals.

The blind drivers and a blind engineer on the Virginia Tech team also suggested cutting off the ends of the DriveGrip gloves, since blind people need touch to sense their environment. They also collaborated on finding the best vibration patterns.

Lewis and Riccobono practiced for a total of seven days during the months before Daytona. Lewis says the Blind Driver Challenge will help blind people stay on the forefront of technology.

"We were worried that the knobs and buttons we used on appliances and phones were going to touch screens. We wanted to make sure there are non-visual interfaces out there for us," Lewis said.

Those interfaces are coming. They are what Hong calls "informational" interfaces. On the Escape, they communicate what is going on outside the car.

One is AirPix, a pad with pressurized air flowing through a grid of pinholes. It works like an air hockey table, but AirPix controls each pinhole's air pulses to form "pictures" that a blind person can view with his or her fingers, like Braille. A similar technology uses a 3-dimensional rubber membrane that changes shape to show road conditions.

The technology is too new to use at Daytona. Yet one day, similar technologies may not only help the blind to drive, but perhaps to navigate streets, use portable computer devices, or view notes from an electronic blackboard.

Explore further: First drone in Nevada test program crashes in demo

Provided by Inside Science News Service

3.8 /5 (13 votes)

Related Stories

Blind can take wheel with new vehicle

Jul 15, 2009

A student team in the Virginia Tech College of Engineering is providing the blind with an opportunity many never thought possible: The opportunity to drive.

Recommended for you

First drone in Nevada test program crashes in demo

10 hours ago

A drone testing program in Nevada is off to a bumpy start after the first unmanned aircraft authorized to fly without Federal Aviation Administration supervision crashed during a ceremony in Boulder City.

Fully automated: Thousands of blood samples every hour

18 hours ago

Siemens is supplying automation technology for the longest and one of the most cutting-edge sample processing lines in any clinical laboratory. The line, or automation track, 200 meters long, in Marlborough, ...

Explainer: What is 4-D printing?

18 hours ago

Additive manufacturing – or 3D printing – is 30 years old this year. Today, it's found not just in industry but in households, as the price of 3D printers has fallen below US$1,000. Knowing you can p ...

First series production vehicle with software control

18 hours ago

Siemens has unveiled the first electric series production vehicle with the central electronics and software architecture RACE. This technology, developed in the research project of the same name, replaces ...

Amputee puts limb system through its paces

21 hours ago

"Amputee Makes History with APL's Modular Prosthetic Limb" is the headline from Johns Hopkins Applied Physics Laboratory, where a team working on prosthetics observed a milestone when a double amputee showed ...

User comments : 14

Adjust slider to filter visible comments by rank

Display comments: newest first

Skepticus
1 / 5 (1) Feb 04, 2011
Great work, but I can think of a few things that need to be overcome, such as thin chains strung across a private country driveway, or chicken wire fence gate. Can the Lidar pick these things up?
Moebius
3.6 / 5 (9) Feb 04, 2011
Are these people insane? It's not scary and exciting, it's just scary. As a motorcycle rider it's bad enough with people who can see that are blind.
Quantum_Conundrum
4 / 5 (4) Feb 04, 2011
What is the point in this when a computer software that can do the dodging for itself makes much more sense and has much faster reaction time?
Eikka
4.2 / 5 (5) Feb 04, 2011
This isn't the blind person driving. This is the blind person acting as a servo motor for the computer! You could replace the person with an electric motor that turns the wheel.

I'm very dissapointed. I thought they'd develop a way to communicate the road ahead to the driver and let him decide what to do. Instead the computer does it all and just goes "left a bit here..."
Telekinetic
3.4 / 5 (5) Feb 04, 2011
This technology allows a blind person to text while driving.
Skepticus
1.8 / 5 (5) Feb 04, 2011
What's next? Blind surgeons doing circumcisions..!(Blind Fury film)
ekim
not rated yet Feb 04, 2011
The demonstration at Daytona took place before blind supporters and fans...

That must have been quite the sight to see.
trekgeek1
5 / 5 (2) Feb 05, 2011
At this point, when a computer and sensors are doing 99 % of the seeing, just let the computer do the next 1% and make it completely autonomous.
RobertAitken
not rated yet Feb 05, 2011
At this point, when a computer and sensors are doing 99 % of the seeing, just let the computer do the next 1% and make it completely autonomous.


Exactly what i thought.
ubavontuba
1 / 5 (1) Feb 05, 2011
I like the vest idea. It makes me wonder; could a device, attached to the skin, stimulate the skin in such a way as to essentially re-purpose the skin's sense of touch into a pseudo retina?

I also wonder; could the brain adapt to this, and actually perceive such input stimulation as sight?

Imagine; Seeing the world with your chest!

I suspect artificial (or cloned) eyes are the next step. But this seems like a possible intermediate solution...
Eikka
1 / 5 (1) Feb 06, 2011
I like the vest idea. It makes me wonder; could a device, attached to the skin, stimulate the skin in such a way as to essentially re-purpose the skin's sense of touch into a pseudo retina?


Simply put: yes.

There are experiments done on placing a grid of electrodes on your tongue to transmit a crude video image, and it works. It's just nowhere sharp enough to do anything.
Eikka
1 / 5 (1) Feb 06, 2011
What a blind person would need to drive a car is

A) sense of direction
B) sense of space

One is accomplished by a gyrocompass that runs a vibrator belt to tell them which way they're turning with high precision, or something equivalent, and the other one could be an ultrasonic sonar, the echoes from which would be heterodyned down to audible frequencies and the user given one for each ear for stereo hearing.

The radar points in the same direction as the user's head, thanks to the gyrocompass, and thus the user can look around and listen to oncoming objects.

The sonar idea has already been proven to work on blind people who learn to click their tongues to listen to their surroundings. As long as the surface reflects sound differently, they can sense objects about the size of cigarette packs and locate them.
CarolinaScotsman
not rated yet Feb 06, 2011
But can they make a car that can be safely driven by idiots? That will be the ultimate breakthrough.
COCO
1 / 5 (1) Feb 07, 2011
this reminds me of time when we had a communist government in power in Ontario - they had all types of quotas - for all walks of life - even the pro teams - fortunaltely the Maple Leaves - a sad sack collection of low performers already made the grade with a legally blind goalie.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.