Computer-enhanced vision adds a 'sixth sense'

April 6, 2010 by Marlowe Hood
The red carpet outside the Kodak Theater is reflected on the sunglasses of a woman watching in Hollywood, California. Cutting-edge technology which monitors and interprets what our eyes see has gone on show at an augmented reality conference in the French Alps, with scientists and engineers demonstrating how the technology can give someone digital feedback about what he or she is gazing at.

Picture this: As your eyes alight for the first time on a skyscraper in a foreign cityscape, a disembodied voice whispers in your ear the phone number of a posh bar on the top floor.

Or this: You have been spotted on the street by an old friend whose name suddenly eludes you. But even before there is time to shake hands, a glance at your smartphone reveals her identity and the date of your last encounter.

Welcome to the world of , the here-and-now enhancement of everyday experience through virtual, interactive technology.

Prototypes of both of these applications -- based on the novel use of eye-tracking tools -- were presented last weekend at the inaugural Augmented Human International Conference.

Over two days, engineers and scientists gathered in the French Alps ski resort of Megeve unveiled cutting-edge research on boosting human perception with information from the Internet, customised databases, or even biofeedback from our own brains.

The first devices for monitoring eye movement collected data from pilots in the 1940s to help improve cockpit design.

They have also been used to figure out the most effective ways to get people to see advertising.

More recently the systems have became interactive, making it possible to instantly provide computer-enhanced feedback to someone about what he or she is gazing at.

These newer technologies has been used mainly by the military, and to develop life-assistance tools for the severely disabled.

But a team of researchers from The Telecommunications Research Center in Vienna decided to take a state-of-the-art eye tracker designed for Web-use analysis out of the laboratory and onto the street.

They hooked up the device -- with one camera trained on the user's eye, and another on the scene being observed -- to a smart phone with a built-in compass and (GPS), to get a fix on the user's orientation and location.

They added sensors that show whether one was looking up or down, and attached the whole kit -- designed to navigate urban landscapes -- to a bicycle helmet.

Closing one's eyes for two seconds triggers a request for information about the building, bridge or monument in view.

A remotely-accessed computer scans geo-referenced databases on the Internet such as Google Earth, and then forwards the result back to the user's cell phone, closing the loop.

"We wanted to make the system as non-intrusive as possible, so we used a text-to-speech engine. Data is received through an ear piece," explained Matthias Baldauf, one of the researchers. "It should be like a sixth sense."

A representative from a major international oil company, asking that he not be identified, said the application could be useful for security training or work on oil platforms.

"We consider this to be a transformative technology," he said.

Another "proof-of-concept" invention presented in Megeve -- functional, but a long way from commercial development -- adapted eye-tracking technology as an a memory aide.

Rather than training a camera on the eye, the "Aided Eye" system developed by a team from the University of Tokyo uses tiny infrared sensors.

While less accurate, additional data about eye movement and the frequence of blinking make it possible to pinpoint a face or a book cover within a field of vision.

And rather than matching the object to content on the Internet, the program devised by the scientists draws from a hand-tailored database of images and files, sometimes called a personal lifelog.

"For the experiment, we registered 100 images for the database," explained Yoshio Ishiguro from the Graduate School of Interdisciplinary Information Studies.

"When the eye trained on an object, it was recognized by the computer and a corresponding file was extracted," he said.

The system is light enough to be mounted onto a pair of reading glasses, but researchers have still not figured out how to provide the wearer feedback.

A tiny screen embedded inside the glasses or an audio system are both options, Ishiguro said.

Explore further: Samsung patent talk: Flying display right before your eyes

Related Stories

'Call me baby': Talking sex dolls fill a void in China

February 2, 2018

With China facing a massive gender gap and a greying population, a company wants to hook up lonely men and retirees with a new kind of companion: "Smart" sex dolls that can talk, play music and turn on dishwashers.

How Facebook could really fix itself

February 1, 2018

Facebook has a world of problems. Beyond charges of Russian manipulation and promoting fake news, the company's signature social media platform is under fire for being addictive, causing anxiety and depression, and even instigating ...

Recommended for you

AI and 5G in focus at top mobile fair

February 24, 2018

Phone makers will seek to entice new buyers with better cameras and bigger screens at the world's biggest mobile fair starting Monday in Spain after a year of flat smartphone sales.

Google Assistant adds more languages in global push

February 23, 2018

Google said Friday its digital assistant software would be available in more than 30 languages by the end of the years as it steps up its artificial intelligence efforts against Amazon and others.


Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Apr 06, 2010
Augmented Reality is the future. Forget your mobile phone. We'll all be wearing glasses with a camera on each side for stereoscopic input of our surroundings. The lenses will be HUD style screens and tiny sensors will detect our eye motion. The glasses will have headphones and all this will be wirelessly connected to other peoples glasses/devices.

This whole process will allow anything to become an interactive medium. Gaming will become a physical experience, with people playing first person shooters around their home or street. Watching films via augmented reality on a clear wall at home.

We just need the tech to become smaller and faster. I say 5-10 years tops.
not rated yet Apr 06, 2010

Are you serious? You want stereoscopic input of reality, while you're standing in it?
1 / 5 (1) Apr 06, 2010
This will be reality, no question.

The eyeglasses we will wear are only the I/O device, the actual computer will be an object in our pocket, much like the mobile phone is today. And for many applications, the majority of the needed number crunching will be done at some remote computer farm.

Such glasses will of course be heavier and bulkier than "regular" glasses, but people will get used to them. (I remember trying on a Rolex wrist watch as a teen ager and I felt my arm stretch by the sheer weight of it.)

24h live video recording will be standard issue, as well as transparent HUD displays of various things. And the thing in your pocket only has an on/off button. You can then communicate with it through the glasses by blinking. For advanced users, additional inputs are possible, like winking, raising an eyebrow, jerking your ears, clicks of tongue, or special eye movements (like drawing a square, etc.).

And these will mostly be company property, like expensive laptops today.
not rated yet Apr 06, 2010
This is great - I'll never have to remember anything again! Why fill my brain with useless facts like my best friend's name and my home address when the computer can just tell me? Now all we need are hover chairs so we can all live idyllic lives like the people in Wall-E.
not rated yet Apr 06, 2010
wow sounds amazing, how on earth did we get by before this technology was invented... oh wait, we bought guide books, and actually made the effort to remember our friends' names.
not rated yet Apr 06, 2010
I think this is great. There are so many useful and helpful applications for this, and now that the technology is coming along, we wont need all these neat little devices with tiny screen (which I am quite tired of) and have something that is physically small but perceptually large (like a display built into eyeglasses). Yes, we are all lazy, but this technology can be VERY empowering, and we cannot deny that. Heck, Im a network technician and I dont bother to try to remember things like registry fixes, because I can just google it, or pull the file off my Flashdrive which I always have. Face it, the human memory is becoming OBSOLETE.
Apr 06, 2010
This comment has been removed by a moderator.
not rated yet Apr 06, 2010
I want realtime visual filters...downloadable like phone apps. Having a grey day? Just fire up a filter!

Q: "What filter are you running Joe?"
A: "Acid Trip 2.3! It rocks man!"
not rated yet Apr 06, 2010
This ties in nicely with earlier releases on Retinitis Pigmentosa tech.
Sight coupled with information for people who couldn't previously see.
Also useful for tourists visiting anywhere!
not rated yet Apr 08, 2010

The camera's aren't for my benefit, they're for the computer (in my pocket) to detect my surroundings. (Like the XBox 360 Natal). With this information of orientation and depth, the computer can determine how I can interact with things.

Let's say I want to watch a film, I can tell the computer to overlay the image on the wall infront of me. It's augmenting reality, not putting me in a virtual reality created from my surroundings. (Although that will also be part of the gaming experience).

What's more, your appearance to someone using augmented reality could be your own custom avatar. Your device would let your friends device see you as anything you wanted.

Gaming could turn your back yard into a virtual warzone, but one that you physically run around in, with no danger to anyone (unless you trip and fall on your own clumsy feet!) :P

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.