Computer-enhanced vision adds a 'sixth sense'

Apr 06, 2010 by Marlowe Hood
The red carpet outside the Kodak Theater is reflected on the sunglasses of a woman watching in Hollywood, California. Cutting-edge technology which monitors and interprets what our eyes see has gone on show at an augmented reality conference in the French Alps, with scientists and engineers demonstrating how the technology can give someone digital feedback about what he or she is gazing at.

Picture this: As your eyes alight for the first time on a skyscraper in a foreign cityscape, a disembodied voice whispers in your ear the phone number of a posh bar on the top floor.

Or this: You have been spotted on the street by an old friend whose name suddenly eludes you. But even before there is time to shake hands, a glance at your smartphone reveals her identity and the date of your last encounter.

Welcome to the world of , the here-and-now enhancement of everyday experience through virtual, interactive technology.

Prototypes of both of these applications -- based on the novel use of eye-tracking tools -- were presented last weekend at the inaugural Augmented Human International Conference.

Over two days, engineers and scientists gathered in the French Alps ski resort of Megeve unveiled cutting-edge research on boosting human perception with information from the Internet, customised databases, or even biofeedback from our own brains.

The first devices for monitoring eye movement collected data from pilots in the 1940s to help improve cockpit design.

They have also been used to figure out the most effective ways to get people to see advertising.

More recently the systems have became interactive, making it possible to instantly provide computer-enhanced feedback to someone about what he or she is gazing at.

These newer technologies has been used mainly by the military, and to develop life-assistance tools for the severely disabled.

But a team of researchers from The Telecommunications Research Center in Vienna decided to take a state-of-the-art eye tracker designed for Web-use analysis out of the laboratory and onto the street.

They hooked up the device -- with one camera trained on the user's eye, and another on the scene being observed -- to a smart phone with a built-in compass and (GPS), to get a fix on the user's orientation and location.

They added sensors that show whether one was looking up or down, and attached the whole kit -- designed to navigate urban landscapes -- to a bicycle helmet.

Closing one's eyes for two seconds triggers a request for information about the building, bridge or monument in view.

A remotely-accessed computer scans geo-referenced databases on the Internet such as Google Earth, and then forwards the result back to the user's cell phone, closing the loop.

"We wanted to make the system as non-intrusive as possible, so we used a text-to-speech engine. Data is received through an ear piece," explained Matthias Baldauf, one of the researchers. "It should be like a sixth sense."

A representative from a major international oil company, asking that he not be identified, said the application could be useful for security training or work on oil platforms.

"We consider this to be a transformative technology," he said.

Another "proof-of-concept" invention presented in Megeve -- functional, but a long way from commercial development -- adapted eye-tracking technology as an a memory aide.

Rather than training a camera on the eye, the "Aided Eye" system developed by a team from the University of Tokyo uses tiny infrared sensors.

While less accurate, additional data about eye movement and the frequence of blinking make it possible to pinpoint a face or a book cover within a field of vision.

And rather than matching the object to content on the Internet, the program devised by the scientists draws from a hand-tailored database of images and files, sometimes called a personal lifelog.

"For the experiment, we registered 100 images for the database," explained Yoshio Ishiguro from the Graduate School of Interdisciplinary Information Studies.

"When the eye trained on an object, it was recognized by the computer and a corresponding file was extracted," he said.

The system is light enough to be mounted onto a pair of reading glasses, but researchers have still not figured out how to provide the wearer feedback.

A tiny screen embedded inside the glasses or an audio system are both options, Ishiguro said.

Explore further: Team shows calibrated multiple-projector spherical display

add to favorites email to friend print save as pdf

Related Stories

An affordable future for eye tracking in sight

Apr 03, 2006

An ambitious five-year project will attempt to make eye-tracking technologies more affordable for people with disabilities and extend the potential use of the devices to enable users to live more independently.

Watching me, watching you

Oct 21, 2009

(PhysOrg.com) -- Software that tracks shoppers' eye movements as they browse supermarket shelves may seem a bit Big Brother, but the latest technology in 'eye-tracking', which monitors what grabs a person's ...

Car or pedestrian -- How we can follow objects with our eyes

Oct 02, 2008

When an object moves fast, we follow it with our eyes: our brain correspondingly calculates the speed of the object and adapts our eye movement to it. This in itself is an enormous achievement, yet our brain can do even more ...

Tourist information wherever you are

Apr 11, 2008

Would you like instant access to information on the buildings and scenery you see on your travels? A novel mobile phone programme, able to provide information on what you see when you see it, was a regional ...

Recommended for you

South Korea's Samsung and LG unveil new smartwatches

15 minutes ago

South Korea's Samsung and LG unveiled new smartwatches Thursday with upgraded functions and design as they step up their drive to lead an increasingly competitive market for wearable devices.

Apple to unveil 'iWatch' on September 9

2 hours ago

Apple will unveil an "iWatch" in September with the maker of the iPhone finally embarking on its much-rumored foray into wearable computing, technology news website Re/code said Wednesday.

User comments : 9

Adjust slider to filter visible comments by rank

Display comments: newest first

shockr
not rated yet Apr 06, 2010
Augmented Reality is the future. Forget your mobile phone. We'll all be wearing glasses with a camera on each side for stereoscopic input of our surroundings. The lenses will be HUD style screens and tiny sensors will detect our eye motion. The glasses will have headphones and all this will be wirelessly connected to other peoples glasses/devices.

This whole process will allow anything to become an interactive medium. Gaming will become a physical experience, with people playing first person shooters around their home or street. Watching films via augmented reality on a clear wall at home.

We just need the tech to become smaller and faster. I say 5-10 years tops.
hooloovoo
not rated yet Apr 06, 2010
@shockr

Are you serious? You want stereoscopic input of reality, while you're standing in it?
gwrede
1 / 5 (1) Apr 06, 2010
This will be reality, no question.

The eyeglasses we will wear are only the I/O device, the actual computer will be an object in our pocket, much like the mobile phone is today. And for many applications, the majority of the needed number crunching will be done at some remote computer farm.

Such glasses will of course be heavier and bulkier than "regular" glasses, but people will get used to them. (I remember trying on a Rolex wrist watch as a teen ager and I felt my arm stretch by the sheer weight of it.)

24h live video recording will be standard issue, as well as transparent HUD displays of various things. And the thing in your pocket only has an on/off button. You can then communicate with it through the glasses by blinking. For advanced users, additional inputs are possible, like winking, raising an eyebrow, jerking your ears, clicks of tongue, or special eye movements (like drawing a square, etc.).

And these will mostly be company property, like expensive laptops today.
danman5000
not rated yet Apr 06, 2010
This is great - I'll never have to remember anything again! Why fill my brain with useless facts like my best friend's name and my home address when the computer can just tell me? Now all we need are hover chairs so we can all live idyllic lives like the people in Wall-E.
jj2009
not rated yet Apr 06, 2010
wow sounds amazing, how on earth did we get by before this technology was invented... oh wait, we bought guide books, and actually made the effort to remember our friends' names.
krundoloss
not rated yet Apr 06, 2010
I think this is great. There are so many useful and helpful applications for this, and now that the technology is coming along, we wont need all these neat little devices with tiny screen (which I am quite tired of) and have something that is physically small but perceptually large (like a display built into eyeglasses). Yes, we are all lazy, but this technology can be VERY empowering, and we cannot deny that. Heck, Im a network technician and I dont bother to try to remember things like registry fixes, because I can just google it, or pull the file off my Flashdrive which I always have. Face it, the human memory is becoming OBSOLETE.
trekgeek1
Apr 06, 2010
This comment has been removed by a moderator.
MikeMike
not rated yet Apr 06, 2010
I want realtime visual filters...downloadable like phone apps. Having a grey day? Just fire up a filter!

Q: "What filter are you running Joe?"
A: "Acid Trip 2.3! It rocks man!"
fixer
not rated yet Apr 06, 2010
This ties in nicely with earlier releases on Retinitis Pigmentosa tech.
Sight coupled with information for people who couldn't previously see.
Also useful for tourists visiting anywhere!
shockr
not rated yet Apr 08, 2010
@hooloovoo

The camera's aren't for my benefit, they're for the computer (in my pocket) to detect my surroundings. (Like the XBox 360 Natal). With this information of orientation and depth, the computer can determine how I can interact with things.

Let's say I want to watch a film, I can tell the computer to overlay the image on the wall infront of me. It's augmenting reality, not putting me in a virtual reality created from my surroundings. (Although that will also be part of the gaming experience).

What's more, your appearance to someone using augmented reality could be your own custom avatar. Your device would let your friends device see you as anything you wanted.

Gaming could turn your back yard into a virtual warzone, but one that you physically run around in, with no danger to anyone (unless you trip and fall on your own clumsy feet!) :P