Improving our depth perception in augmented reality

Sep 18, 2012 by Miles O'brien

(—Sports fans have come to expect some of the extras they see on their TV screen, such as the yellow lines that appear on a football field highlighting where the ball needs to go for a first down. Similarly, NASCAR fans can find their favorite driver in the pack because of those superimposed car numbers on the screen.

"Anyone who's seen a football game has seen the virtual first-down line and it looks like it's really something painted on the field. It's a very compelling graphic. It really looks like it's there," says computer scientist J. Edward Swan, II.

But most people are probably not familiar with the technology that makes this all happen.

This video is not supported by your browser at this time.

, or "AR," is used to superimpose computer-generated virtual objects on our view of the real world.

With support from the National Science Foundation (NSF), Swan and his team at Mississippi State University (MSU) are working to improve in the augmented reality environment.

"Ed Swan's work on perceptual issues for AR goes back more than a decade and includes co-authoring a seminal paper in which computer graphics attributes were shown to be effective in providing a sense of the depth of occluded objects for mobile AR using a ," explains Lawrence Rosenblum, program officer for graphics and visualization within the NSF Directorate for Computer and and Engineering. "As AR continues to develop, with many of the ideas for mobile AR now being ported to PDAs and improved hardware for medical and other relatively stationary applications, his ongoing AR perception research is paving the pathway for working AR systems."

One challenge is aligning an object in the real word so that it precisely lines up with virtual objects. While exactness in millimeters may not be all that critical for a TV sporting event, think about a soldier relying on this tool during battle or a doctor using this technology in an operating room. The virtual graphics might be located inside the patient's body.

"Imagine an application where you have a surgeon trying to align a scalpel in relation to virtual graphics that are also in view," says Swan.

"The surgeon would look at the patient and be able to see into their skin. So, instead of looking to one side and seeing the scanned information on a monitor, they would look at the patient, but it's as if they're looking into the patient, as if the patient has become somewhat transparent at that spot," continues Swan.

Think about it as a sort of X-ray vision!

The most advanced lab that is actually applying augmented reality to medical applications is located at the Technical University of Munich, which Swan visited in 2011. There are no scalpels or heart monitors in Swan's lab at Mississippi State, but there is an augmented reality haploscope.

"A haploscope is a precision device used by vision scientists to present a carefully controlled image to each eye, so it's a stereo image into the right and left eye," he says.

In Swan's lab, volunteers use the haploscope to position and real ones. Graduate student Gurjot Singh explains to the volunteers that they will see an object that looks exactly like a physical one, but it will be computer generated. Their goal is to align those objects.

"The purpose of the head-mounted display device is to display an image in stereo. I have two transparent screens in front of my eyes, which display a stereo screen. This thing on my head is a tracking device, which tracks my head when it moves, and the device sends information to a computer through this wire. And when the computer receives this information, it draws the scene on this screen based on this information, and my location in space. That's how, when I move my head, you can see the scene is updated on the screen," explains Singh.

He says the equipment has improved dramatically in just the past few years.

"The last device used to be so heavy. The optics were made of glass. These are made of plastic, and are much brighter than our last device. It gives a more compelling immersive experience than our last one," says Singh.

Accurate depth perception is vital to military applications of augmented reality. Swan previously worked at the Naval Research Lab, on a project to develop a mobile augmented reality system.

"The idea is that soldiers would walk around with an augmented reality display on their head. They would be able to see the real world with their actual vision and we would be able to show them graphics that, if we did it right, would look just like they were also real objects in the world," says Swan.

On a battlefield, soldiers must quickly assess what's happening on the ground. As AR improves, they will increasingly need to combine that situational awareness with commands or warnings sent through their goggles or helmet.

"You don't want to fill someone's field of view with graphics that are going to block too much of their view of the real world," notes Swan.

Ryan Ismert is general manager for augmented reality at Sportvision, the company that's developed this technology for major league sports broadcasters. And for TV viewers, just like for a soldier or a surgeon, simplicity is the key.

"It is presented as a natural part of the game, explaining something that may otherwise be hard to see, like the first down marker or the path someone took for a reception route," says Ismert.

Augmented reality is also the tool used to paint those country flags on the bottom of swimming pools to distinguish swimmers during the Summer Olympics.

"Visual reasoning adds an extra layer of understanding and engagement to sports, but industry may also benefit from augmented reality tools in everything from aircraft maintenance to capping an underwater oil well," says Ismert.

Swan agrees that industrial applications of AR could be widespread.

"Boeing was the first company to try using augmented reality as part of their manufacturing process, in the early 1990's. One of Boeing's chief scientists recently said that augmented reality was going to be increasingly used at Boeing and at other high-tech manufacturing companies in the coming decades," says Swan.

Explore further: Bringing history and the future to life with augmented reality

Related Stories

Augmented Reality: Science Fiction or Reality? (w/ Video)

Jul 07, 2009

( -- Computer graphics have come a long way since the birth of Atari Games over 30 years ago. Today, computer graphics seem very real and some day researchers will pull graphics out of your television ...

Sony uses Augmented Reality to guide TV buyers

Aug 29, 2011

( -- Sony UK is using Augmented Reality technology in a new online tool to help consumers decide which Sony television size will be best for their walls or tables. For those in a palatial home ...

Recommended for you

Patent talk: Google sharpens contact lens vision

Apr 16, 2014

( —A report from Patent Bolt brings us one step closer to what Google may have in mind in developing smart contact lenses. According to the discussion Google is interested in the concept of contact ...

Neuroscientist's idea wins new-toy award

Apr 15, 2014

When he was a child, Robijanto Soetedjo used to play with his electrically powered toys for a while and then, when he got bored, take them apart - much to the consternation of his parents.

Land Rover demos invisible bonnet / car hood (w/ video)

Apr 14, 2014

( —Land Rover has released a video demonstrating a part of its Discover Vision Concept—the invisible "bonnet" or as it's known in the U.S. the "hood" of the car. It's a concept the automaker ...

User comments : 0

More news stories

Ex-Apple chief plans mobile phone for India

Former Apple chief executive John Sculley, whose marketing skills helped bring the personal computer to desktops worldwide, says he plans to launch a mobile phone in India to exploit its still largely untapped ...

Airbnb rental site raises $450 mn

Online lodging listings website Airbnb inked a $450 million funding deal with investors led by TPG, a source close to the matter said Friday.

Health care site flagged in Heartbleed review

People with accounts on the enrollment website for President Barack Obama's signature health care law are being told to change their passwords following an administration-wide review of the government's vulnerability to the ...

A homemade solar lamp for developing countries

( —The solar lamp developed by the start-up LEDsafari is a more effective, safer, and less expensive form of illumination than the traditional oil lamp currently used by more than one billion people ...

NASA's space station Robonaut finally getting legs

Robonaut, the first out-of-this-world humanoid, is finally getting its space legs. For three years, Robonaut has had to manage from the waist up. This new pair of legs means the experimental robot—now stuck ...

Filipino tests negative for Middle East virus

A Filipino nurse who tested positive for the Middle East virus has been found free of infection in a subsequent examination after he returned home, Philippine health officials said Saturday.

Egypt archaeologists find ancient writer's tomb

Egypt's minister of antiquities says a team of Spanish archaeologists has discovered two tombs in the southern part of the country, one of them belonging to a writer and containing a trove of artifacts including reed pens ...