Researchers advance mobile augmented reality technology

June 17, 2015 by David Robertson, University of Illinois at Urbana-Champaign

Anyone who has seen a spy movie remembers their favorite agent donning special eyewear or using a device to scan surrounding objects and extract information about them. Once an action movie fantasy, augmented reality technology is now almost, well, reality.

"You are interacting with the real world, and putting virtual information on top of that," said Puneet Jain, a Ph.D. student in computer science who has been working on the technology for about two years. He is conducting research and developing the software alongside advisor Romit Roy Choudhury and Justin Manweiler of IBM Research in New York. The project is being funded by NSF, Nokia and Google.

The technology, called OverLay, works inside a smartphone and essentially combines the camera and the to make sense of the surroundings in real time. Once OverLay knows that the user is looking at an object, say a pair of shoes on display, relevant information about those shoes can pop up immediately on the screen. The technology will be a breakthrough for marketers, who will be able to advertise products 24/7.

Such forms of mobile have been a tantalizing topic for researchers since the dawn of smart phones. Jain said that his passion for the project stems from prior experience in developing technology that identifies the location of objects outdoors. Whereas GPS could be utilized outside, geometric understanding of the environment is too erroneous in indoor environments.

Roy Choudhury, an ECE associate professor, noted that "combining vision and sensing achieves the desired outcome. Specifically, as users look at different objects in their environment, the sensor data is used as geometric constraints."

For instance, OverLay observes that objects A and B are statistically separated by certain angles, objects B and C are typically separated by 10 seconds in time and so on.

"With these observations" Jain said, "we came up with an optimization framework to build a geometry out of our surroundings, and once we built that layout, we were able to drastically speed-up computer vision."

The uses for the technology extend beyond defense and surveillance, as seen loosely in fictional spy movies. Jain said that it could be beneficial for infrastructure management, museums, shopping and privacy. For instance, an out-of-reach object, such as a broken air conditioner, could be labeled, with specific information about what part needs to be fixed, using one's phone from a distance. Through augmented reality, museums could provide more interactivity and education about artifacts and exhibits. If photography is prohibited for certain paintings, the camera could automatically become inactive when it's pointed at these exhibits.

Jain said that in the future, the could be deployed by the likes of Google and Amazon, but there are infrastructural, financial and legal obstacles to overcome before a public release of that scale. For the time being, they are working closely with the Illinois Distributed Museum to iron out issues and make mobile augmented reality accessible for a wider audience.

Explore further: Apple buys a maker of futuristic augmented-reality software

Related Stories

Augmented reality for drones

April 3, 2015

Affordable eyes in the sky, drones have fast become a popular and versatile tool for land mapping. Now ESA-backed startup Sysveo has developed a way of integrating user-made augmented reality objects into a drone's video ...

Can it be real? Augmented reality melds work, play

October 14, 2014

(AP)—Mark Skwarek is surrounded by infiltrating militants in New York's Central Park. He shoots one, then hearing a noise from behind, spins to take down another. All of a sudden, everything flashes red. He realizes he's ...

Do we want an augmented reality or a transformed reality?

October 14, 2014

It seems we are headed towards a world where augmented reality (AR) systems will be as common as smartphones are today – it's already about to revolutionise medicine, entertainment, the lives of disabled people and of course ...

Recommended for you

Fish-inspired material changes color using nanocolumns

March 20, 2019

Inspired by the flashing colors of the neon tetra fish, researchers have developed a technique for changing the color of a material by manipulating the orientation of nanostructured columns in the material.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.