Low-powered, high-speed head-mounted display with augment reality chip

Feb 18, 2014
This is a prototype of K-Glass. Credit: KAIST

Walking around the streets searching for a place to eat will be no hassle when a head-mounted display (HMD) becomes affordable and ubiquitous. Researchers at the Korea Advanced Institute of Science and Technology (KAIST) developed K-Glass, a wearable, hands-free HMD that enables users to find restaurants while checking out their menus. If the user of K-Glass walks up to a restaurant and looks at the name of the restaurant, today's menu and a 3D image of food pop up. The Glass can even show the number of tables available inside the restaurant. K-Glass makes this possible because of its built-in augmented reality (AR) processor.

Unlike virtual reality which replaces the real world with a computer-simulated environment, AR incorporates digital data generated by the computer into the reality of a user. With the computer-made sensory inputs such as sound, video, graphics or GPS data, the user's real and physical world becomes live and interactive. Augmentation takes place in real-time and in semantic context with surrounding environments, such as a menu list overlain on the signboard of a restaurant when the user passes by it, not an airplane flight schedule, which is irrelevant information, displayed.

Most commonly, location-based or computer-vision services are used in order to generate AR effects. Location-based services activate motion sensors to identify the user's surroundings, whereas computer-vision uses algorithms such as facial, pattern, and , or object and motion tracking to distinguish images and objects. Many of the current HMDs deliver augmented reality experiences employing location-based services by scanning the markers or barcodes printed on the back of objects. The AR system tracks the codes or markers to identify objects and then align them with . However, this AR algorithm is difficult to use for the objects or spaces which do not have barcodes, QR codes, or markers, particularly those in outdoor environments and thus cannot be recognized.

This is the floor plan of AR Custom Chip for K-Glass. Credit: KAIST

To solve this problem, Hoi-Jun Yoo, Professor of Electrical Engineering at KAIST and his team developed, for the first time in the world, an AR chip that works just like . This processor is based on the Visual Attention Model (VAM) that duplicates the ability of to process visual data. VAM, almost unconsciously or automatically, disentangles the most salient and relevant information about the environment in which human vision operates, thereby eliminating unnecessary data unless they must be processed. In return, the processor can dramatically speed up the computation of complex AR algorithms.

K-Glass recognizes the target object, which is a vehicle in this case, and then a 3D model of the vehicle is displayed on the top-left side with additional information on the top-right side. Credit: KAIST

The AR processor has a network similar to that of a human brain's central nervous system. When the human brain perceives visual data, different sets of neurons, all connected, work concurrently on each fragment of a decision-making process; one group's work is relayed to other group of neurons for the next round of the process, which continues until a set of decider neurons determines the character of the data. Likewise, the artificial neural network allows parallel data processing, alleviating data congestion and reducing power consumption significantly.

This video is not supported by your browser at this time.

KAIST's AR processor, which is produced using the 65 nm (nanometers) manufacturing process with the area of 32 mm2, delivers 1.22 TOPS (tera-operations per second) peak performance when running at 250 MHz and consumes 778 miliWatts on a 1.2V power supply. The ultra-low power processor shows 1.57 TOPS/W high efficiency rate of energy consumption under the real-time operation of 30fps/720p video camera, a 76% improvement in power conservation over other devices. The HMDs, available on the market including the Project Glass whose battery lasts only for two hours, have revealed so far poor performance. Professor Yoo said, "Our processor can work for long hours without sacrificing K-Glass's high performance, an ideal mobile gadget or wearable computer, which users can wear for almost the whole day."

He further commented: "HMDs will become the next mobile device, eventually taking over smartphones. Their markets have been growing fast, and it's really a matter of time before mobile users will eventually embrace an optical see-through HMD as part of their daily use. Through , we will have richer, deeper, and more powerful reality in all aspects of our life from education, business, and entertainment to art and culture."

The KAIST team presented a research paper at the International Solid-State Circuits Conference (ISSCC) held on February 9-13, 2014 in San Francisco, CA, which is entitled "1.22TOPS and 1.52mW/MHz Augmented Reality Multi-Core Processor with Neural Network NoC for HMD Applications."

Explore further: SeeThru AR eyewear device sets sights on consumer market

add to favorites email to friend print save as pdf

Related Stories

SeeThru AR eyewear device sets sights on consumer market

Jan 18, 2014

(Phys.org) —By air, by sea, by workout trails, augmented reality headsets have just got more interesting with Laster Technologies' SeeThru eyewear. Laster recently launched its SeeThru campaign on Kickstarter, ...

Apple granted patent on new augmented reality technology

Mar 20, 2013

(Phys.org) —Apple Inc. has been granted a patent for an application filed with the U.S. Patent Office in 2010 for "Synchronized, interactive augmented reality displays for multifunction devices." The patent ...

Metaio announces AR processing unit for phones

Feb 22, 2013

(Phys.org)—Metaio this week announced its AREngine, an augmented reality chip that closes in on the future of smartphones as AR devices for daily use. The hardware chipset being introduced is a jump up ...

Augmented reality browser Junaio has new look

Jun 21, 2012

(Phys.org) -- The new version of the augmented reality (AR) browser Junaio launched this week with more promise of easy immersive browsing with mobile devices. The idea is to merge via smartphone the vastness ...

Recommended for you

Ex-Qualcomm exec pleads guilty to insider trading

9 hours ago

A former high-ranking executive of US computer chip giant Qualcomm pleaded guilty Monday to insider trading charges, including trades on a 2011 deal for Atheros Communications, officials said.

Media venture creates press litigation fund

10 hours ago

The media venture created by entrepreneur Pierre Omidyar said Monday it was establishing a fund to help defend journalists in cases involving freedom of the press.

User comments : 0