With wave of the hand, researchers create touch-based interfaces

Apr 25, 2013
Carnegie Mellon University researchers have developed a depth camera/projector system that enables people to create computer interfaces on everyday surface using hand gestures. Credit: Chris Harrison/Carnegie Mellon University

Researchers previously have shown that a depth camera system, such as Kinect, can be combined with a projector to turn almost any surface into a touchscreen. But now researchers at Carnegie Mellon University have demonstrated how these touch-based interfaces can be created almost at will, with the wave of a hand.

CMU's WorldKit system enables someone to rub the arm of a sofa to "paint" a remote control for her TV or swipe a hand across an office door to post his calendar from which subsequent users can "pull down" an extended version. These ad hoc interfaces can be moved, modified or deleted with similar gestures, making them highly personalized.

Researchers at Carnegie Mellon's Institute (HCII) used a ceiling-mounted camera and projector to record room geometries, sense and project images on desired surfaces.

But Robert Xiao, an HCII , said WorldKit does not require such an elaborate installation. "Depth sensors are getting better and projectors just keep getting smaller," he said. "We envision an interactive 'light bulb'—a miniaturized device that could be screwed into an ordinary light fixture and pointed or moved to wherever an interface is needed."

The system does not require prior calibration, automatically adjusting its sensing and image projection to the orientation of the chosen surface. Users can summon switches, message boards, indicator lights and a variety of other interface designs from a menu. Ultimately, the WorldKit team anticipates that users will be able to custom design interfaces with gestures.

Xiao developed WorldKit with Scott Hudson, an HCII professor, and Chris Harrison, a Ph.D. student. They will present their findings April 30 at CHI 2013, the Conference on in , in Paris.

"People have talked about creating smart environments, where sensors, displays and computers are interwoven," said Harrison, who will join the HCII faculty this summer. "But usually, that doesn't amount to much besides mounting a camera up on the ceiling. The room may be smart, but it has no outlet for that smartness. With WorldKit, we say forget touchscreens and go straight to projectors, which can make the room truly interactive."

Though WorldKit now focuses on interacting with surfaces, the researchers anticipate future work may enable users to interact with the system in free space. Likewise, higher resolution depth cameras may someday enable the system to sense detailed finger gestures. In addition to gestures, the system also could be designed to respond to voice commands.

"We're only just getting to the point where we're considering the larger questions," Harrison said, noting a multitude of applications in the home, office, hospitals, nursing homes and schools have yet to be explored.

Explore further: Neuroscientist's idea wins new-toy award

Related Stories

LuminAR bulb lights path to augmented reality (w/ video)

Dec 10, 2012

(Phys.org)—Are we moving closer to a computer age where "touchscreen" is in the room, but it is the counter, desktop, wall, our new digital work areas? Are we moving into a new form factor called Anywhere? ...

Recommended for you

Patent talk: Google sharpens contact lens vision

3 hours ago

(Phys.org) —A report from Patent Bolt brings us one step closer to what Google may have in mind in developing smart contact lenses. According to the discussion Google is interested in the concept of contact ...

Neuroscientist's idea wins new-toy award

Apr 15, 2014

When he was a child, Robijanto Soetedjo used to play with his electrically powered toys for a while and then, when he got bored, take them apart - much to the consternation of his parents.

Land Rover demos invisible bonnet / car hood (w/ video)

Apr 14, 2014

(Phys.org) —Land Rover has released a video demonstrating a part of its Discover Vision Concept—the invisible "bonnet" or as it's known in the U.S. the "hood" of the car. It's a concept the automaker ...

Visions of 1964 World's Fair didn't all come true

Apr 12, 2014

Video phone calls? Yeah, we do that. Asking computers for information? Sure, several times a day. Colonies on the moon and jet packs as a mode of everyday transportation. OK, maybe not.

User comments : 0

More news stories

Patent talk: Google sharpens contact lens vision

(Phys.org) —A report from Patent Bolt brings us one step closer to what Google may have in mind in developing smart contact lenses. According to the discussion Google is interested in the concept of contact ...

Tech giants look to skies to spread Internet

The shortest path to the Internet for some remote corners of the world may be through the skies. That is the message from US tech giants seeking to spread the online gospel to hard-to-reach regions.

Wireless industry makes anti-theft commitment

A trade group for wireless providers said Tuesday that the biggest mobile device manufacturers and carriers will soon put anti-theft tools on the gadgets to try to deter rampant smartphone theft.

ESO image: A study in scarlet

This new image from ESO's La Silla Observatory in Chile reveals a cloud of hydrogen called Gum 41. In the middle of this little-known nebula, brilliant hot young stars are giving off energetic radiation that ...

First direct observations of excitons in motion achieved

A quasiparticle called an exciton—responsible for the transfer of energy within devices such as solar cells, LEDs, and semiconductor circuits—has been understood theoretically for decades. But exciton movement within ...

Warm US West, cold East: A 4,000-year pattern

Last winter's curvy jet stream pattern brought mild temperatures to western North America and harsh cold to the East. A University of Utah-led study shows that pattern became more pronounced 4,000 years ago, ...