'Digits' making your remote control obsolete (w/ Video)

Oct 09, 2012
Power in the palm of your hands

(Phys.org)—Forget the TV remote and the games controller, now you can control anything from your mobile phone to the television with just a wave of your hand.

Researchers at Newcastle University and Microsoft Research Cambridge (MSR) have developed a sensor the size of a wrist-watch which tracks the 3-D movement of the hand and allows the user to remotely control any device.

Mapping finger movement and , it gives the user anytime, anywhere – even allowing you to answer your phone while it's still in your pocket and you're walking down the street...

This video is not supported by your browser at this time.

Being presented this week at the 25th Association for Computing Machinery Symposium on and Technology, 'Digits' allows for the first time 3-D interactions without being tied to any external hardware.

It has been developed by David Kim, a MSR funded PhD from Newcastle University's Culture Lab; Otmar Hilliges, Shahram Izadi, Alex Butler, and Jiawen Chen of MSR Cambridge; Iason Oikonomidis of Greece's Foundation for Research & Technology; and Professor Patrick Olivier of Newcastle University's Culture Lab.

"The Digits sensor doesn't rely on any external infrastructure so it is completely mobile," explains David Kim, a PhD student at Newcastle University.

"This means users are not bound to a fixed space. They can interact while moving from room to room or even running down the street. What Digits does is finally take 3-D interaction outside the living room."

To enable ubiquitous 3-D spatial interaction anywhere, Digits had to be lightweight, consume little power, and have the potential to be as small and comfortable as a watch. At the same time, Digits had to deliver superior gesture sensing and "understand" the human hand, from wrist orientation to the angle of each finger joint, so that interaction would not be limited to 3-D points in space. Digits had to understand what the hand is trying to express—even while inside a pocket.

David adds: "We needed a system that enabled natural 3-D interactions with bare hands, but with as much flexibility and accuracy as data gloves."

The current prototype, which is being showcased at the prestigious ACM UIST 2012 conference today, includes an infrared camera, IR laser line generator, IR diffuse illuminator, and an inertial-measurement unit (IMU) track.

David says: "We wanted users to be able to interact spontaneously with their electronic devices using simple gestures without even having to reach for them. Can you imagine how much easier it would be if you could answer your while it's still in your pocket or buried at the bottom of your bag?"

It's All About the Human Hand

One of the project's main contributions is a real-time signal-processing pipeline that robustly samples key parts of the hand, such as the tips and lower regions of each finger. Other important research achievements are two kinematic models that enable full reconstruction of hand poses from just five key points. The project posed many challenges, but the team agrees that the hardest was extrapolating natural-looking hand motions from a sparse sampling of the key points sensed by the camera.

"We had to understand our own body parts first before we could formulate their workings mathematically," Shahram Izadi explains. "We spent hours just staring at our fingers. We read dozens of scientific papers about the biomechanical properties of the human hand. We tried to correlate these five points with the highly complex motion of the hand. In fact, we completely rewrote each kinematic model about three or four times until we got it just right."

The team agrees that the most exciting moment of the project came when team members saw the models succeed.

"At the beginning, the virtual hand often broke and collapsed. It was always very painful to watch," David explains. "Then, one day, we radically simplified the mathematical model, and suddenly, it behaved like a human hand. It felt absolutely surreal and immersive, like in the movie Avatar. That moment gave us a big boost!"

Both the Digits technical paper being presented at UIST 2012 and accompanying video present interactive scenarios using Digits in a variety of applications, with particular emphasis on mobile scenarios, where it can interact with mobile phones and tablets. The researchers also experimented with eyes-free interfaces, which enable users to leave mobile devices in a pocket or purse and interact with them using hand gestures.

"By understanding how one part of the body works and knowing what sensors to use to capture a snapshot," Izadi says, "Digits offers a compelling look at the possibilities of opening up the full expressiveness and dexterity of one of our body parts for mobile human-computer interaction."

By instrumenting only the wrist, the user's entire hand is left to interact freely without wearing data gloves, input devices worn as gloves, most often used in virtual reality applications to facilitate tactile sensing and fine-motion control. The prototype, whose electronics are self-contained on the user's wrist, optically image the entirety of the user's , enabling freehand interactions in a mobile setting.

Explore further: Jacket works like a mobile phone

Related Stories

Hand prosthetic gives teen new independence

Aug 25, 2011

(PhysOrg.com) -- A 15 year old British girl, Chloe Holmes, has been in the news as being among the youngest in Europe to wear a special prosthetic hand with state of the art bionic fingers. The bionic digits ...

ShakeID tracks touch action in multi-user display

Jun 03, 2012

(Phys.org) -- How do you determine who is doing the touching with a multi-user touch display? Microsoft Research has published a paper that presents a technique for doing so. The researchers make their attempt ...

Gesture recognition

Dec 18, 2008

A system that can recognize human gestures could provide a new way for people with physical disabilities to interact with computers. A related system for the able bodied could also be used to make virtual worlds more realistic. ...

Recommended for you

Jacket works like a mobile phone

20 hours ago

A fire is raging in a large building and the fire leader is sending a message to all firefighters at the scene. But they don't need a mobile phone – they simply check their jacket sleeves and read the message ...

User comments : 2

Adjust slider to filter visible comments by rank

Display comments: newest first

ODesign
not rated yet Oct 09, 2012
nice. I expect other parts can be radically simplified too but will have to wait for a version 2.0. If it can get market share enough to fund further development, look for a durable and pretty version without sharp corners to wear like a large bracelet. then it can go mass market like a wii controller.
alfie_null
not rated yet Oct 10, 2012
It would be cool if this eventually made it to the market as an inexpensive device for remote control, or playing games, or whatever.

Even better if it was made available with an accessible API such that creative people outside the manufacturer could figure out new uses to put the device to.

More news stories

SK Hynix posts Q1 surge in net profit

South Korea's SK Hynix Inc said Thursday its first-quarter net profit surged nearly 350 percent from the previous year on a spike in sales of PC memory chips.

FCC to propose pay-for-priority Internet standards

The Federal Communications Commission is set to propose new open Internet rules that would allow content companies to pay for faster delivery over the so-called "last mile" connection to people's homes.

Brazil enacts Internet 'Bill of Rights'

Brazil's president signed into law on Wednesday a "Bill of Rights" for the digital age that aims to protect online privacy and promote the Internet as a public utility by barring telecommunications companies ...

Is nuclear power the only way to avoid geoengineering?

"I think one can argue that if we were to follow a strong nuclear energy pathway—as well as doing everything else that we can—then we can solve the climate problem without doing geoengineering." So says Tom Wigley, one ...

When things get glassy, molecules go fractal

Colorful church windows, beads on a necklace and many of our favorite plastics share something in common—they all belong to a state of matter known as glasses. School children learn the difference between ...

FDA proposes first regulations for e-cigarettes

The federal government wants to prohibit sales of electronic cigarettes to minors and require approval for new products and health warning labels under regulations being proposed by the Food and Drug Administration.