Human-computer interactions could be improved by a new efficient and accurate hand-gesture-recognition model

January 22, 2014
Gesture recognition: Gaining the upper hand
Using raw data from single depth images, a three-step modelling process provides quick, accurate estimations of hand poses for use in numerous applications such as robotics and ergonomics. Credit: C. Xu and L. Chang

Virtual reality, gaming and robotics rely on computers using depth images from cameras to recognize, understand and replicate actual body movements. Programming a computer to accurately identify hand gestures is particularly challenging because of the speed, complexity and dexterity of human hand movements.

Devices such as the popular Microsoft Kinect provide both video images and depth information. Processing software must combine these data to build up a three-dimensional (3D) representation of , but this can be computationally expensive and is not always accurate.

Now, Li Cheng and Chi Xu at the A*STAR Bioinformatics Institute in Singapore have successfully trialed a capable of estimating hand movements and poses using only raw depth data, thus allowing for faster processing. The method appears robust and at times more accurate than high-end programs.

"Hand pose estimation is a difficult problem," states Cheng. "For example, fingers are often clustered together and tend to obstruct each other. Recent programs are still not sufficiently reliable—3Gear, for example, only holds six gestures in its database and ignores all other possibilities. Our new method provides quick results for a large number of different poses."

The researchers trained their model using synthetic images of hand gestures before applying the model to realistic low-resolution, noisy depth images, as typically found in Kinect-type systems. They applied a feature classification technique called a Hough transform to the data, which uses a forest of points to locate the hand and verify the likely orientation of the palm and fingers.

"As the hand can easily roll sideways—a movement not seen in other parts of the body—we used two further steps in the process to improve 3D accuracy," explains Cheng. Rotating a whole image is computationally costly; instead, the researchers applied a second Hough 'forest' with specially modified depth features to account for possible hand rotation. This step produced a list of probable hand poses from which the best fit was chosen (see image).

Unlike previous programs where the palm of the hand is assumed to be flat, the new model is capable of simulating the arch of the palm. This significantly helps in accurately representing gestures. Tests using real hand movements demonstrated that the model could deliver fast and precise results.

The researchers hope their model will provide more efficient and accurate hand pose estimations in computer software. In the future, they also aim to include scenarios that involve hands interacting with physical objects.

Explore further: The hand as a joystick

More information: Xu, C. & Cheng, L. Efficient hand pose estimation from a single depth image. International Conference on Computer Vision, 3–6 December 2013.

Related Stories

The hand as a joystick

May 26, 2011

You need a mouse and a keyboard, a touch-screen or a joystick to control a computer system. In the future, a new gesture command system will make it possible to use just the fingers of a hand.

Microsoft hand research ripens Kinect for work (w/ video)

March 7, 2013

( —Beyond reading body motions, Kinect is getting a workup by researchers at Microsoft, now showing substantial control additions. Microsoft Research this week showed how Microsoft Kinect for Windows may be used ...

Recommended for you

Internet giants race to faster mobile news apps

October 4, 2015

US tech giants are turning to the news in their competition for mobile users, developing new, faster ways to deliver content, but the benefits for struggling media outlets remain unclear.

Radio frequency 'harvesting' tech unveiled in UK

September 30, 2015

An energy harvesting technology that its developers say will be able to turn ambient radio frequency waves into usable electricity to charge low power devices was unveiled in London on Wednesday.

Professors say US has fallen behind on offshore wind power

September 29, 2015

University of Delaware faculty from the College of Earth, Ocean, and Environment (CEOE), the College of Engineering and the Alfred Lerner School of Business and Economics say that the U.S. has fallen behind in offshore wind ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.