The hand as a joystick

May 26, 2011

You need a mouse and a keyboard, a touch-screen or a joystick to control a computer system. In the future, a new gesture command system will make it possible to use just the fingers of a hand.

Before a new vehicle rolls off the assembly lines, it first takes shape as a . In a cave – a room for the virtual representation of objects - the developers look at it from all sides, they »sit« in it, they examine and improve it. For example, are all the switches easy to reach? The developers have so far used a to interact with the computer which displays the virtual car model. In the future, they will be able to do so without such an aid – their alone is intended to be enough to provide the computer with the respective signals.

A multi-touch interface, which h was developed by Georg Hackenberg during his Master's thesis work at the Fraunhofer Institute for Applied Information Technology FIT, made this possible. His work earned him first place in the Hugo Geiger Prizes. »We are using a camera that, instead of providing color information, provides pixel for pixel the distance of how far this point is from the camera. Basically this is achieved by means of a type of gray-scale image where the shade of gray represents the distance of the objects. The camera also provides three-dimensional information that the system evaluates with the help of special algorithms«, explains Georg Hackenberg.

Hackenberg's main work consisted in developing the corresponding algorithms. They ensure that the system is first able to recognize a hand and then able to follow its movements. The result: The 3D camera system processes gestures down to the movements of individual fingers and processes them in real time. Up to this point in time comparable processes with finger support could only detect how hands moved in the image level – they could not solve the depth information, in other words, how far the hand is from the camera system. For this reason it was often difficult to answer with which object the hand was interacting. Is it activating the windshield wipers or is it turning on the radio? Small movements of the hand, such as gripping, have so far been hardly possible to detect in real time – or only with great amounts of computing power. That is no problem for the new system.

commands are also interesting for computer games. A gesture recognition prototype already exists. The researchers want to improve weaknesses in the algorithm now and carry out initial application studies. Hackenberg hopes that the system could be ready for series production within a year, from a technical viewpoint. In the medium term, the researchers hope to further develop it such that it can be used in mobile applications as well, which means that it will also find its way into laptops and cell phones.

Explore further: Revealing faded frescos

Related Stories

Gesture recognition

Dec 18, 2008

A system that can recognize human gestures could provide a new way for people with physical disabilities to interact with computers. A related system for the able bodied could also be used to make virtual worlds more realistic. ...

Cell Phones Using Gesture Control (w/ Video)

Apr 29, 2010

(PhysOrg.com) -- The next generation of cell phone interfaces is currently under development at Ishikawa Komuro Laboratory at the University of Tokyo but instead of using a touchscreen the new interface is ...

Gesture-driven computers

Feb 29, 2008

It isn’t always easy to communicate with a computer. Two Fraunhofer Institutes will be presenting new possibilities of man-machine interaction at CeBIT in Hanover (Germany) on March 4 through 9. They will demonstrate how ...

Recommended for you

Revealing faded frescos

14 hours ago

Many details of the wall and ceiling frescos in the cloister of Brandenburg Cathedral have faded: Plaster on which horses once "galloped" appears more or less bare. A hyperspectral camera sees images that remain hidden to ...

Device could detect driver drowsiness, make roads safer

16 hours ago

Drowsy driving injures and kills thousands of people in the United States each year. A device being developed by Vigo Technologies Inc., in collaboration with Wichita State University professor Jibo He and ...

New capability takes sensor fabrication to a new level

Jun 30, 2015

Operators must continually monitor conditions in power plants to assure they are operating safely and efficiently. Researchers on the Sensors and Controls Team at DOE's National Energy Technology Laboratory ...

Smart phones spot tired drivers

Jun 30, 2015

An electronic accelerometer of the kind found in most smart phones that let the device determine its orientation and respond to movement, could also be used to save lives on our roads, according to research ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.