Writing without keyboard: Handwriting recognition on the wrist

Feb 21, 2013

Typing text messages on the mobile phone via the tiny soft keyboard is very cumbersome. How about simply writing it into the air! This idea drove the development of "airwriting" made by computer scientists of Karlsruhe Institute of Technology. Sensors attached to a glove record hand movements, a computer system captures relevant signals and translates them into text. For the further development of mobile interfaces based on these ideas Professor Tanja Schultz and Dipl.-Inform. Christoph Amma received the "Google Faculty Research Award."

The system offers a new interface for wearable computing applications. These are computer systems worn like clothes, which can be integrated seamlessly into the everyday life of the user. "Information technology is used any time and anywhere, but smart phones are still working on the basis of virtual keyboards and small screens. However, allow for new types of input – in particular for mobile devices or devices integrated in clothing.

"The interaction is embedded seamlessly in everyday life," says Christoph Amma, who has developed the system at the Cognitive Systems Lab (CSL) of KIT. "The airwriting glove is used to write letters into air, as if using an invisible board or pad." For this purpose, and are attached to the thin glove. Contrary to systems based on cameras, these sensors are very small, mobile, and robust. They record the movements of the hand and transmit them to a via a wireless connection. The computer system first checks whether the user is indeed writing. "All movements that are not similar to writing, such as cooking, doing laundry, waving to someone, are ignored. The system runs in the background without interpreting every movement as computer input," says Amma. The computer scientist thinks that the device is perfectly suited for future mixed-reality applications. For instance, in glasses with integrated miniaturized screens, news may be displayed to the user in the field of vision. "When such a system is combined with the possibility to input commands and texts by gestures, you do not even need a hand-held device," emphasizes Amma.

During writing, the airwriting system decodes the letters by a pattern recognition method. Previous approaches mainly focused on the recognition of single gestures assigned to certain commands. Amma's approach goes far beyond: For every letter of the alphabet, a statistical model of the characteristic signal pattern is stored. It also takes into account individual differences in the writing style. The system can recognize complete sentences written in capital letters and presently has a vocabulary of 8000 words. "The system has an error rate of 11%. When it is adapted to the individual writing style of the user, the error rate drops to 3%," Christoph Amma says.

Scientists are now working on further refining the method to filter out writing. Moreover, they want to make the system smaller in order to increase wearing comfort and user acceptance. "This can be achieved with commercial components. An unobtrusive wrist band might be feasible, for example," says Christoph Amma. It is also planned to integrate the system in . In this case, neither the wrist band nor the tiny soft keyboard would be required to writea text message.

The Faculty Research Awards Program supports academic research in computer science, engineering, and related fields. Through the program promotes interaction with universities, the award supports roughly the cost of one graduate student for one yeark. Prior to the Google award, Christoph Amma, Professor Tanja Schultz, and Marcus Georgi received the best paper award at last year's "International Symposium on Wearable Computers". For his initial prototype Christoph Amma was awarded the Plux Wireless Biosignals Prize in 2011.

Explore further: MIT groups develop smartphone system THAW that allows for direct interaction between devices

More information: csl.anthropomatik.kit.edu/

add to favorites email to friend print save as pdf

Related Stories

Gesture recognition

Dec 18, 2008

A system that can recognize human gestures could provide a new way for people with physical disabilities to interact with computers. A related system for the able bodied could also be used to make virtual worlds more realistic. ...

New system helps deafblind people to communicate

Jan 15, 2013

A team of researchers from the Alcoy campus of the Universitat Politècnica de València, together with the companies Innovatec and Indra, has designed a new device to help deafblind people to communicate ...

Glove designers plan messaging path for deaf-blind

Apr 07, 2012

(Phys.org) -- People coping with the double absence of vision and hearing can communicate via mobile devices with the help of a special glove, now under development in Germany’s Design Research Lab. The ...

Google patent sends ring signals to Project Glass

May 19, 2012

(Phys.org) -- Google's September 2011 patent that was filed for a wearable display device was granted this week, which suggests that its envisioned heads-up display device can be controlled by infrared markers ...

Google patent calls on eyes to unlock smart glasses

Aug 09, 2012

(Phys.org) -- In patent language, the application granted to Google this week says simply “Unlocking a screen using eye tracking information” but the message seems clear enough that Google plans ...

Recommended for you

Who drives Alibaba's Taobao traffic—buyers or sellers?

20 hours ago

As Chinese e-commerce firm Alibaba prepares for what could be the biggest IPO in history, University of Michigan professor Puneet Manchanda dug into its Taobao website data to help solve a lingering chicken-and-egg question.

Computerized emotion detector

Sep 16, 2014

Face recognition software measures various parameters in a mug shot, such as the distance between the person's eyes, the height from lip to top of their nose and various other metrics and then compares it with photos of people ...

Cutting the cloud computing carbon cost

Sep 12, 2014

Cloud computing involves displacing data storage and processing from the user's computer on to remote servers. It can provide users with more storage space and computing power that they can then access from anywhere in the ...

Teaching computers the nuances of human conversation

Sep 12, 2014

Computer scientists have successfully developed programs to recognize spoken language, as in automated phone systems that respond to voice prompts and voice-activated assistants like Apple's Siri.

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

B__
5 / 5 (1) Feb 21, 2013
After years of using a keyboard those fine motor skills used to draw and write are almost entirely gone. My hand-written block letters are hardly readable by me (let alone a machine!). I expect I'm not the only one who uses a keyboard more than any form of writing. For us, going back to writing is likely to be *less efficient* because of the difference in motor skills. Now if you were to "type" in the air, that may capitalize on the motor skills we have refined over the years.

Of course we can eventually adapt to any interface, but I think the crucial aspect of a physical interface is stability and biological compatibility (lack of RSI risk).