TapSense touchscreen technology distinguishes taps by parts of finger (w/ video)

October 19, 2011

Smartphone and tablet computer owners have become adept at using finger taps, flicks and drags to control their touchscreens. But Carnegie Mellon University researchers have found that this interaction can be enhanced by taking greater advantage of the finger's anatomy and dexterity.

By attaching a microphone to a , the CMU scientists showed they can tell the difference between the tap of a , the pad of the finger, a fingernail and a knuckle. This technology, called TapSense, enables richer touchscreen interactions. While typing on a , for instance, users might capitalize letters simply by tapping with a fingernail instead of a finger tip, or might switch to numerals by using the pad of a finger, rather toggling to a different set of keys.

Another possible use would be a painting app that uses a variety of tapping modes and finger motions to control a pallet of colors, or switch between drawing and erasing without having to press buttons.

"TapSense basically doubles the input bandwidth for a touchscreen," said Chris Harrison, a Ph.D. student in Carnegie Mellon's Institute (HCII). "This is particularly important for smaller touchscreens, where screen real estate is limited. If we can remove mode buttons from the screen, we can make room for more content or can make the remaining buttons larger."

TapSense touchscreen technology distinguishes taps by parts of finger (w/ video)

TapSense was developed by Harrison, fellow Ph.D. student Julia Schwarz, and Scott Hudson, a professor in the HCII. Harrison will discuss the technology today (Oct. 19) at the Association for Computing Machinery's Symposium on and Technology in Santa Barbara, Calif.

"TapSense can tell the difference between different parts of the finger by classifying the sounds they make when they strike the touchscreen," Schwarz said. An inexpensive microphone could be readily attached to a touchscreen for this purpose. The microphones already in devices for phone conversations would not work well for the application, however, because they are designed to capture voices, not the sort of noise that TapSense needs to operate.

TapSense touchscreen technology distinguishes taps by parts of finger (w/ video)

The technology also can use sound to discriminate between passive tools (i.e., no batteries) made from such materials as wood, acrylic and polystyrene foam. This would enable people using styluses made from different materials to collaboratively sketch or take notes on the same surface, with each person's contributions appearing in a different color or otherwise noted.

The researchers found that their proof-of-concept system was able to distinguish between the four types of finger inputs with 95 percent accuracy, and could distinguish between a pen and a finger with 99 percent accuracy.

Explore further: Wearable depth-sensing projection system makes any surface capable of multitouch interaction (w/ video)

More information: chrisharrison.net/index.php/Research/TapSense

Related Stories

iPhone Software That Controls Robot Movements (w/ Video)

November 18, 2009

(PhysOrg.com) -- At the graduate school of media design at Keio University in Tokyo, a project called "Walky" is under developed. Researchers have developed specifically designed software for the iPhone that can control android ...

TVs, Cell Phones to Learn 'Sign Language'

January 13, 2008

The days of pressing tangible buttons to operate TVs and cell phones may one day be replaced by waving, clapping, and pointing. With two upcoming products, Sony Ericsson´s Z555 gesture phone and JVC´s "Snap & Gesture" TV ...

Recommended for you

A not-quite-random walk demystifies the algorithm

December 15, 2017

The algorithm is having a cultural moment. Originally a math and computer science term, algorithms are now used to account for everything from military drone strikes and financial market forecasts to Google search results.

US faces moment of truth on 'net neutrality'

December 14, 2017

The acrimonious battle over "net neutrality" in America comes to a head Thursday with a US agency set to vote to roll back rules enacted two years earlier aimed at preventing a "two-speed" internet.

FCC votes along party lines to end 'net neutrality' (Update)

December 14, 2017

The Federal Communications Commission repealed the Obama-era "net neutrality" rules Thursday, giving internet service providers like Verizon, Comcast and AT&T a free hand to slow or block websites and apps as they see fit ...

The wet road to fast and stable batteries

December 14, 2017

An international team of scientists—including several researchers from the U.S. Department of Energy's (DOE) Argonne National Laboratory—has discovered an anode battery material with superfast charging and stable operation ...


Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (1) Oct 19, 2011
That's great and all, but the last time I had a smartphone with a resistive touchscreen was like 5 years ago.

Why am I bashing this? Have you ever tried to use your fingernail on a modern capacitive touch screen? Doesn't work, does it?

That leaves us with the finger pad, knuckle (which won't work with low quality capacitive screens), and finger tip touch. I don't know about you but I don't take care to rap my fingers a certain way. And how I touch a screen is very determinate on how I'm holding it. Consider using this this with one hand - I see dropped phones.

While I personally prefer the older resistive screens, I don't see them coming back any time soon because they use more energy.

Which brings us to the second point - using an always on microphone and accompanying processing power. My concern is energy usage, but I'm sure the libertarians will have other concerns as well.

/end rant.
1 / 5 (1) Oct 19, 2011
Just tried using my fingernail on a modern capacitive touch screen. It worked fine for me.
Oct 19, 2011
This comment has been removed by a moderator.
3 / 5 (2) Oct 19, 2011
Just tried using my fingernail on a modern capacitive touch screen. It worked fine for me.

If you have a very good capacitive screen and your nails are trim, it can work as long as your skin gets close enough to it. if you have long nails, like many women do, it won't work at all.

even so, if you try tapping it with only your finger nail all day, you will find it too inconsistent to use on a regular basis.

Pass it to your GF or wife, and she will likely have a much more difficult time than you.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.