Carnegie Mellon group shows iPad skeuomorphism

May 04, 2014 by Nancy Owano weblog

(Phys.org) —The Human Interfaces Group at Carnegie Mellon, led by the group's director Chris Harrison, an assistant professor of Human Computer Interaction, have done work that shows how traditional hand movements to perform tasks such as measuring and erasing can be naturally applied to the digital screen, improving on a natural interaction with computers. They have come up with TouchTools, a gesture design approach. With TouchTools, you manipulate tools on the screen just as you would in real life; the idea is to make software more natural to use. They are showing the world their TouchTools concept, which Gizmodo recently referred to as "skeuomorphism applied to interaction design."

Harrison said, "The core idea behind TouchTools is to draw upon user familiarity and motor skill with tools from the real world, and bring them to interactive use on computers."

The same hand gestures that you would use, for example, to run a tape measure across an item in the real world becomes the same hand gesture you use with a virtual, realistic-looking tape measure across the screen. Grab the virtual tape measure appearing on the screen, and you can roll it out, or move a market to draw, or press a camera shutter button.

Using TouchTools, users replicate a tool's corresponding real-world grasp and press it to the screen as though it was physically present. The system recognizes this pose and instantiates the virtual tool as if it was being grasped at that position. Users can rotate and manipulate the tool as they would its physical counterpart. But what's wrong with touchscreen interactions today? Why bother? The team's argument is that hand movements can be cumbersome in today's interactive environments and their approach is more natural.

This video is not supported by your browser at this time.

"Contemporary applications often expose a toolbar that allows users to toggle between modes (e.g., pointer, pen, eraser modes) or require use of a special physical , such as a stylus. TouchTools can utilize the natural modality of our hands, rendering these accessories superfluous."

The way a computer user has to chord the fingers is not natural, as we do not normally perform actions in the real world by movements with certain numbers of fingers, as we do for screen tasks. Yet, as the group noted, the average person can skillfully manipulate a plethora of tools, from hammers to tweezers, with human hand actions. Despite this remarkable natural dexterity, gestures on today's touch devices are simplistic, relying primarily on the chording of fingers: one-finger pan, two-finger pinch, four-finger swipe.

"We propose that touch gesture design be inspired by the manipulation of physical tools from the real world. In this way, we can leverage user familiarity and fluency with such tools to build a rich set of gestures for touch interaction." This is what the team said in their paper, "TouchTools: Leveraging Familiarity and Skill with Physical Tools to Augment Touch Interaction," prepared for CHI 2014, which took place in Toronto from April 26 to May 1. They reported results after recruiting participants, and an iPad was used. Participants were given physical versions of various test tools to handle. "With only a few minutes of training on a proof-of-concept system, users were able to summon a variety of virtual tools by replicating their corresponding real-world grasps." The authors said they believe that "designing gestures around real-world tools improves discoverability, intelligibility and makes gestures memorable."

This video is not supported by your browser at this time.

One might contend, why go to this effort when a tablet is so easy to use? Would TouchTools be seen as a lot of unnecessary bother? Arguments for the use of skeuomorphism in digital design involve ease of use. Digital emulation with objects and the way humans use them provide easy familiarity.

Gizmodo commented that what the Carnegie Mellon team is trying to convey is, "the library of interactions we currently use is quite thin—and by looking at the world around us, and the 'natural modality' of our 10 fingers, designers might find unexpectedly smart solutions to digital problems."

To be sure, the authors of the TouchTools paper, Chris Harrison, Robert Xiao, Julia Schwarz and Scott E. Hudson, stated that "We hope this work offers a new lens through which the HCI community can craft novel touch experiences."

Explore further: Carnegie Mellon system lets iPad users explore data with their fingers

More information: TouchTools: chrisharrison.net/index.php/Research/Touchtools

Research paper: Harrison, C., Xiao, R., Schwarz, J., and Hudson, S. TouchTools: Leveraging Familiarity and Skill with Physical Tools to Augment Touch Interaction. In Proceedings of the 32nd Annual SIGCHI Conference on Human Factors in Computing Systems (Toronto, Canada, April 26 - May 1, 2014). CHI '14. ACM, New York, NY. dx.doi.org/10.1145/2556288.2557012

add to favorites email to friend print save as pdf

Related Stories

Holodesk prototype puts life in computers (w/ video)

Oct 20, 2011

(PhysOrg.com) -- A research project at Microsoft Research Cambridge has brought forth a prototype called Holodesk, which lets you manipulate virtual objects with your hand. You literally "get your hands on" ...

Recommended for you

Mercedes-Benz 2025 truck shows autonomous system vision

Sep 30, 2014

Coming down the road will be the future truck of 2025 and it will be able to move autonomously. The Mercedes-Benz Future Truck 2025 was recently showcased at this year's International Commercial Vehicle show ...

Body cameras join cops on the beat in US

Sep 30, 2014

For a growing number of police officers in the United States, miniature video cameras are becoming as much a part of the uniform as a pistol, handcuffs and badge.

User comments : 3

Adjust slider to filter visible comments by rank

Display comments: newest first

Code_Warrior
not rated yet May 04, 2014
I wonder how their application can tell the difference between an erase gesture and a scribble gesture? Do they always assume that a scribble gesture means erase when there are annotations present in the area of the scribbling? How does it tell the difference between the gesture to activate the camera and the large eraser? It looked like many of the gestures were similar. Their demonstration works, but how often does it pick the wrong tool?

It's a neat idea, but the tools still looked cumbersome to me and the tool menu at least has the advantage of correctly interpreting the user's intentions. Still, it's worth continuing research work.
box-of-tricks
not rated yet May 04, 2014
And it's got nothing to do with iPads other than they used one for the trial. When are people going to stop saying iPad when they really mean any tablet?
Anda
not rated yet May 05, 2014
Hahaha. Thank you "box" haha

They say ipad because they used an ipad, moron