Researchers use human cues to improve computer user-friendliness (w/ Video)

Mar 04, 2011
Binghamton University researcher, Lijun Yin, wants computers to understand inputs from humans that go beyond the traditional keyboard and mouse. Credit: Jonathan Cohen

Lijun Yin wants computers to understand inputs from humans that go beyond the traditional keyboard and mouse.

"Our research in and computer vision tries to make using computers easier," says the Binghamton University computer scientist. "Can we find a more comfortable, intuitive and intelligent way to use the computer? It should feel like you're talking to a friend. This could also help disabled people use computers the way everyone else does."

Yin's team has developed ways to provide information to the computer based on where a user is looking as well as through or speech. One of the basic challenges in this area is "." That is, how can a simple webcam work more like the human eye? Can camera-captured data understand a real-world object? Can this data be used to "see" the user and "understand" what the user wants to do?

This video is not supported by your browser at this time.

To some extent, that's already possible. Witness one of Yin's graduate students giving a PowerPoint presentation and using only his eyes to highlight content on various slides. When Yin demonstrated this technology for Air Force experts last year, the only hardware he brought was a webcam attached to a .

Yin says the next step would be enabling the computer to recognize a user's state. He works with a well-established set of six basic emotions — anger, disgust, fear, joy, sadness, and surprise — and is experimenting with different ways to allow the computer to distinguish among them. Is there enough data in the way the lines around the eyes change? Could focusing on the user's mouth provide sufficient clues? What happens if the user's face is only partially visible, perhaps turned to one side?

"Computers only understand zeroes and ones," Yin says. "Everything is about patterns. We want to find out how to recognize each emotion using only the most important features."

He's partnering with Binghamton University psychologist Peter Gerhardstein to explore ways this work could benefit children with autism. Many people with autism have difficulty interpreting others' emotions; therapists sometimes use photographs of people to teach children how to understand when someone is happy or sad and so forth. Yin could produce not just photographs, but three-dimensional avatars that are able to display a range of emotions. Given the right pictures, Yin could even produce avatars of people from a child's family for use in this type of therapy.

Binghamton University researcher Lijun Yin hopes that his work in computer graphics and computer vision will make computers easier to use. Credit: Jonathan Cohen

Yin and Gerhardstein's previous collaboration led to the creation of a 3D facial expression database, which includes 100 subjects with 2,500 facial expression models. The database is available at no cost to the nonprofit research community and has become a worldwide test bed for those working on related projects in fields such as biomedicine, law enforcement and computer science.

Once Yin became interested in human-computer interaction, he naturally grew more excited about the possibilities for artificial intelligence.

"We want not only to create a virtual-person model, we want to understand a real person's emotions and feelings," Yin says. "We want the computer to be able to understand how you feel, too. That's hard, even harder than my other work."

Imagine if a computer could understand when people are in pain. Some may ask a doctor for help. But others — young children, for instance — cannot express themselves or are unable to speak for some reason. Yin wants to develop an algorithm that would enable a computer to determine when someone is in pain based just on a photograph.

Yin describes that health-care application and, almost in the next breath, points out that the same system that could identify pain might also be used to figure out when someone is lying. Perhaps a computer could offer insights like the ones provided by Tim Roth's character, Dr. Cal Lightman, on the television show Lie to Me. The fictional character is a psychologist with an expertise in tracking deception who often partners with law-enforcement agencies.

"This technology," Yin says, "could help us to train the computer to do facial-recognition analysis in place of experts."

Explore further: Coping with floods—of water and data

Provided by Binghamton University

3.7 /5 (3 votes)

Related Stories

How could Santa know if you've been good or bad?

Dec 21, 2009

(PhysOrg.com) -- By using technology to detect guilty expressions, of course. CSIRO is using automated expression recognition technology to tell whether someone is in pain and, according to computer scientist, ...

Avatars as communicators of emotions

Jul 09, 2008

Current interactive systems enable users to communicate with computers in many ways, but not taking into account emotional communication. A PhD thesis presented at the University of the Basque Country puts ...

Your Next Computer May Know How You Feel

Apr 14, 2010

(PhysOrg.com) -- Friends, loved ones and pets can sense your mood almost instantly - and one day your computer may be able to do so pretty quickly as well.

Gesture recognition

Dec 18, 2008

A system that can recognize human gestures could provide a new way for people with physical disabilities to interact with computers. A related system for the able bodied could also be used to make virtual worlds more realistic. ...

Recommended for you

Coping with floods—of water and data

Dec 19, 2014

Halloween 2013 brought real terror to an Austin, Texas, neighborhood, when a flash flood killed four residents and damaged roughly 1,200 homes. Following torrential rains, Onion Creek swept over its banks and inundated the ...

Cloud computing helps make sense of cloud forests

Dec 17, 2014

The forests that surround Campos do Jordao are among the foggiest places on Earth. With a canopy shrouded in mist much of time, these are the renowned cloud forests of the Brazilian state of São Paulo. It is here that researchers ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

jc_french
not rated yet Mar 04, 2011
Would there be an advantage to using asymmetrical data points?

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.