New computers respond to students' emotions, boredom

Mar 02, 2012 By Susan Guibert

Emotion-sensing computer software that models and responds to students’ cognitive and emotional states – including frustration and boredom – has been developed by University of Notre Dame Assistant Professor of Psychology Sidney D’Mello and a colleague from Massachusetts Institute of Technology. D’Mello also is a concurrent assistant professor of computer science and engineering.

The new technology, which matches the interaction of human tutors, not only offers tremendous learning possibilities for , but also redefines human-computer interaction.

“AutoTutor” and “Affective AutoTutor” can gauge the student’s level of knowledge by asking probing questions; analyzing the student’s responses to those questions; proactively identifying and correcting misconceptions; responding to the student’s own questions, gripes and comments; and even sensing a student’s frustration or boredom through facial expression and body posture and dynamically changing its strategies to help the student conquer those negative emotions.

“Most of the 20th-century systems required humans to communicate with computers through windows, icons, menus and pointing devices,” says D’Mello, who specializes in human-computer interaction and artificial intelligence in education.

“But humans have always communicated with each other through speech and a host of nonverbal cues such as facial expressions, eye contact, posture and gesture. In addition to enhancing the content of the message, the new technology provides information regarding the cognitive states, motivation levels and social dynamics of the students.”

AutoTutor is an Intelligent Tutoring System (ITS) that helps students learn complex technical content in Newtonian physics, computer literacy and critical thinking by holding a conversation in natural language; simulating teaching and motivational strategies of human tutors; modeling students’ cognitive states; using its student to dynamically tailor the interaction to individual students; answering students’ questions; identifying and correcting misconceptions; and keeping students engaged with images, animations and simulations. In addition to these capabilities, Affective AutoTutor adds emotion-sensitive capabilities by monitoring facial features, body language and conversational cues; regulating negative states such as frustration and boredom; and synthesizing emotions via the content of its verbal responses, speech intonation and facial expressions of an animated teacher.

D’Mello’s study, titled “AutoTutor and Affective AutoTutor: Learning by Talking with Cognitively and Emotionally Intelligent Computers that Talk Back,” that details this new technology will be published in special edition of ACM Transactions on Interactive Intelligent Systems that highlights innovative technology of the last decade.

“Much like a gifted human tutor, AutoTutor and Affective AutoTutor attempt to keep the student balanced between the extremes of boredom and bewilderment by subtly modulating the pace, direction and complexity of the learning task,” D’Mello says.

Considerable empirical evidence has shown that one-on-one human tutoring is extremely effective when compared to typical classroom environments, and AutoTutor and Affective AutoTutor closely model the pedagogical styles, dialogue patterns, language and gestures of human tutors. They are also one of the few ITSs that help learning by engaging students in natural language dialogues that closely mirror human-human tutorial dialogues.

Tested on more than 1,000 students, AutoTutor produces learning gains of approximately one letter grade – gains that have proven to outperform novice human tutors and almost reach the bar of expert human tutors.

Explore further: MIT groups develop smartphone system THAW that allows for direct interaction between devices

add to favorites email to friend print save as pdf

Related Stories

Overcoming boredom

Sep 01, 2011

“I’m bored.” It’s a line that parents likely hear throughout the year. But, as students prepare to head back to classes for the start of a new school year, one University of Alberta researcher says these ...

Your Next Computer May Know How You Feel

Apr 14, 2010

(PhysOrg.com) -- Friends, loved ones and pets can sense your mood almost instantly - and one day your computer may be able to do so pretty quickly as well.

Bored? This is anything but tedious

May 25, 2011

Boredom: Children are quick to distastefully proclaim it and adults are quick to deny it. But University of Calgary Greek and Roman Studies professor Peter Toohey says there is nothing wrong with boredom after all. In fact, ...

Learning: No longer a textbook case

May 02, 2011

Switching from rigid, linear textbooks to technology such as iPads alone won’t boost student performance – so a team of Wake Forest researchers has turned the classroom upside down, allowing students ...

Recommended for you

Computerized emotion detector

Sep 16, 2014

Face recognition software measures various parameters in a mug shot, such as the distance between the person's eyes, the height from lip to top of their nose and various other metrics and then compares it with photos of people ...

Cutting the cloud computing carbon cost

Sep 12, 2014

Cloud computing involves displacing data storage and processing from the user's computer on to remote servers. It can provide users with more storage space and computing power that they can then access from anywhere in the ...

Teaching computers the nuances of human conversation

Sep 12, 2014

Computer scientists have successfully developed programs to recognize spoken language, as in automated phone systems that respond to voice prompts and voice-activated assistants like Apple's Siri.

User comments : 0