Team develops award-winning interactive agent system

January 28, 2016
In the jungle adventure, the system simulates a survival scenario in which the player interacts with the ECA, Adriana. In order to survive, both human and ECA must collaborate, cooperate and build a relationship. Credit: UTEP

University of Texas at El Paso Professor of Computer Science David Novick, Ph.D., and his students have developed a system for virtual agents and an immersive interactive application titled "Survival on Jungle Island."

The UTEP AGENT system was created to study understanding between embodied conversational agents (ECAs) and humans, with emphasis on the effects of paralinguistic behaviors on engagement and rapport. Paralinguistics includes behaviors such as gesture, intonation and rhythm in speech, gaze and turn taking. The team's research includes work on personality traits of humans and agents.

In "Survival on Jungle Island," the ECA and a human interact, using speech and gesture, in a 40-60 minute adventure composed of 23 scenes. A study conducted with the adventure showed that rapport increases when the ECA asks the human to perform task-related gestures and then perceives a human performing these gestures.

In the jungle adventure, the system simulates a survival scenario in which the player interacts with the ECA, Adriana. In order to survive, both human and ECA must collaborate, cooperate and build a relationship.

The game begins with a cinematic sequence of a stormy shipwreck, providing background information as to how the person ended up in the survival situation. This scene is followed by another two scenes, which serve as an inconspicuous tutorial to avoid breaking the impression of an immersive reality. The first scene contains personal questions that let the player and the ECA get to know each other, giving the impression that Adriana is processing what she hears. The second scene moves the story forward by providing an explanation as to how the ECA Adriana has survived in the jungle so far, and shows that she reacts to the player's responses. At the end of the scene, the story continues to develop depending on the player's choice of what to do next.

Novick created the system with the help of his Advanced aGent ENgagement Team (AGENT), comprising postdoctoral fellow Ivan Gris; UTEP undergraduate and graduate students Adriana Camacho, Alex Rayon, Joel Quintana, Anuar Jauregui, Timothy Gonzales, Alfonso Peralta, Victoria Bravo, Jacqueline Brixey, Yahaira Reyes and Paola Gallardo; French Air Force Academy cadets Guillaume Adoneth and David Manuel; and El Paso high school students Brynne Blaugrund and Nick Farber.

To enable the interaction between humans and ECAs, the team combined the use of Unity 4, an animation ; a Microsoft Kinect motion sensing device; and the Windows Speech SDK software. The team also built "middleware," which enabled more rapid development of applications including the jungle adventure.

The UTEP AGENT system recently received the award for Outstanding Demonstration at the 17th ACM International Conference on Multimodal Interaction held in November in Seattle, Washington.

"UTEP's virtual-agent team now ranks among the best in the world," Novick said. "We are both building exciting new virtual-agent technology and learning how to make agents more adaptable to humans."

Explore further: Research shows cue-giving robots help students learn

Related Stories

Research shows cue-giving robots help students learn

May 30, 2012

(Phys.org) -- The well-known fact is that humans can teach robots, but the newer turn in educational circles is all about how robots can teach humans. The stepped-up robots are “animated” and "adaptive" agents that ...

Study links air pollution to children's low GPAs

August 27, 2015

A University of Texas at El Paso (UTEP) study on children's health has found that fourth and fifth graders who are exposed to toxic air pollutants at home are more likely to have lower GPAs.

Using virtual reality to make experiments more realistic

January 27, 2016

Avatars are all around us: they represent real people online and colonise new worlds in the movies. In science, their role has been more limited. But avatars can be extremely useful in linguistics, new research shows. Scientists ...

Recommended for you

Where can I buy a chair like that? This app will tell you

August 23, 2016

If you think you have a knack for interior design, or just want to spruce up your own home, new technology developed by Cornell researchers may help you choose furnishings the way professionals do. And professionals may find ...

Sponge creates steam using ambient sunlight

August 22, 2016

How do you boil water? Eschewing the traditional kettle and flame, MIT engineers have invented a bubble-wrapped, sponge-like device that soaks up natural sunlight and heats water to boiling temperatures, generating steam ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.