A 'social' virtual assistant for migrants

June 28, 2018, CORDIS
A ‘social’ virtual assistant for migrants
Credit: Georgejmclittle, Shutterstock

Migrants have been very high on the EU political agenda for the past few years. But far from the political debate, there are cases where technological innovation can truly make a difference. The KRISTINA project has been developing such technologies with a focus on overcoming language barriers.

As great as it sounds on paper, freedom of movement in Europe can pose enormous challenges. Imagine you get a great job opportunity thousands of kilometres away from home, in a Member State you have never even visited. Sure, it's still Europe, but on a daily basis dealing with the administration or telling the doctor about a health problem in a language you don't speak can quickly become insurmountable.

Here, a true 'multilingual conversational agent' capable of understanding and providing the information you need would be a game changer. "The idea would be to develop such an agent with emotive, social and cultural competence. One capable of: (ASR) with linguistic analysis of the transcripts to understand the concern of the speaker; facial and gesture analysis; dialogue planning that doesn't rely on predefined scripts; understanding of emotive signals; multilingual language generation techniques; and featuring advanced virtual character design," says Leo Wanner, ICREA research professor at Pompeu Fabra University specialising in computational linguistics.

At first read, this list of features can seem technologically out of reach, but with some advances in state-of-the-art technology it could be right around the corner. Actually, work under the KRISTINA (Knowledge-Based Information Agent with Social Competence and Human Interaction Capabilities) project has already enabled Prof. Wanner and his team to create the first generation of this conversational agent and run it on tablets and laptops.

The assistant is very simple to use. Once initiated (which requires going through some settings concerning the language, the desired topic of conversation and the like), KRISTINA's virtual agent appears on screen, greets the person wishing to interact with it, and enquires about the possible concerns or questions he/she might have. "Based on the reaction of the person, a conversation develops during which the agent may provide background information on health issues of interest. The agent can provide advice with respect to health, social and other types of activities, offer to read the newspaper to those with visual impairment, etc.," Prof. Wanner explains. KRISTINA can interact with people in German, Polish, Turkish and Spanish.

Whilst not a Big Data project per se, KRISTINA needed to be 'trained' to accurately recognise the emotions of the individuals in the targeted groups, learn the way they speak and interact. This required a considerable amount of data, such as recordings of conversations between individuals of the targeted groups and experts who simulated the agent. The team annotated these recordings with the relevant meta information, and Prof. Wanner says that "a good share" of this resource will be made available to the research community.

Prof. Wanner admits that it's still too soon for the KRISTINA agent to be deployed as an "off-the-shelf" conversational agent that can assist migrants (or another relevant target group) in their everyday lives.

"Further development is needed. Project partners continue to work on the individual technologies – although, unfortunately, not as a consortium due to the lack of common funding. We are working in order to change this situation, and our objective is to expand the KRISTINA agent to enable it to serve as an assistant in other domains," says Prof. Wanner. According to him, KRISTINA should reach maturity in its second or third generation.

Explore further: Star Trek-style communicator hopes to break down cultural barriers

Related Stories

Weather and environmental forecasts tailored to you

October 11, 2013

Want to know what the weather will be like tomorrow? A simple internet search will provide a forecast, or, more likely, many forecasts. How can you be sure which one is accurate? And what if you also want to know the pollen ...

Crowd workers, AI make conversational agents smarter

February 7, 2018

Conversational agents such as Siri, Alexa and Cortana are great at giving you the weather, but are flummoxed when asked for unusual information, or follow-up questions. By adding humans to the loop, Carnegie Mellon University ...

Recommended for you

Inert nitrogen forced to react with itself

March 21, 2019

Constituting over 78 % of the air we breathe, nitrogen is the element found the most often in its pure form on earth. The reason for the abundance of elemental nitrogen is the incredible stability and inertness of dinitrogen ...

Two-step path to shrinking worker bee gonads

March 21, 2019

The dramatic difference in gonad size between honey bee queens and their female workers in response to their distinct diets requires the switching on of a specific genetic program, according to a new study publishing March ...

Plant immunity cut to size

March 21, 2019

An international team based in Ghent, Belgium (VIB-UGent Center for Plant Systems Biology) and Basel, Switzerland (University of Basel), found a link between a class of enzymes and immune signals that is rapidly triggered ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.