Fujitsu develops task-oriented dialogue technology with AI

Fujitsu Laboratories today announced the development of technology that can be easily set up and autonomously carry on a dialogue, based on AI technology, while accurately understanding a user's request and naturally eliciting the necessary information. The technology is intended primarily for customer service support. With previous technology, dialogue with computers required preparations of dialogue scenarios laying out how to respond when certain things are said, and business systems usually operated based on these scenarios.

Now Fujitsu Laboratories has developed a new technology that can structurally extract the relationships between word meanings of input text to deal with the multiple meanings, ambiguity and other problems particular to Japanese language expressions, enabling a highly accurate understanding of users' speech and realizing smooth dialogue. In addition, by properly incorporating information from external databases, such as linked open data (LOD), while also using a knowledge-based dialogue creation technology that automatically learns response options for natural dialogue from records, Fujitsu Laboratories has developed technology that can autonomously conduct dialogue.

Through these technologies, information service providers can quickly implement a system that suggests recommended products and service plans through a natural dialogue user interface, without preparing complicated scenarios based on their services ahead of time. Moreover, Fujitsu Laboratories carried out a field trial of these technologies on certain customer support tasks for Tokio Marine & Nichido Fire Insurance Co., Ltd. The field trial demonstrated that correct responses could be achieved during a natural conversation. These technologies use Fujitsu Limited's AI technology, Human Centric AI Zinrai. These technologies will be exhibited at Fujitsu Forum 2016, which will be held at the Tokyo International Forum on May 19-20.

Development Background

Currently, messaging applications for users to conduct dialogue with a system are becoming familiar as a method of communication on smartphones. In addition, in order to apply this, there have been a number of public APIs for these message applications to let users converse with systems, and there are increasing expectations that dialogue systems will be implemented for a variety of services. At the same time, in order to use this sort of dialogue system for business applications with a clear goal, unless the system designer is an expert with a certain level of knowledge, it was impossible to create dialogue scenarios that could easily obtain the necessary information from the user.

Issues

With previous technology, in order to have a dialogue with a computer, it was necessary to prepare dialogue scenarios laying out how to respond when certain things are said, and business systems usually operated based on these scenarios. For these types of scenarios, it was necessary to record expected user statements in advance. This meant that, because it was necessary to also think of scenarios to return the dialogue to its goal if the user made an off-topic statement, and because it was necessary to prepare these scenarios for each task or service, the time required to build and implement dialogue systems was an issue.

Newly Developed Technology

Now Fujitsu Laboratories has developed task-oriented dialogue technology that can elicit desired information through an autonomous dialogue with the user, without preparing the complicated scenarios that were necessary with previous systems, and which can suggest recommended products and service plans. The key features of these technologies are as follows.

1. Statement comprehension technology extracts information necessary for a task from user statements with high accuracy

In Japanese, where word order is flexible and there is a lot of variation of expressions, it is more difficult to automatically extract the meaning of a sentence with high accuracy compared with English, where clues such as prepositions can be used. Accuracy remains low on a keyword basis only. In response to these problems, Fujitsu Laboratories developed a new, proprietary speech analysis technology that can structurally analyze the relationship between the meanings of words, using analysis technology from its machine translation engine which includes a large-scale dictionary with millions of words, and can easily and correctly extract necessary information. For example, in the case of a theoretical travel business, when a user inputs a sentence, such as, "I'm going with a friend to Tokyo, where my sister is," the system can correctly understand that the user will be accompanied by the friend, not the sister. In addition, using technology that determines the meaning of user statements, based on machine learning technology, developed jointly with the Inui-Okazaki laboratory at the Graduate School of Information Sciences at Tohoku University, this system has become capable of keeping up a smooth conversation to elicit the desired information while correctly grasping meaning, including questions, answers, requests, intent and desire. For example, the system might determine that the statement, "I had wanted to go to Hawaii," is expressing a wish, and is not the user's travel destination, and respond with, "Hawaii? That would be nice. So, where are you off to today?" This response continues the dialogue in order to get information on the destination. Using these technologies, in an evaluation trial based on a travel business, this system was able to greatly improve its recognition rate, from 67% to 91%, in extracting the travel destination compared with previous keyword methods, providing a smoothly flowing conversation.

2. Knowledge-based dialogue creation technology

When executing such tasks as promoting products or suggesting service plans, there is a tendency to use dialogue scenarios that blandly ask about the information necessary for the task. With this newly developed knowledge-based dialogue creation technology, however, it is possible to appropriately include tens of millions of data points of information using external databases, such as LOD, which makes it easy to build a natural and friendly dialogue system. Moreover, Fujitsu Laboratories used automated learning which makes it possible to build a system that can hold smooth and natural conversations without creating complicated scenarios.

Effects

Fujitsu Laboratories carried out a field trial of these technologies on certain customer support tasks of Tokio Marine & Nichido Fire Insurance Co., Ltd., demonstrating that they can respond correctly during a natural dialogue. These technologies eliminate the need to prepare scenarios for each service, so it is now possible to quickly and easily add a dialogue user interface to a variety of services, and it can be expected they will be used in a variety of services. Users can also expect that, because they will be able to access these services through the messaging applications they are used to using, ICT services will become even more convenient and familiar.

Future Plans

Fujitsu Laboratories will move forward on validation trials of this technology, and aims to implement it in Fujitsu Limited's solutions in fiscal 2016. Moreover, Fujitsu Laboratories will work to expand the applicable areas for these technologies to a broad variety of services, such as customer service, sales, call centers and virtual assistants, as new AI technologies for Zinrai's decision making and support functions.