Can automated deep natural-language analysis unlock the power of inference?

Much of the operationally-relevant information relied on in support of DoD missions may be implicit rather than explicitly expressed, and in many cases, information is deliberately obfuscated and important activities and objects are only indirectly referenced.

Automated, deep natural-language understanding technology may hold a solution for more efficiently processing text information. When processed at its most basic level without ingrained cultural filters, language offers the key to understanding connections in text that might not be readily apparent to humans. DARPA created the Deep Exploration and Filtering of Text (DEFT) program to harness the power of language. Sophisticated artificial intelligence of this nature has the potential to enable defense analysts to efficiently investigate orders of magnitude more documents so they can discover implicitly expressed, actionable information contained within them.

The development of an automated solution may rely on contributions from the linguistics and computer science fields in the areas of artificial intelligence, computational linguistics, machine learning, natural-language understanding, discourse and dialogue analysis, and others.

“Overwhelmed by deadlines and the sheer volume of available foreign intelligence, analysts may miss crucial links, especially when meaning is deliberately concealed or otherwise obfuscated,” said Bonnie Dorr, DARPA program manager for DEFT. “DEFT is attempting to create technology to make reliable inferences based on basic text. We want the ability to mitigate ambiguity in text by stripping away filters that can cloud meaning and by rejecting false information. To be successful, the technology needs to look beyond what is explicitly expressed in text to infer what is actually meant.”

DEFT will build on existing DARPA programs and ongoing academic research into deep language understanding and artificial intelligence to address remaining capability gaps related to inference, causal relationships and anomaly detection.

“Much of the basic research needed for DEFT has been accomplished, but now has to be scaled, applied and integrated through the development of new technology,” Dorr said.

As information is processed, DEFT also aims to integrate individual facts into large domain models for assessment, planning and prediction. If successful, DEFT will allow analysts to move from limited, linear processing of insurmountable quantities of data to a nuanced, strategic exploration of available information.