Army neuroscientists foresee intelligent agents on the battlefield

The battlefield of the future will be complex, with mountains of data moving rapidly between commanders, operations centers and the joint warfighter. In this multi-faceted environment, Army researchers and their partners are seeking solutions.

The U.S. Army Research Laboratory formed an alliance in 2010 with universities and industry to enable "revolutionary advances" in Soldier systems technology by merging neuroscience, psychology, engineering and human factors to deliver those solutions.

"Humans simply cannot process the amount of information that is potentially available," said Dr. Jonathan Touryan, the collaborative alliance manager and a neuroscientist at the lab's Human Research and Engineering Directorate at APG. "Yet, humans remain unmatched in their ability to adapt in complex and dynamic situation, such as a battlefield environment. We need a greater capability to be able to estimate and predict human variability, behavior and intent across different contexts."

Touryan said the researchers are working on new technologies that will enable more seamless integration with intelligent agents, that take advantage of artificial intelligence and machine learning.

"In theory, intelligent agents will have parallel computational power that is much greater than that of humans," he said. "In developing human-agent integration principles, we hope to accentuate the strengths of both while mitigating individual weaknesses."

The alliance, which will continue for two more years, recently held a biennial research management board where laboratory leadership and their partners looked at progress so far and envisioned the path ahead.

"This Alliance is about taking all that we have learned in the field of neuroscience and neurotechnology and moving that out of the laboratory and putting it in the real-world—Army relevant contexts and real-world environments," Touryan said.

During the review, attendees watched demonstrations of research successes, such as a study to gauge how effective communications can be in a life and death situation. Outfitted with a sensors looking at the body and in particular the brain, a driver and passenger drove up and down a busy expressway while the passenger communicated information.

"What we're interested in doing is understanding whether we can look at the synchrony between the physiologies—the brain response or the heart rate response—between the driver and passenger, and use that synchrony to predict whether the driver is going to remember the information the passenger is telling them after the drive is over," said Dr. Jean Vettel, an Army neuroscientist. "We need a greater capability to be able to estimate and predict human variability, behavior and intent across different contexts. This means people, individuals, environments and groups."

The experiment also used a unique visualization tool developed at the laboratory. Known as CLIVE, or Customizable Lighting Interface for the Visualization of EEGs—or brainwaves—the tool shows how well two brains are synchronized.

"As CLIVE was showing this synchronization across the brain, we know as neuroscientists that certain brain regions are implicated in particular tasks," said Dr. Javier Garcia, an Army neuroscientist. "Whenever it is yellow that means there is high synchrony at that particular brain region between the passenger and driver. Whenever it's blue, there's low synchrony between the passenger and the driver."

With 244 individual LED lights, CLIVE can portray many colors and intensities.

"One of the challenges is that it's often very difficult for us to explain or visualize what's going on inside people's brains to other people who are non-scientists," said Dr. David Hairston, an Army neuroscientist. "This was a creative way that we came up with to show people what's actually going on inside someone's head when they're doing different types of activities."

In another demonstration, a participant with brainwave sensors participated in a computer simulation while researchers introduced external distractions and measured how her brain reacted.

"That is a tremendous challenge because these are very subtle electrical signals that can be swamped by a whole number of other processes physical movement or changes in the environment—and we've developed technologies that enable us to measure these signals, both on the physical sensor side as well as through signal processing," Touryan said.

As a basic science research program with many potential applications, the scientists said it is challenging to know exactly how their research will translate into technology in the future Army.

"Within the current Army mission-space, as autonomous systems become more prevalent, an increasing cognitive burden is going to be placed on the Soldier as they engage with autonomous systems and maintain situational awareness," Touryan said. The more we can understand how the brain processes this information the better we'll understand how to communicate with other intelligent agents."

The alliance, officially known as the Cognition and Neuroergonomics Collaborative Technology Alliance, has seven principal organizations as consortium members and a number of institutions as sub-awardees. They are primarily academic institutions like the University of Florida, the University of California San Diego, Columbia University and the University of Texas, San Antonio as well as international partners and a number of smaller programs with other universities and small businesses.

"Collaboration is, I think, essential because our academic partners have a broad understanding of the scientific problem-space - they have unique insights and resources so they make excellent partners when executing basic science research," Touryan said. "We (ARL) provide complementary expertise and understanding within the Army problem-space and the critical gaps that need to be addressed in our research. Together we can develop combined project plans and research programs that push the science forward in a way that leads to technology transition that will be useful in future Soldier systems."