Researchers granted patent for system that fuses human and computer intelligence

August 23, 2013 by Stephanie Koons

In complex crisis situations involving military situation awareness, homeland security and other time-sensitive scenarios, teams of experts must often make difficult decisions within a narrow time frame. However, voluminous amounts of information and the complexity of distributed cognition can hamper the quality and timeliness of decision-making by human teams and lead to catastrophic consequences.

Two professors from Penn State's College of Information Sciences and Technology (IST), John Yen and Michael McNeese, along with Xiacong Fan, an associate professor of computer science and engineering at Penn State Behrend, and Shuang Sun, who received a doctorate from the College of IST in 2006 and is now a project manager at Attune Cytometric Software, have devised a system that merges human and computer intelligence to support decision-making in crisis situations. They were recently awarded a U.S. patent for a collaborative intelligent agent framework that, according to Yen, "finds the sweet spot that combines working in tandem with ."

"I think this patent being granted is not only an encouragement of our research, but it's also timely," he said.

Yen, McNeese, Fan and Sun were awarded U.S. patent number US 8,442,839, "Agent-based Collaborative Recognition-Primed Decision Making." The describes the concept of using a framework, Collaborative Agents for Simulating Teamwork (CAST), originally developed by Yen and his students, and enhancing that with a recognition-primed decision (RPD) model, thereby enhancing analysis through linking and sharing information using knowledge and experience distributed among team members. CAST is a team-oriented agent architecture that supports teamwork using a shared mental model among teammates. R-CAST is the first RPD-enabled agent architecture designed for supporting team-wide collaborations (including human-agent and agent-agent collaborations).

According to Yen et al., there has been little work in looking at cognitive agent architectures (blueprints for intelligent agents that propose computational processes that act like certain cognitive systems, most often, like a person, or acts intelligent under some definition) as a means to assist and support distributed team cognition and decision-making. The approach adopted in the researchers' invention is to develop a cognitive-aware software system that can act as decision aids of human team members in varying ways, including context-sensitive anticipation of others' information needs, proactive information/experience sharing and collaborative situation assessment.

The RPD model is a cognitive model developed by Gary Klein, a research psychologist famous for pioneering in the field of naturalistic decision making, which describes how people make quick, effective decisions when faced with complex situations. In this model, the decision maker is assumed to generate a possible course of action, compare it to the constraints imposed by the situation and select the first course of action that is not rejected. RPD has been used in diverse groups including intensive care unit nurses, fireground commanders, chess players and stock market traders.

In the invention developed by the Penn State researchers, the RPD model is integrated within a CAST architecture to the extent that agents can proactively seek and fuse information to enhance the quality and timeliness of the decision-making process. An interacting team of software agents, each having specialized knowledge and processing capability, analyzes data, forms hypotheses, evaluates alternatives, determines what type of information is needed for a dynamic decision-making environment and supports human analysts.

An example of how of R-CAST is being used to improve upon existing practices, Yen said, is in a project sponsored by the Office of Naval Research for supporting a distributed decision making team for assessing threats and responding to the threats. David Hall, dean of the College of IST, was one of the leading investigators of the project.

"R-CAST automates the proactive exchanges of information relevant to the situation, which is continuously updated as new information arrives," Yen said.

According to Yen, people draw on previous experiences when they need to make decisions in a time-stressed situation. In addition, high-performing teams exhibit some degree of a shared mental model—people on the team have an understanding of the respective roles of each team member and who has what information. The researchers' invention situates the RPD model in a team context, so that team members can infer what their teammates need in a proactive way. The RPD model argues for finding the first workable solution through recognizing the similarity between the current decision situation and previous decision experiences. The model divides a decision-making process into two phases: recognizing which course of action makes sense for the current situation, then evaluating the course of action by imagining it.

"The RPD model acts as an intelligent team partner that is able to share information without overloading people and enhances the quality of information by sharing relevant facts," Yen said.

The invention is particularly valuable to distributed teams of human and software agents analyzing terrorist activities, he said. For example, in the case of individuals being identified as suspects in a terrorist attack, R-CAST could be used to find information on the purchase of bomb materials. The framework is also applicable to enhancing the decision-making of a distributed first responder team, a distributed command and control team involving multiple echelon units in a Net-centric warfare and other domains involving team decision-making under time pressure.

An agent architecture based on collaborative RPD responds to the challenges for decision-making teams in three important ways, the researchers wrote in their patent proposal. First, terrorist detection and investigation rely on the capability of recognizing critical patterns from the current situations. To achieve an improved situation awareness, R-CAST allows agents in a team to anticipate the information needs of the decision-making agent based on their shared mental model, and to proactively seek, fuse and deliver relevant information to the decision-making agent. The information that is shared, Yen said, is driven by the decisions that are made by the team.

"Experience captured directly by analysts can be leveraged," he said. "The key is to determine what is relevant."

Second, agents assisting human analysts in homeland security need to learn expertise from the interactions with human decision makers by generalizing an observed behavior into a knowledge structure that is more reusable. R-CAST can be easily extended to allow agents to learn novel experience in the collaborative story building process.

Finally, since RPD is an abstract model of a human's decision-making process, R-CAST naturally combines the attributes of human cognition with computational intelligence. Through human interface, according to the researchers, "R-CAST achieves a certain degree of adjustable autonomy in making decisions."

"This agent architecture can not only enhance the capabilities of anti-terrorist analysts in identifying terrorist threats, but also pave the way for the next generation of digital assistants that are 'personalized' not only for individuals, but also for teams," they wrote.

Explore further: Researchers advance biometric security

Related Stories

Researchers advance biometric security

June 21, 2012

Researchers in the Biometric Technologies Laboratory at the University of Calgary have developed a way for security systems to combine different biometric measurements—such as eye colour, face shape or fingerprints—and ...

Novel methods to support decision making processes

January 24, 2013

Mathematical analysis based on numbers and numerical estimates is widely used in decision making everywhere from public administration and business to environmental conservation. Methods for decision analysis evaluate different ...

Recommended for you

Nevada researchers trying to turn roadside weed into biofuel

November 26, 2015

Three decades ago, a University of Nevada researcher who obtained one of the first U.S. Energy Department grants to study the potential to turn plants into biofuels became convinced that a roadside weed—curly top gumweed—was ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (2) Aug 23, 2013
When "the degree of autonomy" would mean "being granted >50% of decisionmaking power"?
No, no, no its not just a one dimmension. Just give them more money, and suddenly the whole new stage of such systems emerges, by using for example brain-computer interfaces implanted, or at least external pioneering devices (helmets, glasses, on-eye-lenses, and more: (e.g. to draw representations on sensitive skin areas with some e-wear (e.g. a map on your back)). Then after a while, when quantum computing is ready ... Not separation - no .. why would inhuman intelligence waste biotic resources? Faster then evolving aside, it will be evolving inside and outside.
So cmon hurry up we are just an obstacle, an almost-already-anachronism. Its a shame we are still here.
We have to make room for higher beeings, don't we?
(or google: complexity+allotopy)

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.