Games may help train analysts to overcome bias

Nov 13, 2012

Game-playing may help intelligence analysts with the serious business of identifying biases that can cloud decision-making and problem-solving during life or death situations, according to researchers.

Analytic exercises conducted by researchers at Raytheon that used scenario-based games designed by Col. Jacob Graham, senior research associate in the College of and Technology, Penn State, showed that some of the participants displayed anchoring and confirmation biases as they tried to determine responsibility and motivations for insurgent attacks in the scenario. Confirmation bias is the tendency to accept only information that supports current beliefs and attitudes, while anchoring leads people to overemphasize past judgments or initial hypotheses in spite of new, contradictory evidence.

"Biases are often difficult to identify, but it's important to recognize bias in decision theory and analysis," said Graham, who worked with Donald Kretz and B. J. Simpson, both at Raytheon Intelligence and Information Systems. "For decision makers, biases can make the difference in life and death decisions."

The researchers said that the idea to use games as a way to detect confirmation bias was rooted in recent studies that implicated biases in number of failures.

"There have been a lot of post-9/11 studies that looked at analytic tradecraft and intelligence failures," said Kretz. "What they found was that there are a number of significant obstacles to good and thorough , but what gets mentioned over and over in these studies is cognitive bias."

Graham designed a series of games based on real-life situations that U.S. faced in Iraq. He used information from a collection of and reports that he developed to create decision-making games for analysts.

"It's fictional, but it's very real," said Graham. "We are very careful not to give up secrets on how the U.S. operates."

The researchers, who report their findings at the IEEE Conference on Technologies for Homeland Security, today (Nov. 13), said the games used a chain of messages, intercepted phone calls and intelligence reports that offered insights into the activities of insurgent groups in a poor area of Baghdad.

According to Kretz, three groups of nine participants, all engineers at Raytheon, were placed in the middle of an evolving scenario of a series of insurgent attacks. The subjects, who were college educated but not trained in intelligence analysis, had access evidence following each attack and were asked to assess responsibility and motivation after each attack occurred.

The first group analyzed the data by trying to understand the relationships between known and unknown groups of people—link analysis. Another group pulled information and weighed the importance of the data, called information extraction and weighting. Researchers briefed the final group on how to use competing hypothesis to explain reported insurgent acts. Creating alternative hypothesis is one way to avoid confirmation bias.

The team that trained in considering alternative hypotheses significantly outperformed the other teams when identifying the perpetrators and the intended targets, Kretz said.

"We created this exercise to allow analysts to scan through a series of messages and we were able to prove that they had a certain mindset," said Graham.

Kretz said the study could be used in the future as a way to train intelligence analysts to detect and minimize their , as well as develop smarter technology to support analysis.

"This is really about laying the foundation for creating exercises that we can run with working government analysts," said Kretz.

Explore further: Napster co-founder to invest on allergy research

add to favorites email to friend print save as pdf

Related Stories

Intelligence analysts need not fear 'Watson,' study shows

Mar 08, 2011

A Mercyhurst College study on the future of predictive analytics, which examined the outlook for intelligence analysis in the computerized age, shows machines not yet capable of detecting deliberately deceptive data.

Recommended for you

Napster co-founder to invest in allergy research

15 hours ago

(AP)—Napster co-founder Sean Parker missed most of his final year in high school and has ended up in the emergency room countless times because of his deadly allergy to nuts, shellfish and other foods.

LA mayor plans 7,000 police body cameras in 2015

Dec 16, 2014

Mayor Eric Garcetti announced a plan Tuesday to equip 7,000 Los Angeles police officers with on-body cameras by next summer, making LA's police department the nation's largest law enforcement agency to move ...

Merriam-Webster names 'culture' word of the year

Dec 15, 2014

A nation, a workplace, an ethnicity, a passion, an outsized personality. The people who comprise these things, who fawn or rail against them, are behind Merriam-Webster's 2014 word of the year: culture.

In Curiosity Hacked, children learn to make, not buy

Dec 14, 2014

With her right hand, my 8-year-old daughter, Kalian, presses the red-hot soldering iron against the circuit board. With her left hand, she guides a thin, tin wire until it's pressing against both the circuit board and the ...

Top pilot sees risk in unregulated US drones

Dec 10, 2014

Small drones pose a danger in US skies so long as clear rules governing their operation are not in place, the head of the world's biggest pilots' union said Wednesday.

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.