National crowdsourcing project to better predict world events

A new model for crowdsourcing predictions called Aggregative Contingent Estimation System (ACES) is transforming the way future events are forecast – combining the collective knowledge of many individual opinions in a unique way that improves accuracy beyond what any one person or small group of experts could provide.

Intelligence agencies have historically relied on experienced specialists to provide forecasts on world events. ACES recruits everyday citizens as research participants to help improve the accuracy of forecasting methods for intelligence analysis.

Sponsored by IARPA, an activity within the Office of the Director of National Intelligence, built by Applied Research Associates and supported by a team of academic partners such as Wake Forest University, ACES allows volunteer participants online to identify topics of interest, engage with other experts and compete to make the best predictions.

Eric Stone, an associate professor of psychology at Wake Forest who studies judgment and decision making, is part of the team working on this national project to find ways to help experts make more accurate predictions about world events.

With the goal of creating a more powerful “prediction engine,” for forecasting everything from the price of gas in the U.S. to the nuclear capabilities of Iran, Stone’s research team is looking for individuals to contribute their knowledge in topic areas such as politics, the military, economics, science and technology, and social affairs.

“This kind of enhanced predictive analysis capability will help the intelligence community provide early warning and leading indicators of events,” said Stone, who has published studies on the nature of expertise and how to make well-calibrated, non-overconfident judgments. “The idea is to combine individual judgments from a lot of people who all know a little to provide a tremendous amount of information.  Unlike the current process, our procedure captures, shares and combines expert opinions in a manner to make forecasts as accurate as possible.”

The ACES process provides opportunities for anonymous sharing and deliberating before making forecasts and will provide participants feedback on their contributions and predictive successes.

According to ACES Principal Investigator Dr. Dirk Warnaar, “Some people know important details that can make the future predictable in many cases. They often do not share their insight with others. Our project is designed to find out what people know, have them share this knowledge with others, and ask them to make a prediction based on what they and others know.”

Provided by Wake Forest University