Crowd mentality

Sep 21, 2011 By Heather Wuebker
“As a group, people are collectively more intelligent than you might think,” says professor Mark Steyvers. “We all possess knowledge about different things, some more than others. If we pool these pieces of knowledge together, we can get a pretty accurate look at the big picture.” Credit: iStockphoto.com / Francesco Santalucia

Do you know what the price of gas will be in six months? How about the extent of the expected U.S. troop drawdown in Afghanistan by year’s end? As an individual, predicting such things with any degree of confidence may seem the stuff of science fiction, but UC Irvine cognitive scientists say it’s possible.

“As a group, people are collectively more intelligent than you might think,” says professor Mark Steyvers. “We all possess knowledge about different things, some more than others. If we pool these pieces of knowledge together, we can get a pretty accurate look at the big picture.”

It’s called the “” effect, and researchers are hoping to get your help putting it to the test.

Steyvers and fellow UCI cognitive sciences professors Michael Lee and William Batchelder are part of a national team given a grant by the Intelligence Advanced Research Projects Activity to develop statistical models based on the crowds concept that can, with increasing precision, forecast the future.

Working with the UCI trio are scientists from the University of Maryland, the University of Michigan, The Ohio State University, Fordham University, Wake Forest University, Wichita State University and private company Applied Research Associates.

They constitute one of five national teams to be funded by IARPA and must meet yearly goals to continue receiving support from the government agency.

In July, the UCI/multi-university team launched its models via a software program called Forecasting Ace. Through volunteer participation, the program will collect individual opinions on the likelihood of certain events within a specified time frame.

Before providing input, contributors must complete a short questionnaire in which they rate their subject-matter expertise in areas including science and technology, business and the economy, politics and policy, military and security, and sports and health.

After the self-assessment, participants can choose which questions about the future they’d like to answer. The software then aggregates and analyzes the collected data using models of human decision making created by the UCI scientists.

“Finding wisdom in crowds is fundamentally a cognitive science problem, because it’s about how people acquire knowledge and make judgments,” says Lee. The concept, while relatively recent, is similar to cultural consensus theory, which Batchelder has been developing for 30 years.

The newer computer and cognitive aspects draw on the expertise of Lee and Steyvers, who have been perfecting these modeling methods at UCI over the past four years through numerous lab experiments and analyses of real-world behavioral data. The professors, along with two student researchers, won a best-paper prize for this work at the July conference of the Cognitive Science Society.

The qualitative and quantitative results from these various studies have helped them construct novel ways of identifying experts by combining self-reported expertise levels and behavioral responses in their models.

As the time frame for a forecasting challenge – such as the likelihood of federal legislators agreeing to raise the U.S. debt ceiling by Aug. 2 – draws near, the UCI team runs the collected data through the models. During this process, “ace analysts” – true experts in their field – are identified, and their opinions are appropriately weighted, helping researchers derive a relatively sound prediction of the upcoming event’s outcome.

They currently don’t know a prediction’s degree of accuracy until the event does or does not occur, but continued data aggregation lets them hone the models to account for more complex and precise forecasting.

The resulting applied project has implications for government agencies interested in improving the accuracy of intelligence analysis. It may also have applications in the fields of business, medicine and policy.

Funding for the endeavor began in May and runs through April 2012, at which time it may be extended. During that period, the research program will strive to achieve a 20 percent increase in forecast accuracy compared with state-of-the-art alternatives. Subsequent years will aim for growth of 35, 50 and more percentage points.

Explore further: Researchers urge early help for kindergarten students with low self-regulation

add to favorites email to friend print save as pdf

Related Stories

Now you see him

Mar 08, 2011

Imagine if Harry Potter’s cloak were real , or that you could blot out the sight of something as easily as pressing a mute button to eliminate sound. To some, this seems like “pi in the sky” ...

Consumer confidence hits five-year high in Michigan

Oct 27, 2010

(PhysOrg.com) -- Despite Michigan’s continued economic malaise, residents’ optimism about the future is at its highest in nearly five years, according to Michigan State University’s latest State of the State ...

Recommended for you

World population likely to peak by 2070

23 hours ago

World population will likely peak at around 9.4 billion around 2070 and then decline to around 9 billion by 2100, according to new population projections from IIASA researchers, published in a new book, World Population and ...

Bullying in schools is still prevalent, national report says

23 hours ago

Despite a dramatic increase in public awareness and anti-bullying legislation nationwide, the prevalence of bullying is still one of the most pressing issues facing our nation's youth, according to a report by researchers ...

Study examines effects of credentialing, personalization

Oct 23, 2014

Chris Gamrat, a doctoral student in learning, design and technology, recently had his study—completed alongside Heather Zimmerman, associate professor of education; Jaclyn Dudek, a doctoral student studying learning, design ...

Data indicate there is no immigration crisis

Oct 22, 2014

Is there an "immigration crisis" on the U.S.-Mexico border? Not according to an examination of historical immigration data, according to a new paper from Rice University's Baker Institute for Public Policy.

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

Corban
not rated yet Sep 21, 2011
The wisdom of crowds only works when other people do not know what others are thinking. Otherwise they start to align and make systematic mistakes en masse. See: Herd mentality or lemmings.