New research system uses social media and other tools to gather, analyze expert opinions

Jun 14, 2011

Researchers have developed a new method of eliciting and analyzing opinions from a large group of experts and laypeople to aid complex decision-making, adapting online and social media technologies to lower the cost of such activities while expanding the types of people who can be queried.

The system, called ExpertLens, incorporates elements of such well-known approaches as the Delphi method, the Nominal Group Technique and that are used to collect opinions about problems or to create forecasts. The online system and the associated have performed well during early tests, according to findings published online by researchers from the RAND Corporation in the journal Technological Forecasting & Social Change.

Developers say ExpertLens could have wide application across such areas as public policy, health care, finance and marketing, where expert panels are frequently used to help solve problems or predict an unknown future.

"Expert panels have long been used to pursue research across a broad area of policy," said Siddhartha Dalal, the study's lead author and chief technology officer at RAND, a nonprofit research organization. "This new system allows expert panels to be done online in a robust way that resembles fact-to-face meetings, but with lower costs and easier analysis of the information gathered."

Leaders who are examining complex policy issues frequently need to elicit opinions from large and diverse groups of stakeholders with broad and diverse sets of expertise.

Existing options for gathering such opinions generally include convening meetings of experts where opinions are expressed face to face (the Nominal Group Technique), organizing panels of experts who share their opinions without meeting in person (the Delphi method) and putting out an open call for input to a large community of people (the crowdsourcing method).

Each of the approaches has certain strengths and weaknesses, according to researchers. Face-to-face meetings can be expensive and difficult to organize. In addition, such efforts usually are limited to small groups of people with narrow areas of specialization and can become dominated by a small number of strong personalities.

The Delphi method, developed at RAND during the 1950s, collects the opinions of large groups of experts who participate anonymously. But such efforts are limited to the questions submitted of the expert panels and have no feature for interactive discussions among participants.

More recent approaches focus on channeling the "wisdom of the crowd" through large and diverse groups of people with different levels and areas of knowledge. Experts say that while crowdsourcing methods can reach large groups of people online, they also can be inefficient and unfocused unless there is clear direction and input is monitored.

ExpertLens leverages the advantages of both Delphi and the Nominal Group Technique methods. It also uses the modified principles of crowdsourcing to offer a means to elicit opinions from a broad and diverse pool of experts who are in different locations.

In general, in the first phase of an ExpertLens process participants answer a series of questions. In second phase, they review the group's responses and discuss their answers using online discussion boards. In the third phase, participants re-answer phase one questions based on the additional information they received during the feedback and discussion in the second phase.

The online nature of ExpertLens allows the results to be rapidly compiled and the findings to be analyzed quickly.

"ExpertLens is a major extension of the existing elicitation approaches that brings together the best of many different systems," said Dmitry Khodyakov, another ExpertLens developer and an associate behavioral/social scientist at RAND. "It is a new option to help solve problems when there are no obvious solutions. We will continue to experiment with the system to see where it works best and what features provide the best outcomes."

Thus far, the ExpertLens system has been employed to help study a number of issues. For example, it was used to analyze potential litigation stemming from a hypothetical terrorist bombing of a hotel, evaluate the federal response to the 2009 H1N1 flu pandemic and develop consensus about features that should define continuous quality improvement in health care.

Researchers say they are continuing to test and refine ExpertLens, planning further research to examine the impact of group size, the diversity of participants, the methods used for group discussions and methods to aggregate judgments.

Explore further: Four questions about missing Malaysian plane answered

add to favorites email to friend print save as pdf

Related Stories

Negativity is contagious, study finds

Oct 04, 2007

Though we may not care to admit it, what other people think about something can affect what we think about it. This is how critics become influential and why our parents’ opinions about our life choices continue to matter, ...

In Alzheimer's diagnosis, many heads better than one

Jan 26, 2011

In a marriage of two disciplines that don’t often overlap — politics and medicine — a study by Matthew Gabel, PhD, professor of political science in Arts & Sciences at Washington University ...

Recommended for you

Four questions about missing Malaysian plane answered

23 hours ago

Travelers at Asian airports have asked questions about the March 8 disappearance of Malaysia Airlines Flight 370 while en route from Kuala Lumpur to Beijing. Here are some of them, followed by answers.

Under some LED bulbs whites aren't 'whiter than white'

Apr 18, 2014

For years, companies have been adding whiteners to laundry detergent, paints, plastics, paper and fabrics to make whites look "whiter than white," but now, with a switch away from incandescent and fluorescent lighting, different ...

Freight train industry to miss safety deadline

Apr 16, 2014

The U.S. freight railroad industry says only one-fifth of its track will be equipped with mandatory safety technology to prevent most collisions and derailments by the deadline set by Congress.

User comments : 0

More news stories

Ex-Apple chief plans mobile phone for India

Former Apple chief executive John Sculley, whose marketing skills helped bring the personal computer to desktops worldwide, says he plans to launch a mobile phone in India to exploit its still largely untapped ...

Airbnb rental site raises $450 mn

Online lodging listings website Airbnb inked a $450 million funding deal with investors led by TPG, a source close to the matter said Friday.

Health care site flagged in Heartbleed review

People with accounts on the enrollment website for President Barack Obama's signature health care law are being told to change their passwords following an administration-wide review of the government's vulnerability to the ...

A homemade solar lamp for developing countries

( —The solar lamp developed by the start-up LEDsafari is a more effective, safer, and less expensive form of illumination than the traditional oil lamp currently used by more than one billion people ...

NASA's space station Robonaut finally getting legs

Robonaut, the first out-of-this-world humanoid, is finally getting its space legs. For three years, Robonaut has had to manage from the waist up. This new pair of legs means the experimental robot—now stuck ...

Filipino tests negative for Middle East virus

A Filipino nurse who tested positive for the Middle East virus has been found free of infection in a subsequent examination after he returned home, Philippine health officials said Saturday.

Egypt archaeologists find ancient writer's tomb

Egypt's minister of antiquities says a team of Spanish archaeologists has discovered two tombs in the southern part of the country, one of them belonging to a writer and containing a trove of artifacts including reed pens ...