Researchers study best way to solve wicked problems

November 29, 2007
Researchers study best way to solve wicked problems
Wicked problems are complex problems that change when you apply a solution (Photo by Randy Montoya)

What’s the best way to solve a wicked problem — by working in a large group sharing ideas via the intranet or as individuals? That’s the question George S. Davidson and his research team at Sandia National Laboratories attempted to resolve this summer.

The research, conducted by Davidson, Courtney Dornburg, Susan Stevens, Stacey Hendrickson, Travis Bauer and Chris Forsythe, had some surprising results.

“In this day and age of email and the internet, our expectations were that computer-mediated group brainstorming, i.e. across the web with no face-to face contact, was going to have the best results,” Davidson says. “What we found, however, was that people working as individuals were at least as effective and possibly more so than those brainstorming in a group over the web when trying to solve ‘wicked,’ tangled problems, both in terms of quality and quantity.”

The experiment was funded by Sandia’s internal Laboratory Directed Research and Development (LDRD) program.

Wicked problems are those problems that by their very definition are so tangled that there is no agreement about their definitions, much less their solutions.

Davidson assembled a team that consisted of himself, three psychologists (two of them PhD students), and two cognition researchers to investigate tools and methods for bringing very large numbers of people together to solve difficult problems via the intranet. The team anticipated that the group brainstorming could result in a huge pool of ideas that might lead to solutions. They decided to pursue an electronic brainstorming experiment built around the very common face-to-face technique used at Sandia where people submit ideas written on Yellow Sticky Post-It ® Notes.

As the team designed the experiment, the initial issue was deriving a wicked problem with no predefined “right or wrong answer,” Dornburg says. In June Dornburg, a psychologist who has worked at Sandia for two years, attended a new-hire breakfast hosted by Labs President and Director Tom Hunter. During the breakfast, Hunter posed the new hires with a challenge, asking them to think about the implications of two popular models in management theory. In one view workers are just another natural resource to be used. In contrast, the second model sees employees as assets, which can be made more valuable by investing in their development. Dornburg shared the question with the team, and they unanimously decided to make it the wicked problem for the experiment.

They recruited 120 Sandia employees and contractors and 26 student interns through an internal online daily newsletter and word of mouth to participate in the experiment that lasted four days in August. The participants were broken into two groups, those who worked alone and did not see the ideas of the other participants and those who worked in a group and were able to see and build on the ideas of the other members in the group via the Labs’ intranet. Of the 120 employees and contractors, 69 contributed ideas.

During the experiment, participants logged onto the website anonymously and saw the question displayed at the top of the screen. They were asked to input their ideas — the more the better, but no name calling or abusive language.

The research was unusual because, while studies have been conducted with large brainstorming groups, most have been performed in academic settings with college students as participants. This experiment was, to the research team’s knowledge, the first done in a laboratory/industrial setting over an extended period of time.

In addition, most previous studies looked only at quantity — number of responses — while only a small number examined the quality of the ideas. The Sandia experiment analyzed both. Responses to Hunter’s question were scored by team members according to originality, feasibility, and effectiveness. Responses were also analyzed by STANLEY, a Sandia-developed text analysis tool that looked at common noun phrases and key words.

“We were amazed at the length and quality of the responses, both from the people working as a group and those working individually,” Dornburg says. “People were very engaged, often writing long, detailed responses.”

She adds that what was most interesting is that the quality of ideas from the people responding as individuals was “significantly better across all three quality ratings.”

Dornburg says the finding that individuals are more successful than groups in computer-mediated brainstorming suggests a time- and cost-saving potential for companies. Generally, when electronic group brainstorming is compared to face-to-face brainstorming, it is touted as having the advantages of shorter meetings, increased participation by remote team members, better documentation via electronic recording, and cash savings. But the Sandia research suggests that people working to solve problems on their own might involve less time and, thus less expenses, than electronic group brainstorming.

While individuals working alone nominally faired better in this study, Davidson says, the research also indicates that group on-line brainstorming can be effective when ideas are needed from large numbers of people.

“Despite our findings, it still seems reasonable that there may be modes of [as yet, untested] web-based interactions and strategies that would allow the larger group to have superior performance compared to a limited number of participants working alone, even if those participants are able to reason and write about their ideas with brilliance and clarity,” Davidson says

He expects that in coming years “better software, including threaded discussions with moderators to focus the work and prediction markets to evaluate quality, will become tools that large organizations will use to solve wicked problems.”

Source: Sandia National Laboratory

Explore further: Where are the tools for scientific writing?

Related Stories

Where are the tools for scientific writing?

January 30, 2017

Writing a scientific research paper is tough at the best of times regardless of funding conditions and political intervention. As such, a scientist will turn to any tool they might find to help with this generally arduous ...

CERN scientists brainstorm future role

September 12, 2012

Scientists who searched for the famous Higgs Boson concluded a three-day brainstorming session on Wednesday for mapping a long-term global strategy for particle physics.

The secret life of scientific ideas

September 1, 2015

Many of the most memorable stories in the history of science revolve around the conscious realization of an idea - the "Eureka!" moment. But what triggers these moments? Is there always some serendipitous event preceding ...

Assessing feedback interactions in a creative setting

March 11, 2015

Feedback - the objective response, opinion, or input - is something most of us experience either at work or amongst friends to bodies of work or projects that are complete. But in the world of creative processes - where no ...

Recommended for you

Scientists make new discovery about bird evolution

March 24, 2017

In a new paper published in National Science Review, a team of scientists from the Institute of Vertebrate Paleontology and Paleoanthropology, the Shandong Tianyu Museum of Nature, and the Nanjing Institute of Geology and ...

Study into who is least afraid of death

March 24, 2017

A new study examines all robust, available data on how fearful we are of what happens once we shuffle off this mortal coil. They find that atheists are among those least afraid of dying... and, perhaps not surprisingly, ...

Mathematical framework explains diverse plant stem forms

March 23, 2017

It is well known that as plants grow, their stems and shoots respond to outside signals like light and gravity. But if plants all have similar stimuli, why are there so many different plant shapes? Why does a weeping willow ...

How chewing like a cow helped early mammals thrive

March 23, 2017

You probably haven't given much thought to how you chew, but the jaw structure and mechanics of almost all modern mammals may have something to do with why we're here today. In a new paper published this week in Scientific ...

'Pay to publish' schemes rampant in science journals

March 22, 2017

Dozens of scientific journals appointed a fictive scholar to their editorial boards on the strength of a bogus resume, researchers determined to expose "pay to publish" schemes reported Wednesday.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.