In surveys, people say they'll pay twice what they're actually willing to spend

June 27, 2018 by Misti Crane, The Ohio State University
Credit: CC0 Public Domain

Perhaps people like to think of themselves as big spenders. Or maybe they just aren't very honest.

But when researchers compared what reported they were willing to spend on goods with what they actually shelled out in experiments designed to mimic a real-world shopping experience, there was a big gap.

"People said they'd spend about twice as much, on average, as they actually spent when faced with more realistic circumstances," said co-lead researcher Wuyang Hu of The Ohio State University.

Asking people to estimate how much they'd spend on a good or service is a key part of economics research and the results can drive decision making by business, government leaders and others, said Hu, a professor of agricultural, environmental, and development economics at Ohio State.

"Unfortunately, in many circumstances, it's extremely difficult to observe people's actual behavior, so we have to use these hypothetical, made-up scenarios," Hu said. "The problem is, we know that what people say they will spend and what they actually spend don't always line up."

The difference between these two amounts is called "hypothetical bias" and it stands in the way of research that can shape important decisions, like evaluating the worth of cleaning up after an environmental catastrophe, Hu said. In fact, the whole notion of this phenomenon came about after the 1989 Exxon-Valdez oil spill, when lawyers grappled with arriving at the cost of work to soak up the oil and protect animals, plants and natural resources in Alaska.

Hu and his co-author, Jerrod Penn of Louisiana State University, wanted to better understand the discrepancy between what people report they'll pay and what they'll actually pay. To do that, they reviewed and summarized results found in previous studies, either published or not. They looked at more than 500 articles and chose 132 studies that included hypothetical questions and an attempt to create a real-life spending experience—such as a classroom set up to look like a store in which study participants had actual money to spend. The study appears in the American Journal of Agricultural Economics.

Overall, the difference between hypothetical spending and real spending was almost twofold. Hu stressed that even this could be an underestimate, given that the "real" spending observations are done under circumstances that attempt to mimic real life.

"Sometimes people are aware they're being observed. Sometimes they have no idea. But it's not really a grocery store or a conservation group's office where you're making an actual donation. It's an experiment, so we take those so-called 'real' spending observations with a grain of salt as well," Hu said.

Somewhat surprisingly, the researchers found that student study participants were not less reliable than others—a trend that has emerged in other research, and led to concerns about the validity of economics research that relies largely on college students' feedback about spending.

"Some people have reached the conclusion that because students don't have much money to spend in real life, they tend to overstate what they'd spend, but we didn't find that," Hu said.

The researchers also found evidence that three common techniques designed to mitigate bias were effective: "cheap talk," certainty follow-up and consequentiality.

Researchers who use the cheap talk approach explicitly address the issue to study subjects, acknowledging that people often claim on surveys that they'll spend more than they would actually spend in real life. Researchers then tell them not to do that. Cheap talk techniques reduced differences by 41 percent, the researchers found.

Certainty follow-up involves asking the study participant to rate his or her level of certainty about a given survey response and this method reduced differences by 99 percent.

Consequentiality is when researchers emphasize the importance and value of the survey participants' honest responses. This approach reduced differences between hypothetical and real spending by 95 percent.

Penn, an assistant professor of resource economics at Louisiana State, said the evidence in favor of cheap talk and other efforts to mitigate hypothetical bias is important—especially because some previous studies have cast doubt on the usefulness of such techniques.

"This means that exploring and testing new mitigation methods and how methods can complement each other continues to be a valuable avenue of future research," Penn said.

Hu said he hopes this robust meta-analysis of prior research will lead to improved techniques for factoring in this hypothetical bias or designing studies that do a better job of reducing it.

"Researchers will benefit from having more information about adjusting their results upward or downward to come closer to the truth about how much people will spend," Hu said.

He said he'd like to develop an app that researchers could use to estimate the impact of hypothetical bias based on the nature of their study, the participants and whether the good or service being studied is public or private.

"Hypothetical bias is really the tip of the iceberg in the entire human behavior puzzle that everyone's trying to crack in economics. Hopefully this brings us a little bit closer to the truth," Hu said.

Explore further: People make different moral choices in imagined versus real-life situations

More information: Jerrod M Penn et al, Understanding Hypothetical Bias: An Enhanced Meta-Analysis, American Journal of Agricultural Economics (2018). DOI: 10.1093/ajae/aay021

Related Stories

Why we perceive ourselves as richer than we think we are

May 24, 2018

Everyday billions of people make countless decisions that have economic implications. Buying new clothes, having dinner at a Japanese restaurant, renting a house: most of our decisions determine how much money we spend or ...

How humans repress prejudices

May 17, 2018

Bochum-based philosopher Dr. Beate Krickel has used psychoanalysis to investigate why people are often not aware of their prejudices. In her accounts, she has been elaborating how prejudices can become unconscious. As researcher ...

Recommended for you

When does one of the central ideas in economics work?

February 20, 2019

The concept of equilibrium is one of the most central ideas in economics. It is one of the core assumptions in the vast majority of economic models, including models used by policymakers on issues ranging from monetary policy ...

In colliding galaxies, a pipsqueak shines bright

February 20, 2019

In the nearby Whirlpool galaxy and its companion galaxy, M51b, two supermassive black holes heat up and devour surrounding material. These two monsters should be the most luminous X-ray sources in sight, but a new study using ...

Research reveals why the zebra got its stripes

February 20, 2019

Why do zebras have stripes? A study published in PLOS ONE today takes us another step closer to answering this puzzling question and to understanding how stripes actually work.

Correlated nucleons may solve 35-year-old mystery

February 20, 2019

A careful re-analysis of data taken at the Department of Energy's Thomas Jefferson National Accelerator Facility has revealed a possible link between correlated protons and neutrons in the nucleus and a 35-year-old mystery. ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.