# Explained: Margin of error

##### November 1, 2012 by David L. Chandler

In mid-October, a Gallup poll of likely voters nationwide showed former Massachusetts Gov. Mitt Romney leading President Barack Obama by a 7 percent margin. That same week, a poll by the University of Connecticut and the Hartford Courant, covering virtually the same time period, showed Obama ahead of Romney by 3 points. That's a 10-percentage-point disparity.

The Gallup poll reported a margin of error of plus or minus 2 percent, while the UConn/Hartford Courant poll reported a 3 percent margin of error—so even if you add the maximum claimed errors, that still leaves a 5-point disparity between the results. As some candidates have observed on the campaign trail, the math just doesn't seem to add up.

But such , in this of rapidly shifting tides, have not been all that unusual. So what explains them?

There may be several factors at work. For starters, the concept of "margin of error" is a bit more complex than the numbers usually quoted in media coverage.

What pollsters usually mean by margin of error is something more specific, called the margin of sampling error. That's a statistical measure of how much a result based on interviews with a limited number of voters—the typical sample size is about 1,000 to 2,000 people—differs from the result that you would get if you were able to interview all the likely voters in the country.

"A sample of 1,000 or 1,500 people gives a reasonable estimate," says Adam Berinsky, a professor of at MIT and editor of "New Directions in Public Opinion" (Routledge, 2011), a book about research on public opinion.

Most of the time, studies have shown, talking to 1,000 people gives a result very close to what you would get by polling the whole country. Extrapolating to a nation of 132 million voters—roughly the number of citizens who voted in the 2008 presidential election—gives a 1,000-person poll a margin of sampling error of 3.1 percent. Double the sample size, to 2,000 people, and the margin of sampling error falls to about 2.2 percent. But the improvement diminishes rapidly: A poll of 5,000 people gives about a 1.4 percent margin, and it takes a whopping 10,000-person sample to get the margin down to 1 percent.

One out of 20

What does that margin of error figure actually mean? The definition is that 95 percent of the time, the sampled result should fall within that margin of the result you'd get by sampling everybody. But that also means that one time out of 20, the results would fall outside of that range—even if sampling error were the only source of discrepancies.

But it's more complicated than that, because sampling error is not the only thing that can throw off poll results.

Another potential source of error, and one that's hard to quantify, is the nonresponse error. Pollsters begin by attempting to reach a certain randomly selected set of people that is representative of the overall population—for example, by generating a list of random phone numbers. But there are two problems: Sometimes nobody answers the phone, and even when someone does answer, they often—and increasingly—refuse to respond.

The Pew Center for People and the Press, for example, says that its response rate has plummeted in the last 15 years: Their total response rate to polls, which was 36 percent in 1997, is down to just 9 percent this year. "This is very worrisome to pollsters," Berinsky says.

Most nonresponders are people who answer the phone, but refuse to take the poll. This year, Pew says, 62 percent of people called by their pollsters answered the phone, but only 14 percent of those would answer questions.

A big question is whether the people who don't answer are different in any meaningful way from those who do: Are the nonresponders more liberal, or more conservative, than those who do respond? If so, that could skew a poll's results.

Pew has made a serious effort to assess the possible impact of nonresponse error on its poll results: For one sample, the organization made a concerted effort to follow up with as many nonresponders as possible, asking questions designed to see if they differed from those who had answered the first time. While Pew found few significant differences between poll responders and nonresponders, there were some: Those who answered, it turns out, were much more likely to volunteer for charitable organizations, attend a church, and contact their elected officials.

A common mistake in the reporting of poll results is the application of the margin of sampling error for the entire poll to various subsets of the population: women, men, Democrats, Republicans, independents. But each of these subgroups is smaller than the total group, so the margin of error is actually much greater. "A lot of journalists don't understand that," Berinsky says.

Another potential problem is measurement error, which occurs when a poll's questions are poorly worded, or prefaced by information that biases the responses. The major polling organizations take great care to avoid measurement error, but polls commissioned by partisan organizations sometimes suffer from such errors.

Overall, Berinsky counsels, the best strategy is not to focus on any particular poll, but to look at a rigorous aggregation of poll results, such as those conducted by Pollster.com or Real Clear Politics. Such averages smooth out the variations and errors that may exist in any given or sample. In the 2008 election, he says, "a simple average pretty much gave you the [actual] result."

"If you average it all together," Berinsky adds, "it all works out."

Explore further: Voter engagement higher than usual in 2008

## Related Stories

#### Voter engagement higher than usual in 2008

February 7, 2008

A new Gallup poll said U.S. voters are displaying a record level of interest in this year's presidential election.

#### Poll: Germans accept nuke exit despite rising bill

October 21, 2012

A new poll finds that a large majority of Germans back the government's decision to phase out nuclear power and switch to renewable energies within a decade, despite rising electricity bills.

#### US men prefer beer, women love wine

August 17, 2012

US men like to drink beer, though wine is the beverage of choice for women and older adults, a poll showed Friday.

#### Americans favor Obama to defend against space aliens: poll

June 27, 2012

Nearly two in three Americans think President Barack Obama is better suited than Republican rival Mitt Romney to deal with an alien invasion, according to a survey released Wednesday.

#### Poll: Most people reject evolution theory

March 8, 2006

A Gallup Poll released Wednesday suggests about 53 percent of Americans rejects the theory of evolution as the explanation for the origin of humans.

#### Climate change no longer tops US environment worries

July 3, 2012

Americans no longer see climate change as the world's number-one environmental issue, according to a public opinion poll released Tuesday amid an ongoing heat wave in much of the United States.

## Recommended for you

#### Sexual harassment common among middle school children, study finds

December 9, 2016

The recent suicide of Brandy Vela, a teen in Texas City, Texas, was a potent reminder of the sometimes tragic consequences of bullying. According to Vela's parents, the teen fatally shot herself Nov. 29 following months of ...

#### Digitally reconstructed skull and face may reveal Robert the Bruce, king-hero of the Scots

December 9, 2016

Could this be the face of Robert the Bruce, as it has never been seen before?

#### Archeologist claims to have found proof that Hebrew was the first written alphabet

December 8, 2016

(Phys.org)—Douglas Petrovich, an archaeologist with Ontario's Wilfrid-Laurier University in Canada has sparked controversy in the ancient history scholarly community by making claims that he has found proof that Hebrew ...

#### Amber specimen offers rare glimpse of feathered dinosaur tail

December 8, 2016

Researchers have discovered a dinosaur tail complete with its feathers trapped in a piece of amber. The finding reported in Current Biology on December 8 helps to fill in details of the dinosaurs' feather structure and evolution, ...

#### Mobile money access lifted two percent of Kenyan households out of poverty: study

December 8, 2016

Since 2008, MIT economist Tavneet Suri has studied the financial and social impacts of Kenyan mobile-money services, which allow users to store and exchange monetary values via mobile phone. Her work has shown that these ...

#### Fossilized evidence of a tumor in a 255-million-year-old mammal forerunner

December 8, 2016

When paleontologists at the University of Washington cut into the fossilized jaw of a distant mammal relative, they got more than they bargained for—more teeth, to be specific.

Adjust slider to filter visible comments by rank

##### VendicarD
4.2 / 5 (5) Nov 01, 2012
The problem with Romney of course, is that he has shown himself to be an even bigger liar than George Bush.
##### geokstr
1.6 / 5 (7) Nov 01, 2012
The problem with Romney of course, is that he has shown himself to be an even bigger liar than George Bush.

But both of whom have proven to be much more honest than the Community-Organizer-in-Chief, by at least several orders of magnitude.
##### VendicarD
4.2 / 5 (5) Nov 01, 2012
I keep hearing that from Tea Baggers who still think that their president is not an American.

Their childish whining of claimed dishonesty from Obama are as convincing as their insipid whining about the illegitimacy of his birth certificate.

"But both of whom have proven to be much more honest than the Community-Organizer-in-Chief" - Geokstr

Where are those WMD again Geokstr?

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more