'Big data' questioned in wake of Trump's surprising victory

After off-mark polls, some question value of 'big data'
In this Tuesday, Nov. 8, 2016 photo, supporters watch as results come in at the a GOP election party in Cedar Falls, Iowa. Donald Trump's stunning election win came despite prognosticators' overwhelming insistence he would lose and has forced people to question not just political polling, but all the facets of life that are being informed and directed by data. (Matthew Putney/The Courier via AP)

For an American public that relies on data for everything from where to find the best taco to the likely victor in a baseball game, Election Day offered a jarring wake-up: The data was wrong.

Donald Trump's stunning electoral win came despite prognosticators' overwhelming insistence he would lose. And it has forced many to question not just political polling, but other facets of life that are being informed and directed by data.

"If 'big data' is not that useful for predicting an election then how much should we be relying on it for predicting civil uprisings in countries where we have an interest or predicting future terror attacks?" asked Patrick Tucker, the author of "The Naked Future: What Happens in a World That Anticipates Your Every Move?" His book examines predictive analytics that aim to pinpoint answers to questions as varied as when a person will get married to how many kindergartners in a given class will end up with a cold.

Technology has filled people's lives with crowdsourced, data-driven or otherwise instructive metrics and left many convinced of their validity. We look to Yelp rankings to find a good meal and TripAdvisor to gauge a city's finest hotel. Netflix tells us which shows are best to watch and Zillow tells us the worth of the home we might buy. Amazon, Google, Facebook—all are ubiquitous presences in everyday life with data at their core.

So many took the predictions of polling aggregators as gospel—and their forecasts of Hillary Clinton's chances of winning the presidency went as high as 99 percent.

Tammy Palazzo, a 49-year-old corporate trainer in Maplewood, New Jersey, was among them. She's a political junkie who all day long refreshed FiveThirtyEight, the site from the heralded election-predictor Nate Silver. She purposely sought to look beyond her biases, living in a neighborhood dotted with signs for Clinton, a candidate she admired, and took in a variety of news sources. Then came Tuesday night.

"This is not going the way it's supposed to go," she thought, watching returns come in but not in Clinton's favor. "There was just so much reinforcement from the media that this was pretty locked down."

Allan Lichtman, a history professor at American University, was among the few who defied the confluence of polls and projected Trump's win. His model, developed in 1981, uses history as a guide to who will win the presidency through 13 true-or-false questions looking at economic indicators, military failure and success, social unrest and third-party candidacies. He's been consistently right.

"Polls are not predictions. They are snapshots and they are abused and misused as though they are predictions," he said.

Claudia Deane, a vice president at Pew Research Center, said many traditional methodologies in opinion polling are under attack as survey experts try to adapt to changing technology and communication methods. She says researchers are trying to find ways to accurately tap into social media or other indicators of public opinion that could enhance phone and internet surveys, but scientific methods are still being developed.

"The surveys are trying to predict the popular vote, and we're doing that in a country where we have a 50-50 electorate," she said. "Polling is not built for that kind of precision. It's not an excuse, it's just a mathematical fact."

"Big data" has been a buzzword for the last decade in Silicon Valley. Investors and tech companies, from little-known startups to corporate giants, have poured billions of dollars into software and computer systems that promise to pore through mountains of information and glean useful insights into business trends or consumer behavior. David Dill, a computer science professor at Stanford University, said it has opened the door to do things that weren't possible before. It enabled the collection of vast stockpiles of information all while advances in computing hardware and online networking have made it possible to run more sophisticated analytical programs and crunch bigger sets of data more quickly.

Even so, there's plenty of hype and unreasonable expectations for what analytics can do.

"Even if you have a lot of data and you go after it with the most sophisticated, amazing techniques, it may not tell you anything or it may mislead, because the data doesn't have the information you need," Dill said.

Khalid Khan, who heads analytics at the management consulting firm A.T. Kearney, said people are being influenced by data points without fully understanding what those numbers represent. He encourages companies weighing the direction of data to also consider—and discuss—"the softer side of things."

Speaking of data alone, he said: "If you take it at face value, you're going to get burnt. Without those conversations, you're only doing half of what you need to do when it comes to decision making."

Technology expert Sarah Granger has also seen that overreliance on data and technology, such as people who use mapping software only to end up on a dead-end street. Snafus like that, or presidential polling errors, she said, offer a reminder: "Big data's a great thing. There's so much we gain from it. But it's still in its infancy."

Explore further

Physicists develop analytics to predict poll trends

© 2016 The Associated Press. All rights reserved.

Citation: 'Big data' questioned in wake of Trump's surprising victory (2016, November 10) retrieved 18 July 2019 from https://phys.org/news/2016-11-off-mark-polls-big.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors

User comments

Nov 10, 2016
After they had been labelled "undesirables" is it any wonder they don't want to be outed by pollsters?
Its only since the large angry minority (soon to be the majority) started rejecting liberal political spin that the polls have been so far out. They cant be open to the pollsters as the liberal spin is so widely supported in the media.
Government has to be for the people and truthful or the angry elephant in the room will end up killing people.

Nov 11, 2016
The forces of the status-quo are so entrenched in their houses with their money and jobs that they see a completely different "big picture" from the disadvantaged. They believe their own spin that multinationalism and technologies demeaning effects on a growing minority of people are "regrettable but inevitable". They just don't see the angry elephant in the room.
So they ask how the polls could have got it so wrong when the rise of UKIP and Jeremy Corbyn and Brexit in the UK were predictable and inevitable. As was the Arab spring and the election of Trump and the deselection of Clinton by an electorate she called undesirables to their faces. (How would you have reacted to that)

Nov 11, 2016
The left winger who wants the increasingly parasitical financial services sector to stop dipping into his pension, or the right winger who wants his job back from the third world country it was exported to: Both must be labelled with a broad extremist brush so they can be ignored ..So the polls must have just got it wrong.
Big Data is just as prone to garbage in, garbage out as any IT system.

Nov 12, 2016
The desperate moderates on the left and right of the polite rich middle ground brought Trump and Hitler to power. We ignore them as undesirable extremists at our peril.

Yet the financiers will continue to self legitimise their worldview and use money taken from our pockets to lobby for their status-quo. Even if they have to abolish cash to keep our money in their sticky fingers.

Nov 12, 2016
The desperate moderates on the left and right of the polite rich middle ground brought Trump and Hitler to power. We ignore them as undesirable extremists at our peril
Consider that it was the media that gave you the opinion that trump and hitler were in the same category. And per the article it was the media that misinterpreted the data for you.

The hidden peril is that the world is on the brink of another bubble - a debt crisis. The media-encouraged liberal politicians have promised the people things we cannot afford in order to get elected, as they always do, and now the world needs emergency management just like Greece did.

That's why the critically flawed candidate hillary was foisted, and that's why we now have a businessman in charge who has spent his whole life serving the bottom line.

Hillary set up her server and then lied about it in order to throw the election. Trump was selected for the position back when he publicly took the birther position.

Nov 12, 2016
And the left was radicalized back when the media began making martyrs out of common thugs like trayvon and Michael brown. So now all those with radical tendencies are obediently exposing themselves for removal like their counterparts throughout the middle east.

All the world's a stage and you are following a script. Are you sick of being manipulated yet?

No matter - It's for your own good. Auditions and casting calls already for the next animal acts.

'I'll do anything to get the part!'

Nov 12, 2016
Paid for your Bush/Blair Mass Killings yet?

Fall for "WMD!!"??

We are in the 1930s again. Conservative hate and tantrums abound.

More economic displacement and social division first.

Mass arrests after that.

Killings next.

Nov 12, 2016
Trump is already waffling on Obamacare. I expect more waffling, as he finds out the consequences of his loudly-broadcast promises. What will the hateful folk do when 11,000,000 Latinos are not rounded up by a mass of Deportation Police and bused to the border? What will they say, when he realizes the Wall will not work and is a disaster in-the-planning?

My guess is they will blame liberals.

They will still see Muslims!! Then what?

Nov 12, 2016
We are in the 1930s again. Conservative hate and tantrums abound.

More economic displacement and social division first.

Mass arrests after that.

Killings next
Yes well perhaps we should be rounding up all the psychopaths and putting them in camps where they can't do any more damage to anybody besides each other. Make sense?

But we shouldn't kill them even though they have no compunctions about killing us because that would be wrong.

We're the ones with a conscience.

But as they are unfixable and they also pose the greatest danger to society, identification and sequestration is the only humane alternative.

"... there is a movement in several countries to instigate fundamental legal reform. The most controversial suggestion is to make it possible for individual who have severe personality disorders to be detained in secure mental institutions even if they have been accused of no crime"

-I can understand why psychopaths would be concerned.

Nov 12, 2016
Your concern should be discussed with professionals in the field of psychology.

They may be able to help you.

Nov 12, 2016
The only similarity I drew between Trump and Hitler is the basis of their support in the disadvantaged their appeals to everyone not in the liberal elite. And my invocation of the fear that both names provoke in those who support the dangerously exclusive status-quo.

Hillary Clinton is the epitome of that status quo elite and her defeat was as predictable and as inevitable as Brexit was in the UK. Only the liberal media didn't see it coming.

I agree that something radical has to be done. But the banks wont willingly let us do it and we will need the efforts of everyone to make a society work with less destructive goals and expectations.

It certainly wont be false messiahs like Trump who unify everyone and create wealth and happiness as promised.

Nov 12, 2016
"It certainly wont be false messiahs like Trump who unify everyone "
It is already starting - in opposition. Even now, he is already waffling on his promise to destroy Obamacare. There is no way he will build up a Deportation Force and round up 11,000,000 Latinos and bus them out of the country. No immense wall is going to be built, and we will still have Muslims.

The constraints of reality will reveal him as just a huckster, saying whatever it takes to make the sale. Once again, the poorly-educated and emotionally-vulnerable have voted against their interests in a political tantrum. Who are they going to blame after another economic disaster like Dubya? Will the Trump Wars be nuclear?

The election was rigged. We know it from the contacts between the Trump campaign and Russian government. Now the Russians admit the contacts.

The election is invalid, corrupted by a foreign power hostile to the interests of the United States.

Nov 12, 2016
They may be able to help you
Help? No, I already know how to recognize lying cheating psychopaths. For example they like to use t shirt phrases like Deportation Force and Dubya and Trump Wars. Notice the caps? The bigger their Causes are the more important they feel, the more impressive they think they are.

This in addition to their pointless lies about their work, their education, their faux emotions (sick of killing) designed to garner sympathy.

But all the while they are dropping clues about their real situation as abject failures living off the good intentions of others. The incompetent ones will do this at any rate.

Do wives support their psychopath husbands drug habit all the years before medical marijuana was legal? Do they fall for all that nonsense about combat trauma and hypochondria?

Nov 12, 2016
"She will believe that if she tries harder or simply waits it out, her husband will reform. When he ignores her, abuses her, cheats on her, or uses her, she can simply just decide to "try harder, put more energy into the relationship, and take better care of him." She believes that if she does this, eventually he will notice and will see how valuable she is, and then he will fall on his knees in gratitude and treat her like a queen.

"Dream on."
Your concern should be discussed with professionals in the field of psychology
Hey george do me a favor? Could you ask those shrinks of yours down at the VA about it? After all, according to you they they are monitoring my posts. At any rate they ARE closer to the problem aren't they?

That's a dear.

Nov 12, 2016
"The hidden peril is that the world is on the brink of another bubble - a debt crisis. The media-encouraged liberal politicians have promised the people things we cannot afford in order to get elected, as they always do, and now the world needs emergency management just like Greece did."

Otto those 2 posts were great.

We really are on the verge of losing our Republic. Let's hope that Trump is able to change our course. The rioting snowflakes that are a product of our educational system are very close to reaching a critical majority that wants to rewrite the Constitution and give away our hard won freedoms.

Nov 12, 2016
" What will they say, when he realizes the Wall will not work and is a disaster in-the-planning?"

Gkam please show me just ONE major country that has open borders and allows free uncontrolled immigration.

Nov 12, 2016

This pretty much spells out the differences between the Republicans and Democrats.

Nov 13, 2016
The system only works if every 100 people create 100 jobs. Then childbirth or immigration both creates and employs shopkeepers and builders and plumbers and doctors etc etc.
When unemployment and austerity are called "unfortunate but inevitable" by the liberal elite, who have jobs and wealth, and make the rules, that system is broken and then Trump became inevitable.
The way "the system" works at present jobs are exported or destroyed while the financiers cream off the wealth. Pensions shrink, jobs disappear while financiers take ownership of any real assets as collateral for rising paper money debt which must eventually default.

Financiers do not create wealth, except by printing it. Society has to realise that and stop listening to the spin and lobbying of that liberal elite. Financiers should be paid for the services they render not given control of the whole honey pot as at present.

Nov 13, 2016
"And also those who could not believe that the USA would vote for a racists serial rapist, ......."

Was that Bill Clinton you were referring to?

Nov 13, 2016
Btw a billionaire (unless he's actually broke) that borrowed millions from his rich dad to start a business is also an "epitome of the status quo elite".
I predict that all those angry white people who live in poverty will find very soon that they have been had, as so many people who did business with Trump.

Not yet sure whether Trump is an elite-insider or just a ruthless radical opportunist who has learned to suck hard on the teats of the system and only applauds those who are like him?
My guess is he would love Mexican or Muslim billionaires and hate many of his own poor supporters.

Nov 13, 2016
Good business, according to Trump, who is one of them.
Like lambs to the slaughter they voted for Trump
-So you ref a hillary quote in a spin article from politico? Just how gullible are you little lamb?

If trump voters were led to slaughter it was from reading all the transparent leftist bullshit such as your article and realizing how sick they were of such insulting attempts to manipulate them.

We're SICK of professional talkers and speechifiers. We're SICK of spending money we dont have. We're SICK of idiots using ageing rockers and has-been actors to try to trick us into electing crooks and liars.

Like MR166 said, the degenerates are the ones who lost.

Before hillary is pardoned (or not) we need to find out exactly what it is she did. She set up the server as her only way of receiving official correspondence including classified docs. That alone is intent. That alone is a felony. She should fry for it.

Nov 13, 2016
Btw a billionaire
He wasn't a billionaire when he borrowed the money
(unless he's actually broke)
He spent $100M of his own money on his campaign.
that borrowed millions from his rich dad to start a business
He borrowed $1M to start a successful business. Thousands of people do this every day from banks, corporations, and yes, relatives. You probably work for one. His dad got a nice return for his investment.

What makes YOU think that's a crime? (Answer - all those spin meister teats you've been sucking on.)

Nov 13, 2016
Aha the emails. They still haven't found anything really bad have they
Many were marked classified. But I explained to you exactly what the biggest crime was. Your spin doctors made you think it wasnt.
And what did Trump do? He shredded evidence in 3600 legal cases against him
And what about 'shredded' and 'evidence' and '3600 legal cases' makes you think he actually did anything wrong? Is it the way those words sound to you perhaps?

Trump wasn't jeopardizing state secrets. He wasn't committing treason. And was he ever accused or punished?
epitome of the status quo elite"
Sorry little sheep I don't use words like that. You seem to confuse easily.

And you should note that in your clip it was hillary who characterized trump as 'rooting'. This is called spin. Successful businessmen buy low and sell high which is something to admire.

This is only one of the many reasons he got elected. He knows how to invest wisely, not squander.

Nov 13, 2016
must have made some treacherous deal with Putin to receive active support from the Putin hacking department
Jesus fucking Christ. What makes you think Russians need encouragement to try to hack our computers?

The people who voted for trump know that he was being sarcastic. He was showing how reckless hillary was with sensitive info. And then we learned that yes, they were indeed hacked because of their incompetence. Criminal negligence.

And sheep like you can be convinced that this only happened because trump invited it??
Clinton was not convicted was she?
She_will_be with a new FBI director and a justice dept willing to issue subpoenas. And Obama will have to testify as to why he lied about not knowing he was sending sensitive info to a private server, among other things.

And if you think this isn't going to happen, consider that you didn't think trump would ever get elected.

And then wonder how it could be that you missed so much that was so important.

Nov 13, 2016
I think the polls were on the mark, and Trump was right about only one thing: It was rigged.

Our own federal agencies are corrupted with the residue of the Bush Years. I served on the federal courts back then, and saw the corruption for myself.

We are back in the 1930s again, and the Fascists are overjoyed.

Nov 13, 2016
Fascists think Trump is their man just because he makes offensive statements, is completely ignorant of diplomacy and how governments work, is irrational, impulsive, narcissistic, and pro-violence.

Once again: What will they do when they do not get 11,000,000 Latinos bused out of the country by masses of Deportation Police? Will they be mad there is no immense wall?

Nov 13, 2016

The Sellout is already under way!!

Nov 13, 2016
Neither Clinton as an apologist for the liberal elite version of 'politics of the banks'; nor Trump as a lone wolf looking to open a slaughter house; have any answers that offer long term work and stability.
Even the Elon Musk's or the Ideas people of silicon valley or MIT cannot offer enough job creation, while the banks have rigged the playing field; so that all products are made by the cheapest nations with the worst worker conditions.

If 100 people cannot be gainfully employed serving, healing, teaching, insuring, building and farming etc. for each 100 workers needs; then prosperity cannot be maintained. Financial service industries cannot be allowed to continue to cream wealth out the system until those 100 for each 100 have been paid first.

A banker out of a job is unfortunate, a farmer out of a job causes starvation.

Nov 13, 2016
You are willfully blind. The Russians hack everybody of course. Then they use the information to get Trump elected. That is the point you keep missing. Willful ignorance
1) they were able to hack because of DNC gross incompetence and Clinton criminal negligence and 2) it was the info itself and not the source of it that mattered.

Are you saying that assange wouldn't have published the same sort of info on trump and the RNC if he had found it? Of course he would have.

Assange was originally the champion of liberals and anarchists wasn't he? The criminals he found were all Democrats.

Nov 13, 2016
rasPutin publishes sensitive material on Clinton just before the election
You mean wikileaks. So what? Again you expect us to ignore evidence because of the source. That's unscientific.
He openly celebrates not only Trump's but his own success
So what? If you are going to enter into negotiations with somebody you don't want to start off by insulting them do you? Well maybe I would-
Now your boy is also involved in a giga conflict of interest
No he's not. Many people are surprisingly ignorant about the difference between corporations and the people who own them.

"Most US presidents put their investments in a blind trust, managed by an independent trustee, to avoid potential conflicts of interest for the length of their presidency..."

-And his kids will be running everything. Stop reading mother jones.

The Clinton Foundation in contrast was actively taking in millions for favors while hillary was sec of state. Another reason for Clinton jailtime.

Nov 13, 2016
Something else that mother Jones an CNN may not have told you

"... large donations to the foundation from the chairman of Uranium One, Ian Telfer, at around the time of the Russian purchase of the company and while Hillary Clinton was secretary of state, were never disclosed to the public. The multimillion sums were channeled through a subsidiary of the Clinton Foundation, CGSCI, which did not reveal its individual donors.

"Such awkward collisions between Bill's fundraising activities and Hillary's public service have raised concerns not just among those who might be dismissed as part of a vast right-wing conspiracy."

-This involved her approval of the rights to a major percentage of US uranium deposits while sec of state.

Nov 13, 2016
"Unwitting agents" of the FSB.

Where is Joe McCarthy now that we really need him?

Nov 13, 2016
The crumbling, crooked establishment on both sides gave the people no realistic choice and more than enough disinformation; the riots would've happened either way, they were happening prior to the election, growing in size as more good people become aware.

This link is to some mythological allegory; it sure paints a clearer picture of a crumbling empire than what I've seen from the owned-and-operated mainstream media.
The Nemesis and Scourges of the Western World

Nov 13, 2016
"Clinton acted in compliance with agreed rules."

Otto there is no sense arguing with a fanatic. They have been programmed since Pre-K to think a certain way. Higher education just makes the problem worse because the professors in the universities are so biased towards the progressive agenda.

Nov 13, 2016
166: We only do it to hear you whine.

Nov 13, 2016
The most amazing example of their disconnection from reality is the acceptance of the war in Syria. The reasons for deposing Assad are EXACTLY the same reasons that were given for removing Saddam. Yet this war is totally accepted by the Democrats without question.

Nov 13, 2016
We are more concerned with the actions of Russia in the United States.

Nov 13, 2016
"We are more concerned with the actions of Russia in the United States."

Well you must have been really upset with Hillary's unsecured server then, right?

Nov 13, 2016
We can only hope that the establishment, crooked or not, will keep Trump in check.
Look at the bright side, he's not a racist bigoted fraudster exploitative spoiled brat bully grabocious pervy egomaniacal genocidal disloyal tax evader – he's "smart". Just another "superior competitor." The system's geared to produce these sociopaths, he'll do fine. America will soon be the same again, and greatly so as promised. [/sarc]

Of the people by the people for the people was a nice idea, it looks great on paper. Too bad it never quite really caught on.

Nov 14, 2016
166 thinks trump is a good role model, especially during the Howard Stern interview, when he said it was okay to call his then-developing young daughter a "piece of ass". If I get famous, can I grab her buy the pu$$y?

And when he does his "casino magic" on the United States we will all be proud to ask Putin for help.

166, you haven;'t even PAID for your Bush Wars of Mass Killing and Corporate Profit.

Nov 14, 2016
And the kids have no common interest with dad
And tell me - do you really think that trump is the first president with private Investments and businesses and potential conflicts of interest? Really? That's why the blind trust is used.

One keeps coming to mind but it's certainly not the first or best example.

"Former President and First Lady Carter wave from their aircraft after the inauguration of Ronald Reagan on January 20, 1981. In 1981, Carter returned to Georgia to his peanut farm, which he had placed into a blind trust during his presidency to avoid even the appearance of a conflict of interest."
Criticism of Clinton cannot serve as defence of Trump.
Why jail time? Clinton acted in compliance with agreed rules No she committee treason and perjured herself.

Btw Clinton is history
Ah. So when crooks get away with it then they're no longer crooks. At least if they're Democrats I guess.

Nov 14, 2016
I get famous, can I grab her buy the pu$$y?
Well sure you can george. Psychopaths can do anything they want can't they?

Nov 14, 2016
Paid for your hateful Bush/Blair Wars of Mass Killings and Corporate Profit?

Conservatives have the problem of rationality and reality defeating the theories they sell to the public. Reaganism worked exactly as it was planned, to make the rich richer, and the 99.8% dependent on them. It was never for anything else.

It is Feudalism we got, not Reagan's "Shining City on a Hill", which turned out to be a heavily-guarded castle. And living down in the flats, the rest of us found out the true meaning of "trickle-down".

Nov 15, 2016
Just one more point
racist bigoted fraudster exploitative spoiled brat bully grabocious pervy egomaniacal genocidal disloyal tax evader
Bwahaahaaaaaa no my point is do you think Jimmy Carter was saying the same thing as trump when he said 'I have sinned in my heart' only using different words?

Do you think we should ask Jimmy for a clarification before he passes? as he is pretty old and I think this is a matter of national import.

If in his heart he was thinking of grabbing women here and there then I think we have a right to know, and so does donald.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more