New study pits big data against intuition

August 18, 2015, Massey University
Simone Gressel wants to find out if big data is changing the way managers make business decisions.

A Massey University PhD student is delving into the decision-making processes of companies to understand whether judgement calls based on big data produce better outcomes.

Simone Gressel says she chose her thesis topic because of all of the hype around "". She was interested to understand how were using data, whether it was reducing the role of human judgement and whether managers made better decisions as a result of it.

"Big data is really a trending topic right now. There's a lot of hype but do managers really trust everything the data analysts tell them, or do they still revert to their experience and intuition?" Ms Gressel asks.

Decisions based on data 'usually work out'

She says her study is targeting firms that use big data for . She interviews managers at various levels within the organistion about their perceptions of , how they use data and how much they rely on it for decision-making versus their own intuition.

"We look at three to five decisions they have made in the past that had a good outcome or negative outcome so it's a decision they can recall clearly," Ms Gressel says. "I get them to lead me through that decision – what happened, what data they received, how and why they made the decision they did and how it turned out."

Analysis of these processes gives insight into why some decisions have good outcomes, and why others don't. So far, her research has shown decisions based on data usually work out well, sometimes to the surprise of business managers.

"When I asked people, 'Is this what your gut told you?', they were often surprised because the data told a completely different story to their own perceptions. But when they were convinced by others to follow the data, they found the data outperformed their intuition – and this then led to greater trust in analytics the next time around."

When data tells the wrong story

This is not to say the data was always right. Ms Gressel says she has found two main reasons for negative outcomes: unreliable or unsuitable data, often provided by external sources and not properly incorporated into the organisation's own environment; or the analytics team missing an important component of data because they didn't fully understand the business situation.

"The job of a data scientist is to understand all the business needs as well as the actual data and its components – but it is hard to find someone like that in real life. There is a skills gap," she says.

The most successful management teams will embrace a data culture but also create an environment where managers can question recommendations when it doesn't match their intuition, Ms Gressel says.

"Data challenging is a very important concept if it comes with the acceptance that the data may also be right. It needs to be okay to ask analysts to go back and delve deeper into the data to prove their findings or, perhaps, find a flaw."

Wanted: Companies to share their decision-making process

While Ms Gressel's research has already given her useful insights into the best decision-making practices, she is seeking several more companies to analyse to ensure her findings are robust.

"I am looking for medium and large organisations that use analytics, preferably for predictive decisions, rather than just analysing what has happened in the past."

Ms Gressel is keen to give participating companies something of value in return. While remaining anonymous, firms will receive an executive summary that outlines their decision-making processes, including factors related to organisational culture, which will then be compared to best practice as it is currently defined in both the practical and academic literature.

"The lessons will be specific to their company and I'll highlight things they might not be doing that I've seen lead to success in other organisations," she says.

Explore further: Planning and improvisation actually play well together in export markets

Related Stories

How a data deluge leaves us struggling to make up our minds

July 15, 2015

We make a huge number of decisions every day. When it comes to eating, for example, we make 200 more decisions than we're consciously aware of every day. How is this possible? Because, as Daniel Kahneman has explained, while ...

Big data collaboration to improve chronic disease management

August 3, 2015

Duke-NUS Graduate Medical School Singapore (Duke-NUS) and Holmusk, a global tele-health platform, have recently announced a collaboration to further the potential of big data in healthcare. Big data refers to very large, ...

Recommended for you

Where is the universe hiding its missing mass?

February 15, 2019

Astronomers have spent decades looking for something that sounds like it would be hard to miss: about a third of the "normal" matter in the Universe. New results from NASA's Chandra X-ray Observatory may have helped them ...

What rising seas mean for local economies

February 15, 2019

Impacts from climate change are not always easy to see. But for many local businesses in coastal communities across the United States, the evidence is right outside their doors—or in their parking lots.

Tiny particles can switch back and forth between phases

February 15, 2019

Three years ago, when Richard Robinson, associate professor of materials science and engineering, was on sabbatical at Hebrew University in Israel, he asked a graduate student to send him some nanoparticles of a specific ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.