Do think-tanks matter? A UBC professor says 'think again'

July 22, 2016, University of British Columbia
Assist. Prof. Carey Doberstein of UBC's Okanagan campus. Credit: UBC

A UBC professor is suggesting government policy makers and advisors need to do a re-think when it comes to giving validity to reports coming across their desks.

Carey Doberstein, an assistant professor of political science at UBC's Okanagan campus, recently published an experimental study of public sector workers and determined that many give a written report or study purported to be from a university more than one from a think-tank or advocacy group.

Doberstein conducted a randomized controlled survey experiment involving British Columbia public service staff, asking them to read and assess the credibility of various policy studies. For half of the respondents, the authorship of the studies was randomly switched but the content remained the same. Doberstein then compared the average credibility assessments between the control and experimental groups.

"There were systematic and at times extraordinarily large differences between the credibility assessments provided by these policy professionals on precisely the same policy studies, when the only part I changed was the label of who wrote it," said Doberstein. "Irrespective of the content and just by virtue of presenting it as written by an academic, the report suddenly becomes more credible in the eyes of bureaucrats."

The results surprised him, in part due to the magnitude of the differences observed. For one report, originally authored by the Fraser Institute, the credibility skyrocketed among study participants when they read the same document thinking it came from a university academic.

Another policy study, this time written by a university economist, received very high credibility assessments in the control group. But when authorship was changed to be purportedly written by the Canadian Centre for Policy Alternatives think-tank, its credibility plummeted dramatically.

"Put simply, the think-tank affiliation was a significant drag on the perceived credibility of their report and analysis," said Doberstein. The same was true for reports said to be written by research-based advocacy groups.

Some may interpret this finding positively," he said. "That analysts in government are skeptical of reports or studies that emerge from think tanks or advocacy organizations offering analysis and conclusions that tend to align with the organization's obvious ideological position."

Yet Doberstein says having a report's credibility increase simply by changing the name of the source is concerning as it can appear that policy-relevant research contained within its pages is being ignored by government policy advisors.

"We expect public servants to objectively examine the research evidence available to them," he said. "However, it seems many are taking shortcuts, and in essence giving academics a free pass."

And while this study examined the biases among policymakers in BC, Doberstein notes similar results were observed his subsequent replication experiment involving provincial analysts in Ontario, Saskatchewan and Newfoundland.

Doberstein's study was recently published in Policy Studies Journal.

Explore further: Race and gender of scientists affect perception of credibility

Related Stories

Surgeon general still has important role to play

December 30, 2014

(HealthDay)—The Surgeon General has an important role in educating and mobilizing the public and shaping policy on public health issues, according to an ideas and opinions piece published online Dec. 30 in the Annals of ...

Salacious alibi more credible if you lie about it at first

June 8, 2016

If you are suspected of a crime, you might be better off if you lie about an alibi in the first place and subsequently admit that you were in the company of an adulterous lover: it is even more credible than if you state ...

Research reveals government skill deficiency

June 24, 2016

Research by a leading economist at The Australian National University (ANU) has found that a lack of economic expertise within government is leading to ineffective policy and white elephant infrastructure.

Recommended for you

Coffee-based colloids for direct solar absorption

March 22, 2019

Solar energy is one of the most promising resources to help reduce fossil fuel consumption and mitigate greenhouse gas emissions to power a sustainable future. Devices presently in use to convert solar energy into thermal ...

EPA adviser is promoting harmful ideas, scientists say

March 22, 2019

The Trump administration's reliance on industry-funded environmental specialists is again coming under fire, this time by researchers who say that Louis Anthony "Tony" Cox Jr., who leads a key Environmental Protection Agency ...

3 comments

Adjust slider to filter visible comments by rank

Display comments: newest first

Jayarava
1 / 5 (1) Jul 22, 2016
Unfortunately the plan to counter liberalism outlined by Lewis Powell in 1971 involved, not just the creation of think tanks to spread the conservative economic message, but the founding of schools of business studies and the funding of chairs in existing universities. And this has been carried out. The leading academic economists do not declare their conflicts of interest when advising government - they're often funded by the same group of plutocrats who fund the think tanks.

RobertKarlStonjek
1 / 5 (1) Jul 22, 2016
Pt.2
The reasons why people give more credit to certificates and not to actual competencies include simplicity of decision making, the need to be able to explain one's decision to others (certificates are easier to explain) and an inability to evaluate the competency of an applicant or submission based on its merits rather than the individual's status.

Con-men and confidence tricksters reverse this, faking credentials then offering products or services which they trust the consumer will accept on the basis of the false credentials of the vendor rather than the actual quality of the product, service or idea offered. The superlative, especially in ancient times, was the messiah, prophet or God himself whose word could be accepted without any scrutiny whatsoever.

The study in this article shows that this same condition applies on yet another level.

RobertKarlStonjek
5 / 5 (2) Jul 22, 2016
The principle behind this finding is nothing new. People gain qualifications in part so that they will be taken more seriously. My favourite example, and one that I think shows the mechanism beautifully, came from a teacher friend of mine who was an early adopter of computer technology and lobbied his school to initiate a Computer Awareness class.

Frustrated at the lack of interest he started an Adult Ed (night school) in computer awareness. At the end of the course he gave the students a Certificate of Computer Awareness generated on the computer to impress on the class the value of these new devices.

Eventually his school decided to start a Computer Awareness class and asked for applicants to teach the course to which my friend eagerly applied but he didn't get the job.When he took one of the committee members aside and asked why he wasn't chosen he was told that the successful applicant had a Certificate of Computer Awareness and that he didn't, so they chose the other guy.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.