Study shows even smaller media outlets can have wide impact on national conversation

November 9, 2017, Harvard University
An infographic depicting results by King et al. Credit: Carla Schaffer / AAAS

With a readership that runs into the millions, few would argue that the New York Times doesn't influence public debate on a host of issues. But what about a news outlet with a circulation of only about 50,000?

The answer, says Albert J. Weatherhead III University Professor Gary King, is that even small- to medium-sized media outlets can have a dramatic effect on the content and partisan balance of the national conversation about major public policy issues.

In the first large scale randomized media experiment ever conducted, King and former students Benjamin Schneer, now assistant professor at Florida State University, and Ariel White, now assistant professor at MIT, found that if just three outlets write about a particular major national policy topic - such as jobs, the environment, or immigration - discussion of that topic across social media rose by more than 62 percent, and the balance of opinion in the national conversation could be swayed several percentage points based on that coverage.

"For several hundred years, scholars have tried to measure the influence of the media. Most people think it is influential, but measuring this influence rigorously with randomized experiments has until now been impossible," King said. "Our findings suggest that the effect of the media is surprisingly large. Our study's implications suggest every journalist wields a major power and so has an important responsibility."

Those findings, King said, are the result of more than five years of work, much of which was spent convincing 48 news outlets to agree to take part in the study. Fortunately, about half of these outlets were represented by the Media Consortium, a network of independent news outlets, eager to find a way to measure impact and willing to help.

"Much of the work leading up to this study involved finding a way to bridge the cultural divide between journalism and science," King explained. "Through years of conversations, much trial and error, and a partnership with Media Consortium Executive Director Jo Ellen Green Kaiser, we learned to understand journalistic standards and practices, and the journalists learned to understand our scientific requirements. What ultimately made it all work was a novel research design we developed that satisfied both camps."

Though similar efforts have been tried in the past, they invariably collapsed as journalists chafed at the notion of being told what to report and when to report it.

To address the problem, previous researchers fell back on clever tricks, such as studying areas that fell outside the broadcast area of a particular outlet but, because no one knew whether the areas were truly random, they were hard to evaluate. Such studies faced many problems, particularly their inability to control for a host of factors like race, education, or income.

"From a scientific point of view, we have to be able to tell the journalists what to publish, and preferably at random times," King said. "From a journalistic point of view, these scientific requirements seem crazy, and journalists reasonably insist on retaining absolute control over what they publish. The two sets of requirements seem fundamentally incompatible, but we found a way to create a single research design that accomplished the goals of both groups."

It wasn't only the participation of that made the study noteworthy, though.

"If you're doing something like a medical experiment, you may randomly assign individual people to one of two groups, and then each person is your unit of analysis," he said. "But when a media outlet publishes something - no matter how small - the potential audience it could impact includes basically everybody in the country. That means our unit of analysis can't be a person; it has to be the entire country, which greatly increases the cost of the study." This means that the equivalent of an entire experiment in many other studies constitutes only one observation in this study.

Because collecting each observation was so expensive, and so logistically challenging, King and colleagues used, and further developed, novel statistical techniques to enable them to collect only as much data as needed. They could then examine, after each massive national experiment, whether they now had enough data to draw reliable conclusions.

"That allowed us to keep going until we got to the point where we had exactly as much data as we needed, and no more," King said. "As it turns out, we ran 35 national experiments that produced 70 observations."

To achieve the experimental randomization needed for the study, King's team, the Media Consortium staff, and journalists at the 48 outlets together identified 11 broad policy areas - like jobs, the environment, or immigration. They then simulated the tendency of journalists to influence each other and publish stories on similar topics, sometimes known as "pack journalism," by choosing three or four outlets from their participating group of 48 to develop stories together that fell into the same broad policy area.

"For example, if the policy area was jobs, one story might be about Uber drivers in the Philadelphia area," King said. "We would then identify a two-week period where we predicted there wouldn't be any surprises related to that topic area - so if the President was planning to give a speech about immigration in one of those two weeks, we would not run an experiment on immigration during that time."

Randomization came from researchers flipping a coin to determine which of those weeks would be the publication week, and which the control week.

"At first, our outlets didn't really understand what randomization meant," said Kaiser. "Our project manager Manolia Charlotin and the researchers worked very closely with all the outlets to ensure they followed the researchers' rules. This was a resource-intensive project for us, but the unexpected benefit was that outlets found they also gained many qualitative benefits from collaborating."

In both treatment and control weeks, King, Schneer, and White used tools and data from Harvard-based start-up Crimson Hexagon, to monitor the national conversation in . King is co-founder of Crimson Hexagon and, with a previous generation of now former Harvard students, developed the automated text technology that was originally licensed by Harvard to create the company. He explained that this methodology "is used to evaluate meaning in social media posts. So if you have a set of categories you care about, we identify example posts in these categories, which is what humans are good at," King said. "Then our algorithm can amplify that human intelligence and, without classifying individual posts, can accurately estimate the percent of posts in each category each day."

What they found, King said, is that the effect was larger than anyone anticipated.

"The actual effect is really big," King said. "If three outlets (with an average circulation of about 50,000) get together and write stories, the size of the national conversation in that policy area increases a lot - it's a 62 percent increase on the first day's volume distributed over the week, just from these three little outlets."

"These national conversations about major policy areas are essential to democracy," he added. "Today this conversation takes place, in part, in some of the 750 million publicly available social media posts written by people every day—and all available for research. At one time, the national conversation was whatever was said in the public square, where people would get up on a soapbox, or when they expressed themselves in newspaper editorials or water-cooler debates. This is a lot of what democracy is about." King explained, "The fact that the media has such a large influence on the content of this national conversation is crucial for everything from the ideological balance of the nation's outlets, to rise of fake news, to the ongoing responsibility of professional journalists."

Explore further: Most Americans get science news from general outlets more say specialty outlets more accurate

More information: G. King at Harvard University in Cambridge, MA el al., "How the news media activate public expression and influence national agendas," Science (2017). … 1126/science.aao1100

Related Stories

New study identifies 'disconnect' between media and public

February 9, 2017

Recently, mainstream media have faced heavy criticism from all sides of the political spectrum regarding coverage of the presidential election and other political issues. Now, researchers at the University of Missouri School ...

Recommended for you

University choice and achievement partly down to DNA

October 18, 2018

Research from King's College London has shown for the first time that genetics plays a significant role in whether young adults choose to go to university, which university they choose to attend and how well they do.


Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (4) Nov 09, 2017
I think its funny when I click on a wall st journal article and they tell me I have only 5 more free articles before I have to subscribe.

And when they refuse me that first article I just go back to google news and click on something else.

This is why theyre all dead and they just dont know it yet.

Whats that? journalistic integrity? They lie just as much as everyone else in those clickbait articles of theirs. I can get that shit anywhere else for free.
1 / 5 (1) Nov 09, 2017
What we need is a commission of experts in news, media and communications. People with impeccable resumes from academia, corporations and government who could analyze and vet news stories and let us know which are accurate and which are fake.... maybe give it a catchy title like 'Ministry of Truth'.
5 / 5 (2) Nov 09, 2017
Sadly truth like justice is often a matter of how much you can afford.
Whydening Gyre
5 / 5 (1) Nov 09, 2017
What we need is a commission of experts in news, media and communications. People with impeccable resumes from academia, corporations and government who could analyze and vet news stories and let us know which are accurate and which are fake.... maybe give it a catchy title like 'Ministry of Truth'.

That was sarcasm, wasn't it....:-)
5 / 5 (1) Nov 10, 2017
What we need is a commission of experts in news, media and communications. People with impeccable resumes from academia, corporations and government who could analyze and vet news stories and let us know which are accurate and which are fake.... maybe give it a catchy title like 'Ministry of Truth'.

That was sarcasm, wasn't it....:-)
Actually i think we already have this. Its called Fox News.

-or you could just listen to Hannity.
not rated yet Nov 10, 2017
I don't have quite enough information to figure out just how this benefits democrats.
not rated yet Nov 11, 2017
Of course, this explains why they want to get rid of the 'alternative media', so that corporate fascism can maintain it's hold on the group mind of the USA and the west, overall.
5 / 5 (1) Nov 13, 2017
People know, that the larger outlet is, the more probably gets biased or censored by government
The large news organizations ARE govt. They are extensions of the Democratic party and their purpose is to discredit and demean and defame the opposition while at the same time protecting themselves and their beloved politicians.

They censor by overwhelming and outshouting conservative opinion. They lie, they cheat, and they steal.

All except for Fox.

I suppose this is necessary to maintain the 2 party system, because the vast majority of people in this country are by nature conservative. So false narratives must be created to delude the gullible. And both sides are no doubt complicit in this effort.

But at present truth is on the side of conservatives and it can usually only be found in the conservative media.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.