Measuring political bias of network news

Apr 15, 2011 By Nancy Fowler

(PhysOrg.com) -- That FOX Broadcasting Company has a conservative slant and MSNBC skews liberal may reflect widespread opinion, but a Washington University in St. Louis study suggests that news networks’ biases can be measured.

Published in March 2011 in the journal Behavioral Research Methods, the findings are important in their validation of a new research tool developed by Washington University psychologists, according to lead author Nicholas Holtzman, a graduate student in psychology in Arts & Sciences at the university.

Using the Contrast Analysis of Semantic Similarity (CASS) text analysis software, the researchers evaluated 12 months’ worth of transcripts from MSNBC, FOX and CNN. Confirming their hypotheses and validating the CASS method, FOX demonstrated a conservative bias, while MSNBC exhibited a liberal leaning. CNN fell squarely in between.

Acknowledging the competitive 24/7 news cycle, Holtzman explains that the investigators strategically selected a provocative topic for testing their research method.

“We tried to make a dry process of text analysis a little bit sexier with the media bias component because we’re hoping it will attract some interest so that other researchers will then use this analytic method,” Holtzman says.

Political implications in an election cycle

Tweaking an existing text analysis tool called BEAGLE, developed by co-author Michael Jones and colleagues, the researchers crafted the CASS text analysis method to measure word-to-word correlations. For the study, they focused on 2008 network transcripts containing the word “politics.”

Using four words associated with liberal leanings, four connected to conservatism, and four each denoting negative and positive attributes, the investigators evaluated the number of times one word appeared in proximity with others. For example, the word “Democrat” occurring with a positive word such as “good” or “strong” indicated liberalism; the frequency with which “Republican” occurred with a positive term implied conservatism.

According to Holtzman, the study — which began during one presidential political season and whose results are being released at the start of another — could be valuable in the assessment of candidates.

“I hope that CASS will be used to detect bias in, say, the Republican candidates since we don’t know who the GOP nominee will be,” Holtzman says. “It could help determine who is more right-wing, who is more centered and so forth.”

By comparing speeches and interview texts, researchers also could use the text analysis tool to determine how a politician’s biases shift over time.

“For example, it may be that Barack Obama was substantially more liberal before he became president than he is now. Or maybe George W. Bush was more conservative before he became president,” Holtzman says.

Blogs doubling as Rorschach tests?

A personal blog can speak volumes about its author. Running blog text through the CASS software to assess personality is the next step for Holtzman and his team.

After receiving permission from the bloggers to analyze their text, the investigators will ask bloggers to self-assess their personality traits. Researchers will compare those self-reports to the text analysis results.

“One hypothesis is that people who are narcissistic, for example, will talk about themselves in overly positive ways, and CASS might be able to extract the relevant associations among words — e.g., ‘I’ and ‘wonderful’ in the sentence: ‘I am wonderful,’ reflecting egotistical grandiosity,” Holtzman says.

If researchers can ultimately evaluate personality through blogs, could psychologists in private practice do the same with their clients?

“We’re not there yet; right now we’re just trying to see if it works,” Holtzman says. “Social and personality psychologists do help feed clinicians ideas about how they can help their clients. But it can take 10, 20 or 30 years to get to that point.”

Amateur scientists, beware

In the nascent field of text analysis, which cuts across numerous fields, researchers are happy to share their methods.

“Political scientists are probably just as interested in this as psychologists,” Holtzman says. “We have made connections with people in the political science department at Washington University, and I’ve met with people in the English department. Someone in the business management department at Northwestern University is interested as well.”

In fact, the CASS tool is available for download — to anyone — at casstools.org. It comes complete with exercises for beginners.

What if a senior manager was looking to hire a new associate? Could the software assess candidates’ personalities? It’s possible, Holtzman says.

“Anyone can download the software and perform this analysis,” Holtzman says.

But amateurs should pay attention to the science behind the CASS tool.

“You can’t read someone’s 300-word blog post and make a grand inference about them as a person. We had 50 million words in one of our text totals. The others were between 5 and 10 million,” Holtzman says.

The sheer amount of material available for well-known figures helps ensure the quality of text analysis research on subjects such as national politicians.

“That’s the great thing about analyzing people who are really prominent; there is tons of information about them,” Holtzman says. “So the people we’re most interested in are the people whose analysis would be the most valid."

Explore further: Smartphone app used by experimenters to learn more about aspects of morality

Related Stories

Modern society made up of all types

Nov 04, 2010

Modern society has an intense interest in classifying people into ‘types’, according to a University of Melbourne Cultural Historian, leading to potentially catastrophic life-changing outcomes for those typed – ...

Study finds 'masculine' women get more promotions at work

Jan 27, 2011

Women who demonstrate stereotypical masculine traits should be mindful of their behavior if they want to get ahead in the workplace. That is the finding of researchers at George Mason University and Stanford University who ...

Recommended for you

Religious acceptance of homosexuals on the rise

Sep 12, 2014

The willingness of religious congregations to welcome homosexuals as members—and place them in leadership positions—is on the rise, according to a new Duke University study.

Childhood mentors have positive impact on career success

Sep 11, 2014

New research from North Carolina State University finds that young people who have had mentors are more likely to find work early in their careers that gives them more responsibility and autonomy – ultimately ...

User comments : 13

Adjust slider to filter visible comments by rank

Display comments: newest first

Skultch
3.4 / 5 (5) Apr 15, 2011
I don't watch news on TV. I wonder what is the % of sarcasm used? These are Americans we are talking about; the kings of sarcasm. This could really skew the results if not compensated for, and I don't see how a large sample size alone could correct for that.
tpb
3 / 5 (6) Apr 15, 2011
I would love to know if the words used for selecting liberal-conservative bias were selected or changed to get the result the authors wanted/expected.
These kind of studies are always biased by the results the authors want.
Bob_B
3.7 / 5 (15) Apr 15, 2011
Everyone knows FOX is NOT news. It is all opinion, well maybe the weather isn't opinion, but the rest of FOX is worthless.

I remember when the word FOX at the beginning of a film, I knew it was going to be a good film. Now, FOX is tarnished. Anything with FOX in its' name I avoid as totally worthless.
xznofile
3 / 5 (2) Apr 15, 2011
A lot of ingenuity to reveal voter orientation so PR can spin a product. It's like everyone has assumed political bias except politicians.
Skultch
3.5 / 5 (6) Apr 15, 2011
Fox and MSNBC may be worthless to moderate, thinking, people, but it has great value for those who seek affirmation of their bias.

It's not about ideology as much as it is about market demographics. It's very profitable to be politically polarizing. People want to think their opinion matters and dopamine gets released in their brains whey their opinion is repeated in front of them.
jimbo92107
not rated yet Apr 15, 2011
You can look for statistical correlations with political leanings all you want.

Can it tell you who is lying?

Didn't think so.
SmaryJerry
3.7 / 5 (3) Apr 15, 2011
It sounds interesting but at the same time like it is completely bias towards the programming. The only thing that could prove it's validity is the results. Like Skultch said above, there is sarcasm and numerous ways that the results would be skewed. Like how close does the proxmity have to be and so on. Let's say the sentence is: "There is a strong chance Barack Obama will lose the vote in Florida because of his support and good relationships with anti-orange juice lobbyists" That has strong, good, and support, in proximity with Obama, but how things like that get counted in this system need to be evaluated.
patnclaire
2 / 5 (8) Apr 15, 2011
Using the Contrast Analysis of Semantic Similarity (CASS) text analysis software, the researchers evaluated 12 months worth of transcripts from MSNBC, FOX and CNN. Confirming their hypotheses and validating the CASS method, FOX demonstrated a conservative bias, while MSNBC exhibited a liberal leaning. CNN fell squarely in between.
I have been watching network and cable news for years, just like the rest of you. I would agree with CASS that Fox tends toward the Right while MSNBC tends toward the Left. I think that CNN and its doppelganger HLN tend towards the Left. They let Beck and Dobbs go. Those two gave some balance to the Left-slant that reporters and commentators gave.
I would love to know if the words used for selecting liberal-conservative bias were selected or changed to get the result the authors wanted/expected
Therein lies the rub. The selection of key words as a filter will determine what results shake out. We know that opinion sampling is affected by how the ques
soulman
4.5 / 5 (8) Apr 15, 2011
Everyone knows FOX is NOT news. It is all opinion, well maybe the weather isn't opinion

The weather may not be opinion, but their views on climate certainly are!
Twin
5 / 5 (3) Apr 16, 2011
If "you" have a bias, then any source that presents "both (or more) sides will seem slanted to you.
dutchman
not rated yet Apr 16, 2011
Everyone knows FOX is NOT news. It is all opinion, well maybe the weather isn't opinion, but the rest of FOX is worthless.

I remember when the word FOX at the beginning of a film, I knew it was going to be a good film. Now, FOX is tarnished. Anything with FOX in its' name I avoid as totally worthless.


That may be true for the FOX NEWS channel. However the news on LOCAL affiliates is surprisingly well done and unbiased, despite their propensity to hire newsreaders that apparently "must be" sexy, attractive, preferably blonde, and most of all young females. That is how you can immediately recognize the news on a local Fox affiliate.
Moebius
3 / 5 (2) Apr 17, 2011
Amazing, FOX fans will label CNN liberal and CNN fans will label CNN middle of the road, just as I did a while ago in another post. Maybe not so amazing, you have to be off to be a FOX fan. In fact you have to be incredibly stupid to ignore their constant distortion of the facts and downright lies that they get called on all the time by honest reporters. Or just ignorant, if you watch nothing but FOX you wouldn't know about all the lies since they don't retract them. Intelligent people watch CNN and FOX, always nice to get unbiased news AND biased entertainment.
COCO
not rated yet Apr 19, 2011
Watch Russia Today or dial in WRH and get some fresh and maybe true perspectives.