New paper shows how "conjoint analysis" can tackle hard political issues

Politics is full of surveys purporting to explain why voters act the way they do. But how can we really pinpoint the factors that explain what happens inside the voting booth?

A new paper co-authored by an MIT political scientist suggests that a polling method known as "conjoint analysis" can get traction on political questions that are hard for traditional surveys to assess accurately.

Suppose you are analyzing a campaign pitting two candidates who vary in several personal attributes—such as religion, ethnicity, or gender—and who have different positions on several issues. One way of conducting traditional polling would be to ask voters how important they consider those factors to be. While informative, the results—where voters describe factors as "somewhat important," "very important," and so on—can also be a bit vague.

Conjoint analysis takes this a step further by polling voters about hypothetical matchups between candidates. By randomly varying the attributes or positions of the candidates in the hypothetical matchup, pollsters and political scientists can better specify the relative weight of each factor in the race.

But there is one drawback with all such analyses, as Teppei Yamamoto, an assistant professor of political science at MIT, says: "They don't actually explain what people do in the real world."

That is, pollsters have long suspected that voters are wary of admitting to, say, ethnic or religious biases. And if that is the case, then all analyses of voters' decisions will inevitably miss the mark, to some degree.

But now Yamamoto and his colleagues have found a way to test conjoint analysis's predictive abilities by pitting it against real-world voting results—decisions by Swiss voters to allow certain immigrants to gain citizenship.

"In Switzerland, people actually did the real-world decision-making," Yamamoto says.

The study, summarized in a newly published paper, shows that conjoint analysis fares best among the survey methods used to explain Swiss votes, coming within 2 percentage points of the real-world outcomes. This is significant in interpreting the results of these elections, suggesting that immigrants' countries of origin were important factors in determining whether they succeeded in gaining Swiss citizenship—precisely the kind of factor that respondents might be inclined to downplay in a poll.

"This is an issue that is consequential," Yamamoto explains. "[The voters] had a real incentive to choose the answer they believed in."

Hypothetical surveys, real votes

Yamamoto's paper is published this week in the Proceedings of the National Academy of Sciences. His co-authors are Jens Hainmueller, an associate professor of political science at Stanford University, and Dominik Hangartner, an associate professor at the London School of Economics.

Hainmueller and Hangartner have previously studied the results of immigration votes in Swiss municipalities, which helped the researchers recognize that the issue could provide data against which conjoint analysis could be evaluated.

From 1970 until 2003, many Swiss municipalities used referendums in which citizens directly decided which immigrants could receive permanent citizenship in the country. In a paper published in 2013, Hainmueller and Hangartner showed that immigrants from Turkey and the former Yugoslavia, all other factors being equal, were 40 percent less likely to gain citizenship.

Meanwhile, in an award-winning paper published in 2014, Yamamoto, Hainmueller, and Daniel Hopkins of Georgetown University described how conjoint analysis could be applied to political votes.

The latest paper, in a sense, merges those efforts by testing the conjoint method against the Swiss votes, with a layer of survey results matched up against the empirical vote outcomes.

"In the real world, there are components of decision-making that don't exist in surveys," Yamamoto adds.

To be sure, the researchers regard the study as just one result in the longer-term application of conjoint analysis. The study also takes recent survey data, from the last few years, and applies it to the period when the Swiss voting system was used for immigration decisions. In the paper, the scholars make the case that conditions are similar enough to validate the meshing of those layers of data.

Still, Yamamoto concludes, "This was a unique opportunity to test this. The result we found is something social scientists would find difficult to elicit from a survey."

This story is republished courtesy of MIT News (web.mit.edu/newsoffice/), a popular site that covers news about MIT research, innovation and teaching.