Fewer willing to participate in surveys

June 7, 2013 by Bård Amundsen & Else Lie, The Research Council of Norway
DO YOU HAVE TWO MINUTES? Many people are fed up with being contacted more and more often by marketing surveys, and this is affecting response rates for serious social science research. Credit: Shutterstock

The percentage of respondents to many important research surveys is dropping sharply. Social science researchers are concerned that survey samples can be skewed, which can compromise the validity of results.

For 40 years the Norwegian Institute for Alcohol and Drug Research (SIRUS) conducted an annual survey of young people aged 15–20. The responses provided researchers and the authorities with insight into drug and alcohol use in this age group in Norway. But in 2009 the surveys came to a halt.

"At that point the response rate had fallen from roughly 70 per cent to just over 30 per cent," says Ingeborg Rossow, a senior researcher at SIRUS.

According to the researchers, such a low response rate could cast doubt on the comparability of the results over time. The alcohol and drug use survey was showing that young people in Norway in recent years were drinking significantly less alcohol compared to young people in the 1980s and 1990s. But what if these figures reflected a situation in which only the "good" pupils were answering the survey, making results unreliable? The drug and alcohol researchers did not believe this to be the case, but they could not be certain.

"We now face a similar problem when it comes to surveys about alcohol use amongst the ," says Dr Rossow. "There, too, we must ask ourselves whether the responses we get are representative of the population at large."

Eight percent responding

Ottar Hellevik, a professor of at the University of Oslo, is a nestor in social science-related methodology. He heads up the Norsk Monitor survey, which measures changing values in the Norwegian population. Professor Hellevik says that from years past would be shocked at today's response rates.

"In the Norsk Monitor survey, roughly one-fourth of people we ring up will actually participate," explains the professor. "When we follow up those telephone surveys with a written survey, we are left with a full response rate of around eight per cent. This is far lower than in the 1980s."

"We have to ask ourselves whether the sample of individuals who agree to take part in the survey is adequately representative. Actually, there is much that suggests that the refusals are quite random, which gives us reason to believe we can still use the results we get."

Professor Hellevik is certain that serious surveys suffer from the public's exposure to too many marketing surveys.

"Surveys are an important instrument for social science research in general. If this instrument starts to become compromised, we may need to introduce some form of regulation for who is allowed to conduct surveys," he proposes.

Young males respond least, older women most

"Response rates to surveys began declining suddenly here in Norway around five to six years ago," says Kirstine Kolsrud, a section head at Norwegian Social Science Data Services AS (NSD) in Bergen. NSD assists the research sector with a range of services including collecting data.

In general, NSD sees that young, single men living in urban areas are the least likely to respond, while older women are the most willing. Patterns vary, however, depending on the survey topic. "When we ask people about health issues," says Ms Kolsrud, "we can still get answers from quite a large proportion of those we contact."

NSD contacts people by sending them a survey form in the post, which they may also answer using the Internet. Whereas NSD could once count on a response rate of 60 to 70 per cent of those asked to participate, that figure has now dropped to 40 to 50 per cent. "Officially our target rate is still 70 per cent, but today we know this is unrealistic." Nonetheless, NSD is confident its survey responses are from a representative sample.

Worrisome for health research

"We see a downward trend in response in health-related research, too," notes Camilla Stoltenberg, Director General of the Norwegian Institute of Public Health in Oslo. "But in no way do we get the impression that people are less positive about the actual research being done."

"The decline is most likely due to people reaching a saturation point with too many surveys. Everyone is asking for market feedback, from the hotel you last stayed at to the online bank you use."

Dr Stoltenberg is concerned that the lower participation in health surveys can negatively affect health care services, since health researchers and the health care system often have to depend on people's feedback to further develop these services.

"At the Norwegian Institute of Public Health, we see a steady rise in the need for updated knowledge. One example involves surveys on the quality of health care services, which is a topic currently under close political scrutiny. To be able to assess quality, we are dependent on high participation in our surveys. If we need feedback on, say, whether a new surgery method is better or worse than the current method, we need responses from as many of the patients who have undergone the operations as possible – and preferably from all of them."

International trend

The problem of falling response rates to surveys is not specific to Norway. NSD assists in conducting the European Social Survey (ESS), an ongoing face-to-face interview survey to quantify socio-economic issues, values, attitudes and policies in most European countries. There have been major changes in response rates in many countries.

"Traditionally, Norway and the other Nordic countries have had fairly high response rates," says Kirstine Kolsrud. "But now the Eastern European countries have the highest participation, with 70 per cent or higher in countries such as Poland and the Czech Republic. At the top is Bulgaria with 81 per cent. Norway is in the middle ranks at 58 per cent. The lowest rate is in Germany, at 30.5 per cent."

A good deal of international research is being carried out on response rates in research and how this can affect research results. What characterises those who do not respond? Are the samples of distinctly skewed? Researchers largely agree that while high participation does not guarantee a representative sample group, it does raise the likelihood that the sample represents the intended target group.

Explore further: Just a drop? Alcohol consumption much higher than reported in England

Related Stories

Recommended for you

Study sheds new light on ancient human-turkey relationship

January 17, 2018

For the first time, research has uncovered the origins of the earliest domestic turkeys in ancient Mexico. The study also suggests turkeys weren't only prized for their meat—with demand for the birds soaring with the Mayans ...

Lifting barriers to citizenship for low-income immigrants

January 15, 2018

Taking the Oath of Allegiance at a naturalization ceremony is an emotional moment for many immigrants, and for good reason: it is the culmination of an often arduous process and many years of striving. Citizenship also opens ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (1) Jun 07, 2013
"Traditionally, Norway and the other Nordic countries have had fairly high response rates," says Kirstine Kolsrud. "But now the Eastern European countries have the highest participation, with 70 per cent or higher in countries such as Poland and the Czech Republic. At the top is Bulgaria with 81 per cent. Norway is in the middle ranks at 58 per cent. The lowest rate is in Germany, at 30.5 per cent."

This paragraph probably tells the whole story.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.