This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:

fact-checked

trusted source

proofread

Research eyes optimized design for web, mail surveys

Research eyes optimized design for web, mail surveys
Two styles of web survey designs are shown, with radio buttons on the left and wide bars on the right. Credit: Kristen Labadie | University Communication and Marketing

Singular-mode mail and telephone surveys are going the way of the landline—slowly but surely being phased out in favor of more technology-assisted communication.

Once considered the gold standard for surveys, mail and phone questionnaires now largely include a web-based component. These mixed modalities have given researchers new freedom in questionnaire design and lessened costs, but how do they affect ? University of Nebraska–Lincoln scientists in the Department of Sociology and the Bureau of Sociological Research are examining that question, and a new study illustrates that the design of mixed modality surveys can increase data quality.

The novel study, published in the International Journal of Market Research, suggests that using item-by-item displays, which ask single questions with possible answers, rather than grid displays, or groupings of questions and answers in a block, in both mail and web surveys decreases the "straightlining" effect, or respondents choosing the same answer repetitively.

The study also found that using wide bar designs, which are more typical in web surveys, in both mail and web surveys made little difference in response rates and data quality compared to the radio buttons—small fill-in circles—that are much more common in mail-based surveys. The findings also suggest that questionnaire designers can combine radio buttons on mail surveys and wide bars on web surveys with little impact on data quality.

"The prevailing wisdom for questionnaire design with multiple modes is to use what's called a unified mode design, which is to make things look as similar as possible across the modes," said Kristen Olson, lead author of the study and director of the Bureau for Sociological Research. "This shows a different approach, or an optimized mode design, where you optimize the for each mode, can get better quality answers within each mode and therefore minimize measurement error."

Olson said the findings are good news for survey researchers. More and more large sociological and political surveys have had to include more web components as technology changes, and a mixed modality survey is likely to include more varied demographic groups.

"The population that does web surveys are typically younger, more educated and have internet access," Olson said. "With mail, you get those people, too, but you also tend to pick up older people who don't have or a smartphone. If you mix mail and web together, you get a more representative group of respondents. That's well established."

Olson, who served on a for the American Association for Public Opinion Research examining these data quality questions, said most of the previous research on how battery questions in self-administered surveys are designed has almost exclusively looked at web-based surveys, since mail surveys have historically been formatted in grid styles to lessen the cost per page. With the use of web-based components in addition to mail surveys, this new research is among the first to look at both web and mail surveys together. And it suggests using item-by-item questions and answers in both modalities would improve data quality.

"Item-by-item was better for web surveys and for mail surveys," Olson said. "Even though it took up more space on a mail survey, it didn't seem to increase the perceived burden for the respondent. Taking those questions out of grids will increase the quality of data we get."

Olson acknowledged that item-by-item formatted surveys may cost more to produce but said the upshot is having better quality data.

"In this world that we're moving into, where more and more of our surveys are having multiple modes of data collection, it's really important to know that it can be worth the extra money to have an additional page in the questionnaire. It costs more money, but research is showing that it will likely give you a better data."

More information: Kristen Olson et al, Display of battery items in web and mail surveys: Grids versus item-by-item and radio versus wide buttons, International Journal of Market Research (2023). DOI: 10.1177/14707853231210223

Citation: Research eyes optimized design for web, mail surveys (2023, November 16) retrieved 16 April 2024 from https://phys.org/news/2023-11-eyes-optimized-web-mail-surveys.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Surveys with repetitive questions yield bad data, study finds

2 shares

Feedback to editors