Is online research losing its edge? The case for in-person research in the age of AI

The landscape of research has been profoundly transformed by the COVID-19 pandemic, propelling a mass transition to online research methodologies. However, the advent of sophisticated AI technologies capable of mimicking human responses has introduced new complexities, potentially undermining the authenticity of data collected through online surveys.

As the Director of the Interdisciplinary Behavioral Research Center (IBRC) at Duke University, Patty Van Cappellen, Ph.D., is both a witness to the allure of online research and an advocate for the indispensable value of in-person research, especially with the threat of AI contamination.

The whispers of concern are growing louder in the academic community that relies on online survey data to draw scientific conclusions. The concern with AI's interference in online surveys is multifaceted.

AI like ChatGPT can be used to answer surveys, generating full responses for open-ended questions, selecting multiple-choice answers, and even watching videos in place of the participant to provide answers to content questions or even opinions.

Why do participants even bother to use AI to complete surveys? There are probably many reasons. For one, this saves time and effort for people who are often paid very little for their work (researchers often pay the minimum rate, i.e., $6 for CloudResearch or $8 for Prolific platforms for example) or who have limited resources.

Whatever the reason, this new phenomenon raises the question: How can we learn about people's opinions, emotions, beliefs, behaviors, if the data we get is made up by ChatGPT?

This phenomenon isn't just a hiccup; it's a siren call to reevaluate the quality and authenticity of online research data. Could this signal a renaissance for in-person research?

The controlled environment of in-person research has always been its hallmark, ensuring that participant responses are genuine and not influenced by the distractions or multitasking common with online surveys. But the inherent value of in-person research extends beyond control and precision.

In-person research allows researchers to employ nuanced designs and gather ecologically valid data that digital methods simply cannot capture. As such, it is still the preferred method for collecting behavioral measures, physiology data, video and audio from participants' interactions, to cite a few examples. Moreover, nothing can substitute the hands-on experience and direct human interaction offered by in-person studies.

It's a vital space for refining research methods and gaining candid participant feedback, particularly valuable in early-stage research. It also provides an indispensable learning ground for students and emerging researchers to engage intimately with the nuances of human behavior—a dimension of education that virtual environments cannot match.

However, the truth is, online research isn't going anywhere. Its convenience and expansive reach remain unmatched, especially for engaging hard-to-reach demographics or conducting cross-cultural studies. Given this landscape, we must both fortify our methodologies with safeguards against AI manipulation and facilitate access and reach for in-person research.

The IBRC supports researchers engaged in social and behavioral science on both counts. Patty Van Cappellen highlights two particularly relevant resources. They offer a large and versatile research space, with individual and group testing rooms equipped with computers. The IBRC is conveniently located on 9th street and has free parking to facilitate in-person research.

They also manage a large community participant pool, which can be leveraged for both in-person and online studies. For in-person studies, the pool is meant to increase ease of recruitment without sacrificing representativity and diversity, a common critique against in-person research that has often relied on convenience student samples.

For online studies, the pool offers additional safeguards against low engagement by paying participants a living wage. In addition, they provide advice to add data quality checks for online studies.

In sum, with the rise of AI contaminating online research, in-person research stands to regain prominence, not just for its data integrity but for the invaluable insights gleaned from direct human interaction.