'Cool' sampling sites more likely to show false trends

To manage and conserve natural ecosystems, it is essential to know how biodiversity changes. As one of those questions, it is important to know whether we are we gaining or loosing species. However, getting reliable measurements to study this is a complex task. Data can be collected by researchers during field trips, but a vast amount of data is also provided by different initiatives such as citizen science programs. However, to ensure measurements are reliable, samples need to be representative of the real world. If samples are not representative, they are biased.

In their study, scientists from iDiv, UL and MLU showed that a bias can dramatically change the measurements of biodiversity change. They focused on the so-called site-selection bias, where sampling sites are not selected representatively. For example, if a person wanted to find out how the number of butterfly species is changing in the area over the next ten years, this person will likely not choose a typical and average spot, such as a little meadow behind a supermarket. Instead, a place such as a clearing in the park may seem well suited. Thus, the sampling site is biased towards where many butterfly species can be found. This is the site-selection bias.

Site-selection bias may amplify or even reverse trends

Using a computer simulation, the scientists showed that this bias can lead to false conclusions about how biodiversity changes. "If we measure biodiversity at places that have unusually many species, statistical principles suggest that we will more likely observe a drop in species numbers over time," said first author Dr. Andrea Mentges from iDiv and UL. The biased choice of where to take samples may manipulate the collected data and even lead to wrong conclusions. "We found that site-selection bias can even make our measurements point into the wrong direction: while for example in reality we are gaining species, our measurements would falsely suggest that the number of species goes down. This makes clear how important it is to choose representative sampling sites."

The scientists analyzed how wide-spread such site-selection biases are, and found that they can potentially occur in many data sources, such as data collected by museums, national parks, citizen scientists and academic researchers. Whether the site-selection bias affects the measurements depends on the purpose of the collection. The data on the butterflies counted in the local park may still be informative, said Andrea Mentges: "Such data can be used to find out if a certain butterfly species can still be found in the area or if new species are immigrating. But if we use it to find out whether butterfly numbers are increasing or decreasing in the surrounding landscape, we might be wrong."

Citizen scientists more likely to choose 'cool' spots

For their study, the scientists also looked at 44 citizen science initiatives. In many programs, participants were allowed to choose sites themselves. This free site selection can potentially lead to a bias—depending strongly on the training participants received. Andrea Mentges said: "We assume that if people are not explicitly told that it is important to choose a 'normal' site they will always tend to choose a 'cool' spot. That's why we think such programs can potentially be problematic when there is no in-person training or instructions available online that also focus on site selection."

In Germany, such training is not very common in citizen science initiatives. Although information material is available to interested participants this may not always prevent a site-selection bias—for example, when participants are encouraged to look for meadows and flower-rich wayside for their observations. "These are great initiatives that collect a lot of useful data," said Andrea Mentges. "But it may not be suitable for some research questions—for example, whether the number of a certain butterfly species such as the small tortoiseshell is going up or down in general." However, such site-selection biases can easily be prevented. Prof Dr. Jonathan Chase, head of the Biodiversity Synthesis group at iDiv and professor at MLU, said: "The most objective way would be to use systematic, computer-based site-selection schemes, these are also applied more and more often. But it's always a fine line between scientific accuracy and practical feasibility. And, of course, citizen science programs owe their existence to the motivation of the many volunteers contributing with their data."