Observing group-living animals with drones and computer vision

A drone is flying over a herd of plains zebras in central Kenya. It is flying quite high so that the animals are not bothered by it. These zebras are really interesting for collective and spatial behavior studies, according to the researchers Ben Koger and Blair Costelloe, who are monitoring the drone.

The plains zebras live in multi-level societies: small groups of females and a male combine to form larger herds of dozens of animals. This social and spatial structure could influence behavioral processes such as decision-making and information sharing and have implications for understanding our own complex societies. Traditionally, it has been very difficult to conduct this kind of research. But new techniques their team has developed using imaging drones and artificial intelligence open up new possibilities.

To explore animal groups such as zebras or gelada monkeys, Ben Koger, Blair Costelloe, Iain Couzin, and other researchers from the Max Planck Institute of Animal Behavior, the "Centre for the Advanced Study of Collective Behavior" (CASCB) at the University of Konstanz, and Aarhus University developed a new method for collecting data about animal behavior and the animals' surrounding natural physical landscape using drones and computer vision.

The researchers use imaging drones to record entire groups of animals in natural settings. Behavioral ecologist Blair Costelloe describes the method: "We created an analytical pipeline that lets us take aerial drone footage and extract information about the locations, movement, and behavior of the animals. We can measure their spatial distribution and their behavioral states and get rich information about their surroundings, including the 3D-structure of the environment."

Bringing tracking from the lab to the field

Previously researchers mostly got high precision data sets about animal group dynamics in highly-controlled labs conditions where you could repeat experiments over and over. But the team asked themselves, "Could we use imaging drones and new computer algorithms to take the same lab approaches but bring them into the natural landscapes?"

It is possible—but several challenges had to be solved. "We were often recording 20 or more different individuals at a time. Quantifying where each of the individuals is in a single half hour video observation as a human would take weeks," Ben Koger explains. "The first challenge was how could we automatically detect the animals we were interested in?"

The solution was training powerful deep learning algorithms. The second challenge: The researchers were interested in the animals' movements, and yet the videos they recorded included not only animal movement but also drone movement and distortions from the hilly landscape they were filming over. All those different elements needed to be untangled before they could get meaningful data.

Advantages of the new method

"The power of our image-based method is that it's a general solution," Koger says. Since the drones not only observe the animal group but also the landscape, you get a very broad data set, which includes information on the social and environmental context of all animals of the observed group.

This is possible because they explicitly model the 3D landscape they are recording. This means the method can be used in any open landscape and lets researchers explicitly examine the effects of habitat on behavior. "That's a really powerful approach that so far has been very difficult," Blair Costelloe says.

Another advantage, unlike another common method, is that animals don't need to be captured and fitted with movement sensors, which can be a risky and expensive procedure, especially when working with endangered species such as the Grevy's zebra.

Potential for use

Worldwide, wildlife populations are declining due to habitat loss, climate change, and other threats. Learning more about how groups of animals behave in complex natural environments can help inform conservation actions, and also generate new insights into the lives and behavior of wildlife species.

In their paper, published in the Journal of Animal Ecology, the team outlines certain areas of research where their method has a strong potential to generate new insights, such as spatially mediated behavioral processes, multi-animal collective behaviors, and animal-environment interactions.

The paper includes case studies on Grevy's zebras in Kenya and gelada monkeys in Ethiopia. "One of the strengths of our methods is that it can be adapted to a lot of different species and environments," Blair Costelloe says. That is why she is optimistic for the use of the new method. "I think there's a potential for this method to help us develop more of a mechanistic understanding of how individual behaviors generate the higher-order phenomena that are of interest for conservation," says Costelloe.

The team is now working on the generated data to soon give more insights into the group behavior of geladas as well as African ungulates such as zebras.