Understanding collective animal behavior may be in the eye of the computer

Jan 16, 2014
This shows snapshots of video data of collective behavior (collected by NYU undergraduate student Gozde Ustuner) from experiments with (a) ants, (b) fish, (c) frogs, (d) chickens, and (e) humans. Human faces have been obscured to protect privacy. A machine learning system nearly matched human observation of the video in its assessment of collective behavior of each of the species. Credit: NYU School of Engineering

No machine is better at recognizing patterns in nature than the human brain. It takes mere seconds to recognize the order in a flock of birds flying in formation, schooling fish, or an army of a million marching ants. But computer analyses of collective animal behavior are limited by the need for constant tracking and measurement data for each individual; hence, the mechanics of social animal interaction are not fully understood.

An international team of researchers led by Maurizio Porfiri, associate professor of mechanical and aerospace engineering at NYU Polytechnic School of Engineering, has introduced a new paradigm in the study of social in animal species, including humans. Their work is the first to successfully apply toward understanding collective animal behavior from raw data such as video, image or sound, without tracking each individual. The findings stand to significantly impact the field of ethology— the objective study of animal behavior—and may prove as profound as the breakthroughs that allowed robots to learn to recognize obstacles and navigate their environment. The paper was published online today in Scientific Reports.

Starting with the premise that humans have the innate ability to recognize behavior patterns almost subconsciously, the researchers created a framework to apply that instinctive understanding to machine learning techniques. Machine learning algorithms are widely used in applications like and weather trend data, and allow researchers to understand and compare complex sets of data through simple visual representations.

This video is not supported by your browser at this time.
Human interaction captured on video: People exhibited more social interaction than frogs but far less than ants, whether measured by ISOMAP's machine learning system or humans viewing the videos of 10-day experiments. Credit: NYU School of Engineering

A human viewing a flock of flying birds discerns both the coordinated behavior and the formation's shape—a line, for example—without measuring and plotting a dizzying number of coordinates for each bird. For these experiments, the researchers deployed an existing machine learning method called isometric mapping (ISOMAP) to determine if the algorithm could analyze video of that same flock of birds, register the aligned motion, and embed the information on a low-dimensional manifold to visually display the properties of the group's behavior. Thus, a high-dimensional quantitative data set would be represented in a single dimension—a line—mirroring human observation and indicating a high degree of organization within the group.

"We wanted to put ISOMAP to the test alongside human observation," Porfiri explained. "If humans and computers could observe social animal species and arrive at similar characterizations of their behavior, we would have a dramatically better quantitative tool for exploring collective animal behavior than anything we've seen," he said.

The team captured video of five social species—ants, fish, frogs, chickens, and humans—under three sets of conditions—natural motion, and the presence of one and two stimuli—over 10 days. They subjected the raw video to analysis through ISOMAP, producing manifolds representing the groups' behavior and motion. The researchers then tasked a group of observers with watching the videos and assigning a measure of collective behavior to each group under each circumstance. Human rankings were scaled to be visually comparable with the ISOMAP manifolds.

This video is not supported by your browser at this time.
Whether measured by the ISOMAP machine learning system or by humans viewing videos like this of 10 days of experiments, frogs exhibited the least collective behavior of the 5 species studied. The researchers' findings were in line with known qualities of each species. Credit: NYU School of Engineering

The similarities between the human and machine classifications were remarkable. ISOMAP proved capable not only of accurately ascribing a degree of collective interaction that meshed with human observation, but of distinguishing between species. Both humans and ISOMAP ascribed the highest degree of interaction to ants and the least to frogs—analyses that hold true to known qualities of the species. Both were also able to distinguish changes in the animals' collective behavior in the presence of various stimuli.

The researchers believe that this breakthrough is the beginning of an entirely new way of understanding and comparing the behaviors of social animals. Future experiments will focus on expanding the technique to more subtle aspects of ; for example, the chirping of crickets or synchronized flashing of fireflies.

Explore further: Insect mating behavior has lessons for drones

Related Stories

Recommended for you

Insect mating behavior has lessons for drones

17 hours ago

Male moths locate females by navigating along the latter's pheromone (odor) plume, often flying hundreds of meters to do so. Two strategies are involved to accomplish this: males must find the outer envelope ...

Godwits are flexible... when they get the chance

May 29, 2015

Black-tailed godwits are able to cope with unpredictable weather. This was revealed by a thorough analysis of the extraordinary spring of 2013 by ecologist Nathan Senner of the University of Groningen and ...

Do you have the time? Flies sure do

May 28, 2015

Flies might be smarter than you think. According to research reported in the Cell Press journal Current Biology on May 28, fruit flies know what time of day it is. What's more, the insects can learn to con ...

User comments : 0

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.