Understanding collective animal behavior may be in the eye of the computer

Jan 16, 2014
This shows snapshots of video data of collective behavior (collected by NYU undergraduate student Gozde Ustuner) from experiments with (a) ants, (b) fish, (c) frogs, (d) chickens, and (e) humans. Human faces have been obscured to protect privacy. A machine learning system nearly matched human observation of the video in its assessment of collective behavior of each of the species. Credit: NYU School of Engineering

No machine is better at recognizing patterns in nature than the human brain. It takes mere seconds to recognize the order in a flock of birds flying in formation, schooling fish, or an army of a million marching ants. But computer analyses of collective animal behavior are limited by the need for constant tracking and measurement data for each individual; hence, the mechanics of social animal interaction are not fully understood.

An international team of researchers led by Maurizio Porfiri, associate professor of mechanical and aerospace engineering at NYU Polytechnic School of Engineering, has introduced a new paradigm in the study of social in animal species, including humans. Their work is the first to successfully apply toward understanding collective animal behavior from raw data such as video, image or sound, without tracking each individual. The findings stand to significantly impact the field of ethology— the objective study of animal behavior—and may prove as profound as the breakthroughs that allowed robots to learn to recognize obstacles and navigate their environment. The paper was published online today in Scientific Reports.

Starting with the premise that humans have the innate ability to recognize behavior patterns almost subconsciously, the researchers created a framework to apply that instinctive understanding to machine learning techniques. Machine learning algorithms are widely used in applications like and weather trend data, and allow researchers to understand and compare complex sets of data through simple visual representations.

This video is not supported by your browser at this time.
Human interaction captured on video: People exhibited more social interaction than frogs but far less than ants, whether measured by ISOMAP's machine learning system or humans viewing the videos of 10-day experiments. Credit: NYU School of Engineering

A human viewing a flock of flying birds discerns both the coordinated behavior and the formation's shape—a line, for example—without measuring and plotting a dizzying number of coordinates for each bird. For these experiments, the researchers deployed an existing machine learning method called isometric mapping (ISOMAP) to determine if the algorithm could analyze video of that same flock of birds, register the aligned motion, and embed the information on a low-dimensional manifold to visually display the properties of the group's behavior. Thus, a high-dimensional quantitative data set would be represented in a single dimension—a line—mirroring human observation and indicating a high degree of organization within the group.

"We wanted to put ISOMAP to the test alongside human observation," Porfiri explained. "If humans and computers could observe social animal species and arrive at similar characterizations of their behavior, we would have a dramatically better quantitative tool for exploring collective animal behavior than anything we've seen," he said.

The team captured video of five social species—ants, fish, frogs, chickens, and humans—under three sets of conditions—natural motion, and the presence of one and two stimuli—over 10 days. They subjected the raw video to analysis through ISOMAP, producing manifolds representing the groups' behavior and motion. The researchers then tasked a group of observers with watching the videos and assigning a measure of collective behavior to each group under each circumstance. Human rankings were scaled to be visually comparable with the ISOMAP manifolds.

This video is not supported by your browser at this time.
Whether measured by the ISOMAP machine learning system or by humans viewing videos like this of 10 days of experiments, frogs exhibited the least collective behavior of the 5 species studied. The researchers' findings were in line with known qualities of each species. Credit: NYU School of Engineering

The similarities between the human and machine classifications were remarkable. ISOMAP proved capable not only of accurately ascribing a degree of collective interaction that meshed with human observation, but of distinguishing between species. Both humans and ISOMAP ascribed the highest degree of interaction to ants and the least to frogs—analyses that hold true to known qualities of the species. Both were also able to distinguish changes in the animals' collective behavior in the presence of various stimuli.

The researchers believe that this breakthrough is the beginning of an entirely new way of understanding and comparing the behaviors of social animals. Future experiments will focus on expanding the technique to more subtle aspects of ; for example, the chirping of crickets or synchronized flashing of fireflies.

Explore further: The secret's in the (robotic) stroke: Researchers tease out cues that impact schooling fish behavior

Related Stories

Recommended for you

Male monkey filmed caring for dying mate (w/ Video)

Apr 18, 2014

(Phys.org) —The incident was captured by Dr Bruna Bezerra and colleagues in the Atlantic Forest in the Northeast of Brazil.  Dr Bezerra is a Research Associate at the University of Bristol and a Professor ...

Orchid named after UC Riverside researcher

Apr 17, 2014

One day about eight years ago, Katia Silvera, a postdoctoral scholar at the University of California, Riverside, and her father were on a field trip in a mountainous area in central Panama when they stumbled ...

In sex-reversed cave insects, females have the penises

Apr 17, 2014

Researchers reporting in the Cell Press journal Current Biology on April 17 have discovered little-known cave insects with rather novel sex lives. The Brazilian insects, which represent four distinct but re ...

Fear of the cuckoo mafia

Apr 17, 2014

If a restaurant owner fails to pay the protection money demanded of him, he can expect his premises to be trashed. Warnings like these are seldom required, however, as fear of the consequences is enough to ...

User comments : 0

More news stories

Biologists help solve fungi mysteries

(Phys.org) —A new genetic analysis revealing the previously unknown biodiversity and distribution of thousands of fungi in North America might also reveal a previously underappreciated contributor to climate ...

Researchers successfully clone adult human stem cells

(Phys.org) —An international team of researchers, led by Robert Lanza, of Advanced Cell Technology, has announced that they have performed the first successful cloning of adult human skin cells into stem ...

NASA's space station Robonaut finally getting legs

Robonaut, the first out-of-this-world humanoid, is finally getting its space legs. For three years, Robonaut has had to manage from the waist up. This new pair of legs means the experimental robot—now stuck ...

Ex-Apple chief plans mobile phone for India

Former Apple chief executive John Sculley, whose marketing skills helped bring the personal computer to desktops worldwide, says he plans to launch a mobile phone in India to exploit its still largely untapped ...

Filipino tests negative for Middle East virus

A Filipino nurse who tested positive for the Middle East virus has been found free of infection in a subsequent examination after he returned home, Philippine health officials said Saturday.

Egypt archaeologists find ancient writer's tomb

Egypt's minister of antiquities says a team of Spanish archaeologists has discovered two tombs in the southern part of the country, one of them belonging to a writer and containing a trove of artifacts including reed pens ...

Airbnb rental site raises $450 mn

Online lodging listings website Airbnb inked a $450 million funding deal with investors led by TPG, a source close to the matter said Friday.