Research accelerated with computerized system that analyzes animal videos

December 15, 2014, Princeton University
Princeton researchers have developed a computerized system for evaluating animal behavior. A video system records an animal, in this case a fruit fly, for one hour. Then a computer program analyzes the video to create a map of the behaviors. The map can reveal, for example, the differences in movements between male and female fruit flies. Credit: Joshua Shaevitz

Studies of animal behavior have come a long way from the days when scientists followed their subjects around with pen and notepad. But although cameras have replaced clipboards, evaluating the resulting videos is still a cumbersome process.

To make that job easier and more comprehensive, researchers at Princeton have built a computerized system that analyzes videos to reveal what animals do, how often and for how long, and then generates an easy-to-understand map of the behaviors.

By looking at the map, the researchers can tell what sorts of movements the animal did without having to spend time watching the video. They used the system to track the of (Drosophila melanogaster) and found that it accurately detected behaviors such as grooming a leg or waggling a wing.

The researchers' goal was a system that could create accurate records of behaviors for use in studies of the mechanisms behind those behaviors – in other words, the genes and brain circuits that govern movements. Knowing which genes and neural circuits govern behavior will help answer basic scientific questions and could shed light on the mechanisms behind conditions such as autism.

Described in the journal The Royal Society Interface, the system was built for monitoring Drosophila flies, which are commonly used in studies of genes, but the researchers say that the system can also analyze the movements of other creatures, including worms, mice and humans.

The new system is a significant advance over current approaches because it takes note of all the animal's activities, not just behaviors that seem important to researchers, said Joshua Shaevitz, associate professor of physics and the Lewis-Sigler Institute for Integrative Genomics, who led the study. "We don't know which behaviors are important to a fruit fly, or a mouse," he said. "Instead, we ask the computer to find behaviors that are frequently repeated, which tend to be the ones that are worth studying."

Individual fruit flies were filmed for about one hour, resulting in over 300,000 video frames. Then, a computer program converts the fly’s position in each frame into mathematical descriptions that represent movements. The computer then groups the movements on a map, so that one region of the map represents leg movements while another represents wing movements, and so on. By looking at the map, the researchers can tell what sorts of movements the fly did, such as grooming its front leg, grooming its head, or waggling its wing. Credit: Joshua Shaevitz

Researchers need reliable ways to catalog behaviors in studies of how genes and neural pathways control behavior. A common experiment involves deleting a gene in an organism such as a fruit fly or worm to look for any resulting loss of function or change in the organism's behavior. But without reliable behavioral data, researchers cannot make firm conclusions about the role of the deleted gene.

With the new system, a high-speed camera records the animal for a given period of time, and the computer sorts the resulting video frames. For the current study, the researchers recorded individual fruit flies for a period of one hour, resulting in an enormous amount of data. The challenge was how to sort the frames and translate the images into data that could be evaluated.

"We have to figure out how the body parts of the fly are positioned in relation to each other at each point in time, and then find a way to tell how those are moving, while keeping the amount of data manageable using the available computing power," said Gordon Berman, an associate research scholar in the Lewis-Sigler Institute for Integrative Genomics. "Then you have to project the information into some form that allows you to understand the data."

The researchers accomplished this goal by writing a computer program that converts the information about a fly's position and movements into mathematical descriptions that are then grouped according to their similarities. The computer places these groupings on a two-dimensional map, so that different regions of the map represent different types of movements.

To verify that the system worked, the collected data from 50 male and 50 female flies. They identified hundreds of behaviors, including ones that had never been documented and others that were consistent with findings from human observers. For example, some of the findings consistent with previous observations were that male fruit flies kick out their legs when grooming, whereas females don't, and that young females are more active than young males. "Males and females do things slightly differently, and we can pick up on that right away," said Shaevitz.

Male fruit flies kick out with their legs when grooming:

whereas females do not:

The team is now expanding the capabilities of the method, Shaevitz said. "The technique that we have developed can also be used to study patterns of behavior in humans," Shaevitz said. "There are several conditions that are diagnosed on the basis of behavior, such as autism. Having a more quantitative way of measuring behavior could improve the accuracy of diagnoses."

Explore further: Dust-covered flies reveal hidden logic of grooming behavior

More information: Gordon J. Berman, Daniel M. Choi, William Bialek, Joshua W. Shaevitz. "Mapping the stereotyped behaviour of freely moving fruit flies." The Journal of the Royal Society Interface (2014). DOI: 10.1098/rsif.2014.0672 Published online 20 August 2014 http://rsif.royalsocietypublishing.org/content/11/99/20140672

Related Stories

Dust-covered flies reveal hidden logic of grooming behavior

August 20, 2014

(Phys.org) —A dust-covered fly might spend more than 20 minutes removing the offending grit and grime from its body. With only its own legs to complete the chore, the fly needs a plan of attack. Thanks to a strategy hardwired ...

JAABA: New software speeds analysis of animal behavior

December 10, 2012

(Phys.org)—Using new software developed at HHMI's Janelia Farm Research Campus, a computer can be trained to recognize characteristic animal behaviors like an experienced biologist. Rather than scrutinizing hours of video ...

Identifying and tracking fruit flies with fluorescence

November 8, 2012

(Phys.org)—FBI, a camera system to recognzie the genetic identity of fruit flies and track their movements will revolutionize the study of their behavior. An article on this advance is being published today in PLOS ONE.

How female fruit flies know when to say 'yes'

October 7, 2014

A fundamental question in neurobiology is how animals, including humans, make decisions. A new study publishing in the open access journal PLOS Biology on October 7 reveals how fruit fly females make a very important decision: ...

Recommended for you

A decade on, smartphone-like software finally heads to space

March 20, 2019

Once a traditional satellite is launched into space, its physical hardware and computer software stay mostly immutable for the rest of its existence as it orbits the Earth, even as the technology it serves on the ground continues ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.