JAABA: New software speeds analysis of animal behavior

December 10, 2012, Howard Hughes Medical Institute

(Phys.org)—Using new software developed at HHMI's Janelia Farm Research Campus, a computer can be trained to recognize characteristic animal behaviors like an experienced biologist. Rather than scrutinizing hours of video to catalog how often fruit flies chase one another, for example, scientists can quickly teach the software what to look for, and let it convert the video into useful statistics.

With affordable technology for capturing video and new research tools that enable large-scale studies, some behavioral scientists are finding themselves with more data than they could ever hope to analyze. "We need automated techniques to keep up with the technology," says Janelia fellow Kristin Branson, who led the development of the new . Branson's team developed the Janelia Automatic Animal Behavior Annotator, or JAABA, to automate and standardize the essential but time-consuming task of classifying animal behavior.

Branson's team described JAABA in the December 2, 2012, issue of the journal . They have shown the software can be trained to recognize behaviors in a variety of animals, including adult fruit flies, fruit fly larvae, and mice. The software is freely available for download at http://jaaba.sourceforge.net and can be used by biologists with no computer science or machine learning expertise, Branson says.

Branson says her own lab at Janelia collects about 100 hours of fruit fly videos each week, with 20 flies moving about in each video. The flies in the videos have been genetically modified so that researchers can switch on small subsets of , and Branson and her colleagues would like to determine how that genetic tweak affects the flies' behavior. But the detailed necessary to detect subtle differences in behavior are daunting.

"It's cheap and easy to collect a lot of video," she says. "It's not that easy to say anything quantitative about it." Branson and many of her colleagues use computer vision software she and others have developed to track animals' movements in the videos, generating information about the animals' trajectories and positions in each frame. "This gives us some very simple information, like speed or how long an animal spends in a certain part of its environment," she says. "But we knew there was much richer information there."

To make that information more accessible, Branson set to work writing a program that could be trained to recognize characteristic behaviors the same way experienced scientists do. The result was JAABA.

To use JAABA, scientists begin by training the software. For each frame of a training video, the user tells the computer whether a particular behavior—such as walking or wing grooming—is taking place. Using features that can be computed from just a few frames, such as an animal's speed or distance to other animals, combined with context gleaned from surrounding frames of the video, JAABA develops criteria for detecting the behavior.

Branson says the software learns quickly, producing behavior detectors based on training data in just 15 to 45 seconds. That means users can test its ability to predict a behavior at any time during the training process. The immediate feedback can help users understand how the software is performing and recognize areas in which it requires additional training, she says. "Most biologists don't have a background in machine learning, so having an interactive framework where you can see what the classifier can and can't do gives them some comfort interacting with these complicated algorithms," she explains.

Branson says her team made the first prototype of JAABA within a few weeks. Then they spent the next two years ensuring the program would work consistently on diverse data, and converting their basic algorithms into software that would be accessible to without computer science expertise. "The basic idea was quite simple," she says, "but what took a lot of time was making it very robust and usable."

Once Branson and her colleagues had made the program more intuitive, they gathered 12 fruit fly researchers into a conference room, loaded JAABA onto their laptops, and gave them a 15-minute introduction to the software. The users were able to train JAABA to identify chasing behavior in fruit flies with greater than 97 percent accuracy.

JAABA not only frees scientists from manually identifying behaviors, it also standardizes the definition of those behaviors. The same video might be analyzed slightly differently by two different scientists, Branson explains, especially at points where an animal is transitioning between behaviors, or with individuals who behave less stereotypically than others. Automating the analysis helps ensure consistency, she says.

Branson adds that while certain behavioral patterns—such as those associated with courtship and locomotion—are well defined in model organisms like mice and , there are many unstudied behaviors that could be informative to scientists. Eventually, she hopes that machine learning programs like JAABA will help scientists define what constitutes a behavior and discover new ones.

Explore further: Researchers train computers to analyze fruit-fly behavior

Related Stories

Researchers train computers to analyze fruit-fly behavior

April 8, 2009

Scientists at the California Institute of Technology (Caltech) have trained computers to automatically analyze aggression and courtship in fruit flies, opening the way for researchers to perform large-scale, high-throughput ...

Identifying and tracking fruit flies with fluorescence

November 8, 2012

(Phys.org)—FBI, a camera system to recognzie the genetic identity of fruit flies and track their movements will revolutionize the study of their behavior. An article on this advance is being published today in PLOS ONE.

Space -- the next frontier for Hillary Clinton?

April 26, 2012

She has been the US secretary of state, a senator and nearly became president, but Hillary Clinton joked Thursday that she might want to try another role -- space tourist.

Recommended for you

Quick quick slow is no-go in crab courtship dance

January 16, 2018

Female fiddler crabs are sensitive to changes in the speed of a male's courtship display, significantly preferring displays that accelerate to those that are performed at a constant speed or slow down.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.