Researchers scan video to monitor nighttime behavior of giraffes

Disney researchers have developed a video technique for automatically detecting unusual behaviors of giraffes at night, providing naturalists with an improved tool for monitoring the health and safety of the animals in the wild and under human care.

"Direct observation of animals has provided scientists with many valuable insights, but this approach is rarely sustainable for long periods of time," said Peter Carr, a research engineer at Disney Research. "Automated data collection and analysis promises to provide a more comprehensive understanding of animal behavior, and video cameras are often the most feasible method to perform this monitoring."

Carr, working with Joseph Soltis, a research scientist at Disney's Animal Kingdom near Orlando, Fla., and Victor Gan, a Disney Research lab associate and a graduate student in computer science at the University of British Columbia, invented a method for automatically reviewing the videos of nighttime giraffe behavior captured by infrared thermal cameras and creating a summary that highlights only the behaviors that are unusual.

They will present their findings at the "Automated Analysis of Video Data for Wildlife Surveillance Workshop" held in conjunction with WACV 2015, the IEEE Winter Conference on Applications of Computer Vision, Jan. 6-9, in Waikoloa Beach, Hawaii.

The giraffe, the tallest living terrestrial animal, appears to people as a most distinctive creature, but it poses some recognition challenges to computer vision systems. Though computer vision researchers have had some success in detecting human subjects in videos, less work has been done on animal detection. And giraffes can be particularly difficult to detect because their heads and necks can be configured in a wide variety of poses and, because they so often are among trees and with other giraffes, parts of their bodies are often blocked from view.

The solution that Carr and his team found was to create separate detectors for the giraffe's head and the giraffe's body. They then developed a method - a "neck filter" - for determining which heads are likely connected with which bodies.

The team trained their system to detect giraffes in typical poses. But animal experts can learn the most by studying atypical behaviors.

"In other words, they don't need to know when our detector is successful - when it finds a giraffe going about its normal routine - but when the detector fails, when the giraffe is doing something we couldn't anticipate," Carr said. "It's impossible to directly test for a missed detection, but we can look for clues."

The key, Gan explained, was to keep track of when the head detector would fail. If the detector stopped detecting the head, they would assume that the giraffe was exhibiting unusual behavior. Of course, the same thing would occur if the giraffe entered or exited the scene or was temporarily occluded by a tree or another animal.

When they tested their method, monitoring four giraffes at Disney's Animal Kingdom Lodge over a two-hour period, they found their algorithm could detect all rare events. The resulting summary video, however, was 25 minutes long. By adjusting the method, they could reduce the summary to just five minutes, while detecting five of the seven rare events.

The researchers suggested that incorporating more complex probabilistic models should make it possible to reduce the summarization length while maintaining the sensitivity to rare events.