Computer science professor builds web app for bird identification

(Phys.org) —Digital technology is about to add big data to the bird enthusiast's traditional tools of binoculars and a field guide.

Peter Belhumeur, a Columbia computer science professor whose app for recognizing leaves was launched in 2011, has now created Birdsnap, an electronic guide for identifying birds. Birdsnap uses the computer technology that can recognize human faces to identify 500 common birds in North America.

"It's all part of the same thing, using this technology to recognize the things around you," says Belhumeur. While state-of-the-art facial recognition algorithms identify similarities between parts of the human face—the nose, chin or eye, for instance—Birdsnap homes in on parts of a bird—the beak, eye, wing, neck or feet—and finds visual similarities to other birds. "It's all automatic," he says.

An expert in facial recognition, Belhumeur built Birdsnap with David Jacobs, a computer science professor at the University of Maryland, and a computer science Ph.D. candidate at Columbia, Thomas Berg.

Birdsnap not only identifies birds, it can also show users how to make an identification. "The parts of the bird that the computer looks at are often the same as the parts a person would look at to distinguish between birds," says Berg. "So by displaying these key parts of the bird, we can teach someone to better identify birds by themselves, even when they don't have Birdsnap with them."

Demonstrating the website in his office, Belhumeur pulls up a side-by-side comparison of the snowy egret and the great egret. The differences show up on the screen with circles around distinguishing characteristics. The snowy egret has a black beak while the great egret has a yellow one, as well as a longer neck.

Although Belhumeur says he and his colleagues were "building a field guide," Birdsnap is much more than the kind of dog-eared manual that serious birders might keep in their knapsacks.

Users can search for birds throughout the United States, in a particular region or locally. They can sort the birds from most to least frequently seen or organize the birds according to evolutionary history in a "tree of life." Icons indicate which birds are arriving, departing or migrating through the area on the date of the search. Click on a specific bird and you get photos, a description, maps, sound recordings and information such as the scientific names for the order, family, subfamily, genus and species. Upload a photo of a bird you've seen, and the software will attempt to identify it.

Belhumeur says he enjoys working across disciplines—in this case, with biologists. "It's fun to come in as a computer scientist to say, 'Here's something we can do with your data,'" he says. "I get to take this technology and move it into a space that I am very interested in."

Leafsnap, the tree-recognition app that Belhumeur developed with Jacobs in collaboration with the Smithsonian Institution, took nearly 10 years to develop, in large part because of the time-consuming and expensive process of collecting and photographing leaves. It now has more than a million users. Birdsnap, which was developed using resources already available online, was up and running in about six months, with an iPhone app coming in early June.

Bird photos were taken mostly from Flickr images already tagged with a scientific name; parts were labeled by workers found through an online labor market; and descriptions were sourced through Wikipedia and then verified. The maps were based on data from eBird, a joint venture of Cornell University's Lab of Ornithology and the National Audubon Society, and BirdLife, an international network of conservation groups.

"I love doing this," says Belhumeur, who enjoys taking photographs and walking outdoors at his family farm in Connecticut. "It's a lot of fun and I get to look at pictures of birds all day."