Bird watching in the 21st century

Jul 18, 2014 by Abby Robinson

Was that a catbird or a mockingbird you just saw? The answer is now just a smartphone tap away. Birdsnap—a new app developed with the help of a University of Maryland computer scientist—can identify birds from photos, using methods borrowed from facial recognition software. Simply snap a photo of a bird, and the app can tell you what species it is and all about it.

Birdsnap is the latest in a series of electronic field guides that David Jacobs, professor of at UMD, has helped to develop with his colleague Peter Belhumeur, professor of computer science at Columbia University. Their developments are based on the idea that the same techniques used by computers to recognize faces could also be used to identify plants and animals.

In 2011, Jacobs and Belhumeur launched Leafsnap, a free iPhone app that identifies trees from pictures of their leaves, in partnership with W. John Kress, research scientist and curator in the Department of Botany at the Smithsonian's National Museum of Natural History.

Encouraged by their success with plants, Jacobs and Belhumeur decided to try identifying animals. They originally chose to look at dogs and released the corresponding app, Dogsnap, in 2012. However, there's not a huge need to identify dogs from photos, according to Jacobs, so he and Belhumeur sought a more in-demand use for their techniques. Birdsnap, which debuted in Apple's App Store in May, extends the methods of the earlier apps to identifying birds.

To identify a bird from a photo, the app first identifies the different parts of the bird, such as the beak, based on thousands of examples of what each body part looks like. It also considers the beak's position relative to the other parts of the bird, such as the tail and wings. Next, each feature is compared against a database of the 500 most common bird species in North America. The algorithm calculates the likelihood of each species matching all of the features and returns the most likely candidates.

One way in which Birdsnap improves on Leafsnap and Dogsnap is by incorporating available data on bird sightings from the Cornell Lab of Ornithology. Users specify the date and place where a photo was taken, and Birdsnap looks up which birds are most common in that location at that time of the year. This helps the algorithm to narrow the list of possible matches.

However, Birdsnap goes beyond simple visual recognition. The app, along with its associated website (www.birdsnap.com), contains a wealth of information about the birds in its database. Birds can be sorted according to how common they are in a particular place and time or according to their evolutionary relationships. The app also shows which birds are arriving, departing or migrating through, based on the location and time of year. Clicking on a bird will show you photos, a description (sourced from Wikipedia and independently verified), and maps showing the bird's range and likelihood of sighting in different places throughout the year, as well as play audio files of the bird's song. It also shows side-by-side comparisons of similar species that highlight the most distinguishing features.

Although Birdsnap is geared toward recreational bird watchers, Jacobs notes that the , and others like it, could be useful for serious research. "There's a lot of interest in studying how the distribution of species is varying over time," Jacob says. "Are we seeing in parts of the country where we didn't typically see them?" Such information could be useful in studying the effects of climate change, for instance.

Jacobs' next steps include looking at other animals and other parts of the world (a version of Leafsnap for the United Kingdom was released May 16, 2014). Jacobs hopes to create a system that automatically generates new field guides, so that "if someone wanted to create a for beetles, that person could take images of different species of beetles and we could automatically generate a field guide that included those species."

Explore further: Computer science professor builds web app for bird identification

add to favorites email to friend print save as pdf

Related Stories

Crow or raven? New birdsnap app can help

May 28, 2014

Researchers at Columbia Engineering, led by Computer Science Professor Peter Belhumeur, have taken bird-watching to a new level. Using computer vision and machine learning techniques, they have developed ...

What's that bird? Check your smart phone

Jan 15, 2014

The Cornell Lab of Ornithology has released a free iPhone app to help people identify 285 birds in North America. Created with support from the National Science Foundation, the app asks just five questions, then displays ...

Professor develops mobile app to identify plant species

Jun 08, 2011

(PhysOrg.com) -- Not every child can dream up a smartphone application and see it come to life. But that’s what happened when 8-year-old William Belhumeur suggested his father make an app that identifies ...

Recommended for you

Bird brains more precise than humans'

20 minutes ago

(Phys.org) —Birds have been found to display superior judgement of their body width compared to humans, in research to help design autonomous aircraft navigation systems.

User comments : 0