Professor develops mobile app to identify plant species

Jun 08, 2011 By Beth Kwon
Sample images from the LeafSnap mobile application

( -- Not every child can dream up a smartphone application and see it come to life. But that’s what happened when 8-year-old William Belhumeur suggested his father make an app that identifies plants using visual recognition technology.

As a professor of computer science at the engineering school and director of Columbia’s Laboratory for the Study of Visual Appearance, Peter Belhumeur has worked on face recognition software since the mid-1990s. He quickly saw that the same algorithms that can process the curve of an eyebrow or the angle of a cheekbone could be applied to the shape of a leaf.

“The idea of building classifiers that say, ‘Is this person in the photo a man or a woman?’ or ‘Is that leaf a sugar maple or a silver maple?’ uses a lot of the same sort of math and technology,” says Belhumeur.

With the help of computer scientist David Jacobs at the University of Maryland and John Kress, research botanist and curator at the Smithsonian Institution, Belhumeur developed LeafSnap, an electronic field guide that is now available on the iPhone and iPad, and on Android phones later this year. It is easy enough for a child to use, but goes well beyond the basics for botanists.

The team started by photographing leaves from the Smithsonian’s vast library. But they soon realized a viable application would have to be able to recognize leaves in the wild, not just museum specimens. So Belhumeur’s student volunteers collected thousands of leaves from Central Park—up to 50 samples each from the park’s 145 species—and photographed them with their iPhones.

A leaf’s shape is its least variable feature and easiest to capture in a photo, so the team focused on characteristics like smooth versus jagged, many-lobed or single-lobed. They then programmed the computer to perform a sort of process of elimination. “The computer basically ranks images by most similar to least similar,” says Neeraj Kumar, a Ph.D. candidate in computer science who manages LeafSnap’s software coding and is in charge of the volunteer leaf identifying team.

Back in a Schapiro Hall lab, the team trained the computer to distinguish one species from another. “We pick one feature we extract from the leaf, and using that we can say, ‘This looks more like all of these maples I’ve seen and less like something else,’” says Kumar.

The app, which is free, allows a user to photograph a leaf, upload it and see a list of possible matches within seconds. There is also a complementary website ( with profiles of each species. Initial interest is high; the has been installed 150,000 times.

In addition to the Central Park trees, LeafSnap’s database covers the 160 species in Washington, D.C.’s Rock Creek Park; between the two parks, most native species in the northeast are represented. Belhumeur’s team hopes to eventually map species across the United States and use a crowd-sourcing element to let users add their own images to the database. “This is the sort of system we need because species are disappearing off the planet at an alarming rate, and the process of identification is very slow,” explains Belhumeur.

Belhumeur went to Brown as an undergraduate and received his Ph.D. in engineering sciences from Harvard. He came to Columbia in 2002 after eight years as an electrical engineering professor at Yale. For him, LeafSnap bridges his high-tech background and love of nature. As a child in Providence, R.I., he remembers looking up at trees with his parents and trying to identify leaves with a field guide. Now his family has a farm in Cornwall, Conn., where they raise cattle, sheep, pigs, geese and chicken. “It was fun to take that visual recognition technology and drop it into this domain, working with biologists and doing something I cared about as a kid,” he says.

His son is already thinking of new apps. “I think it is time for Fishsnap and Bugsnap,” says William, “so there is still a lot of work to do.”

Explore further: Male monkey filmed caring for dying mate (w/ Video)

Related Stories

Mother and kid goat vocals strike a chord

May 11, 2011

( -- Mother and kid goats recognize each other’s calls soon after the mothers give birth, new research from Queen Mary, University of London reveals.

Modern society made up of all types

Nov 04, 2010

Modern society has an intense interest in classifying people into ‘types’, according to a University of Melbourne Cultural Historian, leading to potentially catastrophic life-changing outcomes for those typed – ...

The kids are alright

May 26, 2011

Children should be seen and not heard... who says? A Philosophy academic at The University of Nottingham is challenging the adage by teaching primary school children to argue properly.

Recommended for you

Male monkey filmed caring for dying mate (w/ Video)

Apr 18, 2014

( —The incident was captured by Dr Bruna Bezerra and colleagues in the Atlantic Forest in the Northeast of Brazil.  Dr Bezerra is a Research Associate at the University of Bristol and a Professor ...

Orchid named after UC Riverside researcher

Apr 17, 2014

One day about eight years ago, Katia Silvera, a postdoctoral scholar at the University of California, Riverside, and her father were on a field trip in a mountainous area in central Panama when they stumbled ...

In sex-reversed cave insects, females have the penises

Apr 17, 2014

Researchers reporting in the Cell Press journal Current Biology on April 17 have discovered little-known cave insects with rather novel sex lives. The Brazilian insects, which represent four distinct but re ...

Fear of the cuckoo mafia

Apr 17, 2014

If a restaurant owner fails to pay the protection money demanded of him, he can expect his premises to be trashed. Warnings like these are seldom required, however, as fear of the consequences is enough to ...

User comments : 5

Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (1) Jun 08, 2011
Methinks Mushroomsnap is in order !!!
not rated yet Jun 08, 2011
something similar to this developed for an MA final thesis back in 2005/2006 for mushroom hunting and identification called mycomob.
not rated yet Jun 09, 2011
something similar to this developed for an MA final thesis back in 2005/2006 for mushroom hunting and identification called mycomob.

Sweet Jesus!

I wonder how much work it would be to convert this to a smartphone app. It may be more difficult to use a visual program to identify mushrooms however, and I'm not sure if I'd trust such a program with the mushrooms that I eat! Who knows though, maybe its possible. Eh he he he he ;)
not rated yet Jun 09, 2011
Can't wait for a fish app added to an underwater camera for scuba divers!
not rated yet Jun 09, 2011
Reminiscent of a Star Trek scanner.

More news stories

Biologists help solve fungi mysteries

( —A new genetic analysis revealing the previously unknown biodiversity and distribution of thousands of fungi in North America might also reveal a previously underappreciated contributor to climate ...

Researchers successfully clone adult human stem cells

( —An international team of researchers, led by Robert Lanza, of Advanced Cell Technology, has announced that they have performed the first successful cloning of adult human skin cells into stem ...

NASA's space station Robonaut finally getting legs

Robonaut, the first out-of-this-world humanoid, is finally getting its space legs. For three years, Robonaut has had to manage from the waist up. This new pair of legs means the experimental robot—now stuck ...

Ex-Apple chief plans mobile phone for India

Former Apple chief executive John Sculley, whose marketing skills helped bring the personal computer to desktops worldwide, says he plans to launch a mobile phone in India to exploit its still largely untapped ...

Filipino tests negative for Middle East virus

A Filipino nurse who tested positive for the Middle East virus has been found free of infection in a subsequent examination after he returned home, Philippine health officials said Saturday.

Egypt archaeologists find ancient writer's tomb

Egypt's minister of antiquities says a team of Spanish archaeologists has discovered two tombs in the southern part of the country, one of them belonging to a writer and containing a trove of artifacts including reed pens ...

Airbnb rental site raises $450 mn

Online lodging listings website Airbnb inked a $450 million funding deal with investors led by TPG, a source close to the matter said Friday.