Human eyes assist drones, teach machines to see

March 21, 2016

Drone images accumulate much faster than they can be analyzed. Researchers have developed a new approach that combines crowdsourcing and machine learning to speed up the process.

Who would win in a real-life game of "Where's Waldo," humans or computers? A recent study suggests that when speed and accuracy are critical, an approach combing both human and machine intelligence would take the prize. With drones being used to monitor everything natural disaster sites, pollution, or wildlife populations, analyzing drone in real-time has become a critically important challenge. Publishing in the journal Big Data, researchers, including Stéphane Joost from EPFL, present a new approach to rapidly interpret aerial images taken by camera drones that combines human crowdsourcing and machine learning.

To develop and test their approach, the researchers traveled to the Kuzikus wildlife reserve in the heart of Namibia. There they surveyed the compound using camera-drones to count the resident wildlife population, including ostriches, kudus, wildebeests, zebras and rhinos. Confronted with the daunting number of images captured, and the difficulty of having computers reliably distinguish between rocks, bushes, and animals, the researchers turned to the internet. Using an online crowdsourcing platform,, they uploaded their images for manual analysis by an army of digital volunteers.

The task put before the digital volunteers was straightforward. They were asked to click through a stack of images, identify all animals, and outline them on their screens. According to the study's senior author Stéphane Joost, the response was astounding: "Within two days, they had evaluated 98% of the 26,000 images that had been uploaded." Next, half of these annotated images were used to train an automatic object recognition algorithm, which was then tested on the remaining images. "The 500 digital volunteers did generate a number false positives, tracing features that in actual fact were not animals. Despite that, their analysis was certainly good enough to serve as training data for the computer algorithm," says Joost.

Essential aid following disasters

Artificial intelligence has already become a mainstay in disaster response operations, where it is primarily relied on to scour tweets and text messages for what can be life-saving information. "Twitter can erupt with up to tens of thousands of tweets sent out of the disaster zone per minute. It would take humans days to sort the relevant from the irrelevant information," says Joost. "Today, this challenge has been solved and tweets can be analyzed quickly using a combination human and artificial intelligence. Our challenge was to transpose this solution to apply it to the analysis of image data."

From high above, drones carrying high-resolution cameras offer rescue workers an invaluable vantage point from which to locate damaged buildings, blocked roads, and other important features. And fortunately, automatically spotting and identifying animals in the Savanna is likely to be far more challenging than locating infrastructure destroyed in a disaster. Following the completion of the pilot study in Namibia, the new tool was used in its first serious mission: it was adopted by the World Bank to speed up damage and needs assessment in the aftermath of tropical cyclone Pam that hit the Island State of Vanuatu in 2015.

Explore further: Drones revolutionize ecological monitoring

More information: Ferda Ofli et al. Combining Human Computing and Machine Learning to Make Sense of Big (Aerial) Data for Disaster Response, Big Data (2016). DOI: 10.1089/big.2014.0064

Related Stories

Drones learn to search forest trails for lost people

February 10, 2016

Researchers at the University of Zurich, the Università della Svizzera italiana, and the University of Applied Sciences and Arts of Southern Switzerland have developed software enabling drones to autonomously detect and ...

Mining social media can help improve disaster response efforts

January 20, 2016

Leveraging publicly available social media posts could help disaster response agencies quickly identify impacted areas in need of assistance, according to a Penn State-led team of researchers. By analyzing the September 2013 ...

Recommended for you

Microsoft aims at Apple with high-end PCs, 3D software

October 26, 2016

Microsoft launched a new consumer offensive Wednesday, unveiling a high-end computer that challenges the Apple iMac along with an updated Windows operating system that showcases three-dimensional content and "mixed reality."


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.