Forecasting future may one day become as practical as predicting weather, thanks to Big Data advances

June 27, 2014 by Emily Kale
Forecasting future may one day become as practical as predicting weather, thanks to Big Data advances
The Early Model Based Event Recognition using Surrogates project’s mission is to forecast significant societal events in Latin America ranging from protests to politics by analyzing the billions of pieces of data in the ocean of public information such as tweets, website queries, restaurant cancellations, and even satellite imagery.

( —It was standing room only in a crowd of computer engineers, artificial intelligence experts, philosophers, and virtual reality gurus—all gathered around to hear how forecasting the future may one day be as commonplace as forecasting the weather, thanks to advances in the field of Big Data.

"There is essentially more data being uploaded than what we're actually experiencing," says panelist Devi Parikh, assistant professor of electrical and computer engineering, who leads the Computer Vision lab at Virginia Tech. "Every second of life produces somewhere near two hours of content."

Parikh may have startled people unacquainted with high-volume data management and stream computing, but this day in Kelly Hall—home of the Institute of Critical Technology and Applied Science at Virginia Tech—Parikh was surrounded by believers.

After all, this is the home of the Early Model Based Event Recognition using Surrogates project, or EMBERS—Virginia Tech's effort to harness this torrent of data to generate early warnings of events with a broad social impact.

And as Wu Feng, a professor of computer science and electrical and computer engineering, points out, "It is not just the volume of data we are interested in, but the veracity and variety as well."

In the past year, the team has used such an approach to correctly forecast many important events such as the riots after Paraguay's president was impeached, the Hantavirus outbreaks in Argentina, and the recent mass protests in Brazil and Venezuela.

As leaders of the only project to receive the final and third year of support from the Intelligence Advanced Research Projects Activity, the team was eager to share information about its achievements and plans for growth. 

Project leader and Discovery Analytics Center Director Naren Ramakrishnan explained how all of the event alerts are emailed in real time and compared against a gold standard report organized by a third party.

The team actually receives a report card scoring their predictions. They also test what they could have forecasted and determine what they might have overlooked. 

Local project collaborators spawn from the departments of computer science, statistics, and mechanical engineering, and the Virginia Bioinformatics Institute, as well as the Institute of Critical Technology and Applied Science.

Questions arose about machine learning, artificial intelligence, and the social implications of data mining.

"All machines do this to an extent," says Dhruv Batra, assistant professor of electrical and computer engineering who leads the Machine Learning and Perception lab at Virginia Tech. "Auto-fill, search trends, and ad-generating are everyday examples of ."

When people wonder why something like Google or Facebook or Twitter is free, Batra says they are failing to realize that "the product is in fact, you."

Parikh notes that when machines read images it is substantially harder than reading text.  One thing this particular program does is read images of parking lots outside of health centers and hospitals to monitor upticks in "fill rate." 

"Teaching a machine to extract meaning from an image involves labeling an exorbitant number of objects," Parikh explains.

Listening to the team describe their work was akin to taking a trip in a time machine. It is not too far removed from science fiction, and questions about morality quickly sprang forth.

"Let's say you can predict the future and you intervene to prevent an uprising," Parikh says. "How can you be sure that an uprising was actually going to happen?"

 In short, how do you test your theory? 

Anil Vullikanti, a panelist and an associate professor of computer science, had many anecdotes regarding the role of human intervention in data forecasting. 

"You will always have people intentionally trying to clog the system with misinformation to throw you off course, but we are trained to separate the signal from the noise and minimize distractions."

Interestingly enough, the spread of an idea is not so different from the spread of an illness.

As computer networks become better equipped to handle vast amounts of data and our algorithms for interpreting the data become more sophisticated, the team may be able to forecast pivotal social phenomena all over the world.

Explore further: Researcher seeks to lessen failures in computerized visual recognition programs

Related Stories

Machines may help predict, eliminate allergens

April 29, 2014

( —It's well known that allergies and asthma are on the rise around the globe. Food allergies alone rose 50 percent in children from 1997 to 2011, according to a recent study at the Centers for Disease Control ...

Making elastic cloud computing a reality

April 8, 2014

( —University of New South Wales researchers are using artificial intelligence to create a computer network capable of regulating its own consumption of public cloud services.

Efficient processing of big data on a daily routine basis

June 10, 2014

Computer systems today can be found in nearly all areas of life, from smartphones to smart cars to self-organized production facilities. These computer systems supply rapidly growing data volumes. Computer science now faces ...

Recommended for you

When words, structured data are placed on single canvas

October 22, 2017

If "ugh" is your favorite word to describe entering, amending and correcting data on the rows and columns on spreadsheets you are not alone. Coda, a new name in the document business, feels it's time for a change. This is ...

Enhancing solar power with diatoms

October 20, 2017

Diatoms, a kind of algae that reproduces prodigiously, have been called "the jewels of the sea" for their ability to manipulate light. Now, researchers hope to harness that property to boost solar technology.


Adjust slider to filter visible comments by rank

Display comments: newest first

4.8 / 5 (4) Jun 27, 2014
They should give a tip of the hat to Isaac Asimov who developed the idea of predicting the future using computers to analyze past history. His character Hari Seldon developed the discipline called 'psychohistory' in Asimov's "Foundation" series of Sci-Fi novels begun back in the 1950s.
not rated yet Jun 27, 2014
Yes jackjump. That was the first thing I thought of when I started reading the article. I wonder if the originators of this effort somehow got inspiration from reading that series earlier in their life.
1 / 5 (1) Jun 27, 2014
whether seldons insights strengthen the reins of tyranny
or predict the resource requirements of the proletariat

it's not whether the uprising is going to happen , it's about whether it should.

the difference between knowledge and wisdom is a value judgement .
for the sake of the living , let the dead worry about the dead
lost in time , like tears in the rain
5 / 5 (1) Jun 28, 2014
I can't even get past the title on this one. I can predict the future just as accurate as they predict my weather. My weather today - thunderstorms in the evening and there was not one cloud in the sky all night.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.