Microsoft / Technion effort mines old news for predictions

February 3, 2013 by Nancy Owano weblog
Main components and ow of analysis of event prediction pipeline. Credit: Kira Radinsky, Eric Horvitz.

(—Microsoft Research and Technion-Israel Institute of Technology have been working on software that can predict events. The pursuit could lead to a tool that can provide better information that goes beyond conclusions and forecasts drawn from human expertise, educated guesses, and intuition. The software might help mine data toward the goal of knowing when outbreaks of disease or outbreaks of violence could occur, among other kinds of information. The software collaboration has involved testing with over 20 years' worth of New York Times articles, taken from an archive from 1986 to 2007, along with various Web data sources, to establish better ways of seeing what leads to major events such as disease and violence.

Eric Horvitz, Distinguished Scientist and co-director of Microsoft Research, teamed up with Technion-Israel Institute's Kira Radinsky, a PhD researcher.

Their system was tested on data where they found patterns and determined correlations between such as drought in Africa with post-drought events such as cholera outbreaks. Following those , alerts about a downstream risk of cholera could have been issued nearly a year in advance.

The researchers described the manner in which they crawled and parsed the archives of New York Times articles. "We say that a chain of events belongs to a domain D, if it consists one of the domain relevant words, denoted as wi(D). For example, for the challenge of predicting future deaths, we consider the wordskilled," dead," death," and their related terms. For the challenge of predicting future disease outbreak, we consider all mentions of cholera, "malaria, " and dysentery."

While they are not unique in exploring conditions surrounding , the researchers noted that epidemiologists pursuing like relationships issue studies that are frequently retrospective analyses rather than predictive studies. The two researchers are looking for a software tool that can guide better decisions for near term actions.

Horvitz said the project will continue. He would like to mine more newspaper archives and digitized books. He is optimistic that a more refined version could assist experts at government agencies planning humanitarian responses among other uses. "We've done some reaching out and plan to do some follow-up work with such people," he said.

In their research paper, "Mining the Web to Predict Future Events," Radinsky and Horvitz wrote that, "Beyond alerting about actionable situations based on increased likelihoods of forthcoming outcomes of interest, predictive models can more generally assist by providing guidance when inferences from data run counter to expert expectations."

They said they hoped their work will stimulate additional research on leveraging past experiences and human knowledge to provide valuable predictions about future events and interventions of importance.

Explore further: Lifebrowser: Data mining gets (really) personal at Microsoft

More information:

Related Stories

Lifebrowser: Data mining gets (really) personal at Microsoft

March 17, 2012

( -- Microsoft Research is doing research on software that could bring you your own personal data mining center with a touch of Proust for returns. In a recent video, Microsoft scientist Eric Horvitz demonstrated ...

UN says cholera epidemic in Somalia

August 12, 2011

(AP) -- World Health Organization officials said Friday that famine-hit Somalia faces a cholera epidemic as dirty water and poor sanitation are leading to an increase in outbreaks of the disease.

Recommended for you

Samsung to disable Note 7 phones in recall effort

December 9, 2016

Samsung announced Friday it would disable its Galaxy Note 7 smartphones in the US market to force remaining owners to stop using the devices, which were recalled for safety reasons.

Swiss unveil stratospheric solar plane

December 7, 2016

Just months after two Swiss pilots completed a historic round-the-world trip in a Sun-powered plane, another Swiss adventurer on Wednesday unveiled a solar plane aimed at reaching the stratosphere.

Solar panels repay their energy 'debt': study

December 6, 2016

The climate-friendly electricity generated by solar panels in the past 40 years has all but cancelled out the polluting energy used to produce them, a study said Tuesday.


Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Feb 03, 2013
I'll be interested to see how this work turns out, and if it can be applied on a large enough scale with sufficiently detailed outputs to be useful.
not rated yet Feb 03, 2013
Asimov's Psychohistory at work.. :)
not rated yet Feb 03, 2013
I wonder how smart this sort of predictive ability could become once it was automated, it could have all sorts of uses then.
not rated yet Feb 04, 2013
I wish Microsoft would mine their own product news. Also other articles about how their products have fared would offer gold nuggets apparently not known to Microsoft marketing and R&D.
1 / 5 (1) Feb 04, 2013
I predict that the human race is on its way to hell in a handbasket due to uncontrollable energy use, mismanagement of natural resources and spiralling global population. No web-mining or software required.
not rated yet Feb 04, 2013
Two related ventures:
- scans the web for mentions of future events and publishes them as events and calendars
-, a Google CIA start-up, by the way, offers real time analysis of mentions of future events.

It seems Microsoft/Technion is aiming at some kind of 'predictive pattern recognition'. Curious to see that in action.
1 / 5 (1) Feb 04, 2013
Save us from those that think computer models are anymore predictive than the inaccurate opinions that go into them. Best example - climate change models. Run them backwards and try to predict past climate that has already occurred - yet to find a model that is capable of doing this, so how accurate can their future predictions be? Sounds more like Bible code analysis than science.
1 / 5 (1) Feb 05, 2013
I predict that the human race is on its way to hell in a handbasket due to uncontrollable energy use, mismanagement of natural resources and spiralling global population. No web-mining or software required.

'Mrshipley' forgot the most powerful reasons of all toward "hell in a handbasket". Unsustainable debt of multiple national treasuries and government overreach, of which national debt is the most obvious indication.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.