Microsoft / Technion effort mines old news for predictions

February 3, 2013 by Nancy Owano weblog
Main components and ow of analysis of event prediction pipeline. Credit: Kira Radinsky, Eric Horvitz.

(—Microsoft Research and Technion-Israel Institute of Technology have been working on software that can predict events. The pursuit could lead to a tool that can provide better information that goes beyond conclusions and forecasts drawn from human expertise, educated guesses, and intuition. The software might help mine data toward the goal of knowing when outbreaks of disease or outbreaks of violence could occur, among other kinds of information. The software collaboration has involved testing with over 20 years' worth of New York Times articles, taken from an archive from 1986 to 2007, along with various Web data sources, to establish better ways of seeing what leads to major events such as disease and violence.

Eric Horvitz, Distinguished Scientist and co-director of Microsoft Research, teamed up with Technion-Israel Institute's Kira Radinsky, a PhD researcher.

Their system was tested on data where they found patterns and determined correlations between such as drought in Africa with post-drought events such as cholera outbreaks. Following those , alerts about a downstream risk of cholera could have been issued nearly a year in advance.

The researchers described the manner in which they crawled and parsed the archives of New York Times articles. "We say that a chain of events belongs to a domain D, if it consists one of the domain relevant words, denoted as wi(D). For example, for the challenge of predicting future deaths, we consider the wordskilled," dead," death," and their related terms. For the challenge of predicting future disease outbreak, we consider all mentions of cholera, "malaria, " and dysentery."

While they are not unique in exploring conditions surrounding , the researchers noted that epidemiologists pursuing like relationships issue studies that are frequently retrospective analyses rather than predictive studies. The two researchers are looking for a software tool that can guide better decisions for near term actions.

Horvitz said the project will continue. He would like to mine more newspaper archives and digitized books. He is optimistic that a more refined version could assist experts at government agencies planning humanitarian responses among other uses. "We've done some reaching out and plan to do some follow-up work with such people," he said.

In their research paper, "Mining the Web to Predict Future Events," Radinsky and Horvitz wrote that, "Beyond alerting about actionable situations based on increased likelihoods of forthcoming outcomes of interest, predictive models can more generally assist by providing guidance when inferences from data run counter to expert expectations."

They said they hoped their work will stimulate additional research on leveraging past experiences and human knowledge to provide valuable predictions about future events and interventions of importance.

Explore further: Mathematical model predicts cholera outbreaks

More information:

Related Stories

Lifebrowser: Data mining gets (really) personal at Microsoft

March 17, 2012

( -- Microsoft Research is doing research on software that could bring you your own personal data mining center with a touch of Proust for returns. In a recent video, Microsoft scientist Eric Horvitz demonstrated ...

Recommended for you

Roboticists learn to teach robots from babies

December 1, 2015

Babies learn about the world by exploring how their bodies move in space, grabbing toys, pushing things off tables and by watching and imitating what adults are doing.

Xbox gaming technology may improve X-ray precision

December 1, 2015

With the aim of producing high-quality X-rays with minimal radiation exposure, particularly in children, researchers have developed a new approach to imaging patients. Surprisingly, the new technology isn't a high-tech, high-dollar ...

Making 3-D imaging 1,000 times better

December 1, 2015

MIT researchers have shown that by exploiting the polarization of light—the physical phenomenon behind polarized sunglasses and most 3-D movie systems—they can increase the resolution of conventional 3-D imaging devices ...


Adjust slider to filter visible comments by rank

Display comments: newest first

not rated yet Feb 03, 2013
I'll be interested to see how this work turns out, and if it can be applied on a large enough scale with sufficiently detailed outputs to be useful.
not rated yet Feb 03, 2013
Asimov's Psychohistory at work.. :)
not rated yet Feb 03, 2013
I wonder how smart this sort of predictive ability could become once it was automated, it could have all sorts of uses then.
not rated yet Feb 04, 2013
I wish Microsoft would mine their own product news. Also other articles about how their products have fared would offer gold nuggets apparently not known to Microsoft marketing and R&D.
1 / 5 (1) Feb 04, 2013
I predict that the human race is on its way to hell in a handbasket due to uncontrollable energy use, mismanagement of natural resources and spiralling global population. No web-mining or software required.
not rated yet Feb 04, 2013
Two related ventures:
- scans the web for mentions of future events and publishes them as events and calendars
-, a Google CIA start-up, by the way, offers real time analysis of mentions of future events.

It seems Microsoft/Technion is aiming at some kind of 'predictive pattern recognition'. Curious to see that in action.
1 / 5 (1) Feb 04, 2013
Save us from those that think computer models are anymore predictive than the inaccurate opinions that go into them. Best example - climate change models. Run them backwards and try to predict past climate that has already occurred - yet to find a model that is capable of doing this, so how accurate can their future predictions be? Sounds more like Bible code analysis than science.
1 / 5 (1) Feb 05, 2013
I predict that the human race is on its way to hell in a handbasket due to uncontrollable energy use, mismanagement of natural resources and spiralling global population. No web-mining or software required.

'Mrshipley' forgot the most powerful reasons of all toward "hell in a handbasket". Unsustainable debt of multiple national treasuries and government overreach, of which national debt is the most obvious indication.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.