Microsoft / Technion effort mines old news for predictions

Feb 03, 2013 by Nancy Owano weblog
Main components and ow of analysis of event prediction pipeline. Credit: Kira Radinsky, Eric Horvitz.

(Phys.org)—Microsoft Research and Technion-Israel Institute of Technology have been working on software that can predict events. The pursuit could lead to a tool that can provide better information that goes beyond conclusions and forecasts drawn from human expertise, educated guesses, and intuition. The software might help mine data toward the goal of knowing when outbreaks of disease or outbreaks of violence could occur, among other kinds of information. The software collaboration has involved testing with over 20 years' worth of New York Times articles, taken from an archive from 1986 to 2007, along with various Web data sources, to establish better ways of seeing what leads to major events such as disease and violence.

Eric Horvitz, Distinguished Scientist and co-director of Microsoft Research, teamed up with Technion-Israel Institute's Kira Radinsky, a PhD researcher.

Their system was tested on data where they found patterns and determined correlations between such as drought in Africa with post-drought events such as cholera outbreaks. Following those , alerts about a downstream risk of cholera could have been issued nearly a year in advance.

The researchers described the manner in which they crawled and parsed the archives of New York Times articles. "We say that a chain of events belongs to a domain D, if it consists one of the domain relevant words, denoted as wi(D). For example, for the challenge of predicting future deaths, we consider the wordskilled," dead," death," and their related terms. For the challenge of predicting future disease outbreak, we consider all mentions of cholera, "malaria, " and dysentery."

While they are not unique in exploring conditions surrounding , the researchers noted that epidemiologists pursuing like relationships issue studies that are frequently retrospective analyses rather than predictive studies. The two researchers are looking for a software tool that can guide better decisions for near term actions.

Horvitz said the project will continue. He would like to mine more newspaper archives and digitized books. He is optimistic that a more refined version could assist experts at government agencies planning humanitarian responses among other uses. "We've done some reaching out and plan to do some follow-up work with such people," he said.

In their research paper, "Mining the Web to Predict Future Events," Radinsky and Horvitz wrote that, "Beyond alerting about actionable situations based on increased likelihoods of forthcoming outcomes of interest, predictive models can more generally assist by providing guidance when inferences from data run counter to expert expectations."

They said they hoped their work will stimulate additional research on leveraging past experiences and human knowledge to provide valuable predictions about future events and interventions of importance.

Explore further: New 100 Gbps high-speed transatlantic network to benefit science collaborations

More information: research.microsoft.com/en-us/u… future_news_wsdm.pdf

Related Stories

UN says cholera epidemic in Somalia

Aug 12, 2011

(AP) -- World Health Organization officials said Friday that famine-hit Somalia faces a cholera epidemic as dirty water and poor sanitation are leading to an increase in outbreaks of the disease.

Recommended for you

Sweeping air devices for greener planes

26 minutes ago

The large amount of jet fuel required to fly an airplane from point A to point B can have negative impacts on the environment and—as higher fuel costs contribute to rising ticket prices—a traveler's wallet. ...

World record in data transmission with smart circuits

45 minutes ago

Fewer cords, smaller antennas and quicker video transmission. This may be the result of a new type of microwave circuit that was designed at Chalmers University of Technology. The research team behind the ...

Robots recognize humans in disaster environments

1 hour ago

Through a computational algorithm, a team of researchers from the University of Guadalajara (UDG) in Mexico, developed a neural network that allows a small robot to detect different patterns, such as images, ...

User comments : 8

Adjust slider to filter visible comments by rank

Display comments: newest first

gopher65
not rated yet Feb 03, 2013
I'll be interested to see how this work turns out, and if it can be applied on a large enough scale with sufficiently detailed outputs to be useful.
alekseyt
not rated yet Feb 03, 2013
Asimov's Psychohistory at work.. :)
NeptuneAD
not rated yet Feb 03, 2013
I wonder how smart this sort of predictive ability could become once it was automated, it could have all sorts of uses then.
gwrede
not rated yet Feb 04, 2013
I wish Microsoft would mine their own product news. Also other articles about how their products have fared would offer gold nuggets apparently not known to Microsoft marketing and R&D.
Mrshipley
1 / 5 (1) Feb 04, 2013
I predict that the human race is on its way to hell in a handbasket due to uncontrollable energy use, mismanagement of natural resources and spiralling global population. No web-mining or software required.
Havana
not rated yet Feb 04, 2013
Two related ventures:
- Zapaday.com scans the web for mentions of future events and publishes them as events and calendars
- RecordedFuture.com, a Google CIA start-up, by the way, offers real time analysis of mentions of future events.

It seems Microsoft/Technion is aiming at some kind of 'predictive pattern recognition'. Curious to see that in action.
Dug
1 / 5 (1) Feb 04, 2013
Save us from those that think computer models are anymore predictive than the inaccurate opinions that go into them. Best example - climate change models. Run them backwards and try to predict past climate that has already occurred - yet to find a model that is capable of doing this, so how accurate can their future predictions be? Sounds more like Bible code analysis than science.
ricarguy
1 / 5 (1) Feb 05, 2013
I predict that the human race is on its way to hell in a handbasket due to uncontrollable energy use, mismanagement of natural resources and spiralling global population. No web-mining or software required.


'Mrshipley' forgot the most powerful reasons of all toward "hell in a handbasket". Unsustainable debt of multiple national treasuries and government overreach, of which national debt is the most obvious indication.