A new way to get climate information

April 20, 2012 By Douglas Gantenbein

Climate scientists, environmental scientists, governmental officials, shipping companies, business owners, farmers—all need access to accurate information about the weather.

The challenge certainly isn’t the availability of data about weather. are swimming in it. The National Weather Service maintains a thousand weather stations in the United States, for instance, all of which constantly collect data on temperatures, wind speed, rainfall, relative humidity, and more. Other weather-focused sensor networks, such as FLUXNET, create additional data streams.

What’s difficult is extracting information from all that data—information people can use to make decisions about managing forests, planting crops, or even deciding where to build a wind farm or a hotel. To make such decisions, people need reliable information on the climate—the typical weather patterns they can expect to encounter at different times and places.

Providing reliable climate information is the object of a new analytical tool called FetchClimate, a cloud-based, intelligent data-retrieval service.

FetchClimate, which drew a great deal of interest in March during Microsoft Research’s annual TechFest, is the creation of Drew Purves, head of the Computational Ecology and Environmental Science Group, part of the Computational Science Lab at Microsoft Research Cambridge.

Purves worked with Microsoft Research colleague Vassily Lyutsarev, and a team of students from Moscow State University, supported by Microsoft Research Connections.

With FetchClimate, scientists and others can retrieve vast amounts of climate information without laboriously searching through thousands of data files. FetchClimate queries multiple data sources, then transforms the data as needed, and searches can be refined by region, dates, resolution (such as square miles or square yards), and much more.

Purves uses the example of forests in the eastern United States as an example of how FetchClimate can help researchers.

He says that huge amounts of forest data have been collected for decades—measuring tree growth and deaths as well as different aspects of forest health, sometimes down to plots of land only a few meters square.

“It’s a fantastic set of data,” Purves says. “But we want to relate what we can see going on in the forest data to the climate. Let’s say you go to one particular place and ask, ‘What was the average temperature between 1992 and 1997?’ That’s really easy to say, but actually retrieving that information is simply awful, and when you finally do, you’re probably rewarded with a 25-gigabyte binary file that you can’t even open.”

The difficulty in extracting such seemingly rudimentary information from climate data is what led Purves to help create FetchClimate.

“I imagined my ideal climate-information service,” he says, “and had it built.”

With FetchClimate, researchers can get climate information via a web-based interface or by inserting a few lines of code inside a .NET program. And because FetchClimate is “intelligent”—it selects the best possible data source for the data a researcher seeks—it not only finds desired information, but it also offers an estimate of the accuracy of that information.

FetchClimate, Purves says, can offer tremendous value to climate scientists studying issues such as climate change. For these scientists, the first step in determining what might happen to the Earth’s atmosphere, oceans, and geography is to relate processes such as cloud formation or tree growth to the current climate. FetchClimate can help with that by mining vast stores of data and enabling scientists to view the results in many ways.

Importantly, FetchClimate isn’t a short-term weather forecaster—the National Weather Service and its international counterparts can handle that. Rather, it focuses on what the average conditions are in an area. That’s more important for large-scale, long-term planning based on expected weather conditions for, say, a typical summer in Britain, although FetchClimate also lets you see how those typical patterns have changed over time.

FetchClimate is not just for climate scientists. An agricultural scientist might want to relate crop yields to expected annual rainfall in an area. An epidemiologist might want to understand how large-scale temperature change affects the spread of mosquitoes. A hotel owner might want to compare typical winter temperatures and snowfall before investing in a ski resort. And skiers might want to know those typical temperatures and snowfall amounts before deciding on their next vacation spot.

Purves often uses the example of wind farms.

“To plan where to put those and to predict how much power they might give you, you don’t need to know whether it’s going to be windy tomorrow,” he says. “You need to know how windy different places are on average and how the wind speed varies throughout the year or throughout the day.”

FetchClimate can find that information in a few seconds with a few clicks of a mouse.

“Of course, wind farms are only one example,” Purves says. “Think of solar panels, or planning what crops to plant, or, for that matter, projecting sales of winter coats or air-conditioning units.”

The project came to fruition with remarkable speed. Purves began thinking in earnest about such a tool in late 2010. Microsoft Research Connections provided funding for two students in Moscow, who, along with Lyutsarev, wrote the basic underpinnings of FetchClimate in two months. Four months later, they had added features such as Dynamic Data Display, what FetchClimate uses to visualize data. They also created a browser-based version of FetchClimate, which is more user-friendly.

Purves describes FetchClimate’s “intelligence” as the ability to execute a set of rules against data it finds and then choose an appropriate algorithm for mining that data and turning it into information.

“I think of FetchClimate as having a thin layer of intelligence between the user and the data,” he says. “In the future, it might have a much thicker layer, so you can ask higher-level questions. You might say to it, ‘Give me the prediction of wheat yields here,’ and it could identify and combine the appropriate data to answer that question.”

Purves and his Research Cambridge colleague Matthew Smith, have used FetchClimate in a study of terrestrial carbon—specifically, modeling the ability of plants to absorb excess carbon dioxide and act as a brake for rising global temperatures. The researchers were able to use FetchClimate to create an easy-to-use model that quickly shows the carbon-sequestration potential for nearly any spot on the planet, thanks to FetchClimate’s ability to tag a wealth of carbon data with climate information.

“You had carbon models before, but they weren’t properly tied to data,” Purves says, “so, actually, they were kind of guesses. We’ve got the only model of the carbon cycle that would stand up in court.”

Purves also sees large-scale modeling of the potential impact of climate change as a key role for FetchClimate. By taking past data about rainfall, temperatures, and other weather information, and adding predictions from various models, FetchClimate could be used to model scenarios such as the implications for Seattle if its weather conditions began to resemble those of San Francisco.

Explore further: New "normals" give an updated look at weather patterns

Related Stories

Climate and the statistics of extremes

January 18, 2012

(PhysOrg.com) -- Swiss mathematicians have shown that the risk of extreme climate events is largely underestimated. They are developing a model for better understanding the impact of climate change.

Predicting the microbial 'weather'

April 17, 2012

New computer models are letting scientists forecast changes in the population of microbes in the English Channel up to a week in advance.

Free tool kit to assist big-data scientists

July 19, 2011

Two years ago, during a cyberinfrastructure meeting convened by the U.S. National Science Foundation, principal investigators from across the country found their scientific concerns begin to converge.

Recommended for you

Heavy oils and petroleum coke raising vanadium emissions

December 15, 2017

Human emissions of the potentially harmful trace metal vanadium into Earth's atmosphere have spiked sharply since the start of the 21st century due in large part to industry's growing use of heavy oils, tar sands, bitumen ...

Climate change made Harvey rainfall 15 percent more intense

December 14, 2017

A team of scientists from World Weather Attribution, including researchers from Rice University and other institutions in the United States and Europe, have found that human-caused climate change made the record rainfall ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.