Evolutionary methodology produces more accurate long-term weather forecasts
Paul Roebber sat on the New Jersey beach, looked out at the ocean and let his mind wander. "I started thinking about sharks hunting for prey," the meteorologist says, "and about predicting dynamic systems in nature."
It reminded him how computers let scientists simulate complex systems, such as predator-prey interactions. Weather forecasters, too, rely on statistical models to find and sort patterns in large amounts of data. Still, the weather remains stubbornly difficult to predict.
Roebber has a history of making it less stubborn. He's a leader in bringing new approaches to meteorology, and his research has changed how experts around the world forecast weather. Often, his innovations come from adapting something that has nothing to do with meteorology. And in the summer of 2016, that Jersey Shore reverie about sharks was sparking another one.
Forecasters use "ensemble" models, which average many different weather models, to create more accurate predictions. Roebber wondered if a mathematical equivalent of Charles Darwin's evolution theory might get more out of ensemble forecasting, and do so without requiring additional data, which can be expensive to gather.
He devised a method in which one computer program sorts 10,000 other ones, improving itself with strategies that mimic nature, such as heredity, mutation and natural selection. "This was just a pie-in-the-sky idea at first," says Roebber, a UWM distinguished professor of atmospheric sciences, who had considered it as far back as 2010. "Now, in the last year, I've gotten $500,000 of funding behind it."
This latest forecasting method outperforms the models used by the National Weather Service. And when compared to standard weather prediction modeling, Roebber's evolutionary methodology performs particularly well on longer-range forecasts and extreme events, when an accurate forecast is needed the most.
The importance of meteorology goes far beyond advising people to pack an umbrella for work or send the kids to school in warmer clothes. About 40 percent of the U.S. economy is somehow dependent on weather prediction. Even a small improvement in the accuracy of a forecast could save millions of dollars annually for industries like shippers, utilities and agribusiness.
It's not the first time Roebber has been involved in breakthrough work. In 2007, he launched Innovative Weather, which provides custom weather-related services to clients such as We Energies, the Milwaukee Brewers and the Lake Express ferry. Now in his 22nd year at UWM, he credits natural curiosity and a persistent personality with framing not only his prominent career, but also carrying him through a personal battle with cancer.
Weather forecasting models are far from perfect because complete and exact data is impossible to get. "When we measure the current state of the atmosphere, we are not measuring every point in three-dimensional space," Roebber says. "We're interpolating what happens in the in-between."
Turns out, nature's state of high uncertainty is something you can't just ignore. Its role was first identified in the 1960s by Edward Lorenz, an MIT meteorologist and mathematician.
Lorenz found that when he inadvertently shortened a number with six decimal places to three, the tiny error resulted in unexpected changes to the forecasts.
Roebber explains this concept in terms of narrowly missing a bus. Even though you miss it by only 60 seconds, you still must wait a full 15 minutes before the bus comes around again. Once that 60-second delay becomes 15 minutes, perhaps you to miss an important meeting. While absent, you're assigned a task that requires weeks of your time. Small errors cascade into big consequences.
Lorenz's discovery, called the "Butterfly Effect," became the founding principle of chaos theory, which first acknowledged the overlooked role of apparent randomness in predicting nature. For meteorologists, it meant that obtaining a perfect weather prediction was impossible, especially in the long term.
Interestingly, as a graduate student at MIT in the early 1980s, Roebber took a chaos theory course with Lorenz that proved to be his most difficult class there. It underscored the importance of time spent seeking better forecasts, even though a perfect forecast is unattainable. In meteorology, he says, an incremental gain in accuracy – perhaps only 1 degree Fahrenheit in a longer-range forecast – has a cumulative economic impact.
In college, Roebber reveled in the interdisciplinary aspects of meteorology. "For me," he says, "creativity comes from being open to broad interests."
For example, after reading about the architecture of artificial intelligence, he was inspired to improve how meteorologists predict snow amounts. Such predictions had been particularly challenging because forecasters had no way to know the snow ratio – the amount of water contained in each inch of snow.
That ratio varied widely, so you never knew if you were going to get heavy, wet snow or the light, fluffy kind. The best forecasters could do was predict a large range of inches, which could mean the difference between a fairly minor nuisance and a major plow event.
Roebber built simulations that were organized like networks of neurons in the brain: The computer programs formed a system of interconnected processing units that could be activated or deactivated. This artificial neural networks tool proved especially proficient at predicting scenarios with large data gaps and reams of variables. And it significantly advanced snowfall prediction efforts.
"Paul crosses discipline boundaries like citizens of the EU cross national borders," says Lance Bosart, a distinguished professor of atmospheric science at the State University of New York in Albany.
Bosart is so respected among his peers that the 2017 American Meteorology Society's annual meeting held a symposium in his name. He also happened to oversee Roebber's postdoctoral work. "He was in the absolute forefront," Bosart says, "of using neural networks to improve weather forecasting 10 years ago."
Roebber's creative drive isn't surprising when you consider his upbringing in a family of intellectuals. The son of a chemistry professor at Northeastern University in Boston, Roebber remembers the PBS program "Nova" as a TV staple in the household. By the time he finished high school, he knew he wanted to pursue the physical sciences in college.
During his graduate years at MIT, he studied explosive cyclones, like the one in the film "The Perfect Storm." The resulting thesis continues to be Roebber's most-cited publication by other atmospheric science researchers.
With his latest research, Roebber is taking another brick out of the wall between forecasters and accuracy: the flaws inherent in ensemble modeling.
Lacking the addition of new data, models used in a group tend to agree with one another rather than the actual weather. In other words, the information in each model is often too similar, and in the absence of more diversity, it's hard to distinguish relevant variables from irrelevant ones – what statistician Nate Silver calls the "signal" and the "noise." Roebber's solution applies Darwinian evolution principles.
In nature, species diversity foils the possibility of one threat destroying an entire population at once. Darwin observed this in a population of Galapagos Islands finches in 1835. The birds divided into smaller groups, each residing in different locations around the islands. Over time, they adapted to their specific habitat, making each group distinct from the others.
Roebber decided to use evolution to engineer more variation in forecasting models. He began by subdividing the variables into conditional scenarios: The value of a variable would be set one way under one condition, but be set differently under another condition.
The computer program then picks out the variables that best accomplished the stated goal and recombines them, and does so in meteorological "niches," just as the finches did in their Galapagos niches. The offspring models have the most successful traits.
"We dig deeper into the data by configuring the variables in a different way," Roebber says. "One difference between this and biology is, I wanted to force the next generation to be better in some absolute sense, not just survive."
He is already using the technique to forecast minimum and maximum temperatures for seven days out. And the evolutionary computer program does as well or better than the most skilled human expert forecasts.
In a 2010 paper, Roebber calculated $2 million in potential annual savings for Ohio electric utilities if they'd substitute this evolutionary method for ensemble forecasts.
Being prepared and saving money was the idea behind launching Innovative Weather in 2007. The forecasting group, manned by Roebber, meteorologist Mike Westendorf and a team of skilled student interns, provides a 24/7 service of customized forecasts for a dozen-plus clients. It helps the Brewers decide when to close Miller Park's roof, Lake Express prepare for rough sailings and We Energies handle Wisconsin's worst winter storms.
"When you forecast for clients with specific needs, it's more intense than general forecasting," says Westendorf, Innovative Weather's director. "There are thousands of dollars on the line for these clients who need to ensure that their resources are available at the time they are needed."
Savings come from averting damage to infrastructure, but also from managing staff more efficiently. Too few on duty during bad weather, and customers are left longer without power longer. Too many working adds up to wasted labor costs, especially on weekends and holidays.
"With Innovative Weather, we now receive a range of potential weather scenarios, but also have a realistic idea of the potential severity of an event," says Duane Miller, manager of gas and electric distribution at We Energies, which calls on the service 15 to 20 times a year.
Roebber has developed several predictive tools that set Innovative Weather apart. It was the first to adopt his method of predicting snow in inches. Consulting with We Energies, he also created a lightning-strike detector and a method to identify storms that are likely to cause a power outage.
Innovative Weather's successful launch was one of Roebber's career highlights. But it also marked the start of his most trying personal saga. Because soon afterward, he was diagnosed with Stage 4 colon cancer, and it had already spread to his liver.
Ever the mathematician, he remembers a most daunting statistic: "My five-year survival rate was 7 percent, I was told."
Between 2007 and 2012, he underwent a dozen surgeries and survived three near-fatal complications from pulmonary emboli. Throughout several rounds of chemotherapy, he kept teaching his courses to avoid imposing on colleagues. And the experience changed how he views his professional accomplishments.
"My research has always been important to me," Roebber says. "But when I was sick, I thought about the impact I've had on my students. To me, that's the lasting contribution I give."
Student success and research intertwine in Roebber's world. Once, after disagreeing with someone else's list of traits that are most predictive of grad school student success, Roebber mathematically modeled the question and found a more inclusive set of indicators.
In fact, a main goal in creating Innovative Weather was to offer students paid internship experiences in a field where few are available. It allows both undergraduate and graduate students the chance to be involved in applied research while working with businesses.
In the middle of 2012, Roebber was declared cancer-free. Now, nearly five years later, he couldn't resist redoing the survival math. He puts the conditional probability of living for the next five years at 91 percent. For the general public, it's 93.