Mathematics yields techniques for more accurate weather predictions

July 26, 2017, National University of Singapore
Mathematics yields techniques for more accurate weather predictions
Figure 1 shows an illustration of the ensemble Kalman filter concept. Each green dot represents a possible estimate of earth atmosphere at a given time. Forecast simulations, represented by blue dots, are produced based on previous estimates. Then the current satellite image, represented a yellow dot, is assimilated to adjust the forecasts into more accurate estimates. Credit: National University of Singapore

Weather forecasting is one of the most computationally challenging problems in science. Forecasts are produced by simulating a numerical model of the Earth's weather patterns and then predicting its future behaviours. For such simulations to correctly reflect actual weather conditions with high accuracy, it is important to update the model with the latest satellite data. This is an extremely challenging problem, where billions of model components that can affect the weather are involved. Fortunately, there are advanced mathematical structures which can be used to simplify this task.

Computational power available today can process several hundred simulations at one time. According to classical mathematical theory, this would not be sufficient to accurately simulate the complexities of the Earth model. Interestingly, made today can be quite accurate. This is made possible by using an algorithm known as the ensemble Kalman filter (EnKF), which performs well for this task. Such stark contrast between theory and reality is mystifying. Prof TONG Xin from the Department of Mathematics, NUS in collaboration with Prof Andrew MAJDA from New York University, are studying this phenomenon and have shown that the predictive power of EnKF may come from a systematic low effective dimensionality and localisation structure.

The ensemble Kalman filter (EnKF) (see Figure 1) is a popular computational method for forecasting. Existing theoretical studies of the EnKF consider mostly the low dimension-large sample size scenario, in which there is a large number of samples with few model components. Unfortunately, the situation is often reversed in real-life applications. This makes existing theory difficult to improve the performance of existing EnKF algorithms for practical scenarios. In order to resolve this issue, two structures are investigated: a low effective dimension and a localisation covariance (see Figure 2). While these structures were identified heuristically before, a rigorous justification requires knowledge of advanced statistics, dynamical system and control theory.

Figure 2 shows a correlation localisation pattern between components. The component correlation of a 40-dimensional nonlinear turbulent model, the Lorenz 96, is plotted above. Although the dynamics of this model are complicated and nonlinear, the plot shows an obvious correlation (diagonal line pattern). In particular, components which are located away from the diagonal line are close to being independent. Credit: National University of Singapore

This research lays the mathematical foundation for the use of EnKF in high dimensions for selected applications. It provides the fundamental conditions in which EnKF can operate effectively. Armed with these latest findings, new EnKF algorithms with improved accuracy can be developed for selected applications. For example, one important issue for EnKF application is the optimisation of an algorithmic parameter, known as the localisation radius. This problem was studied previously only by trial and error. The results by Prof Tong provide a new theoretical perspective for this problem, enabling users to develop guidelines for determining the localisation parameters.

Despite the fact that EnKF was derived in a linear setting, it works very well with nonlinear models in practice. The current research outcomes from this work are conducted only for linear models and would need to be expanded to cover nonlinear models so as to be able to understand and tap the full potential of EnKF.

Prof Tong said, "Linear and nonlinear problems are vastly different in theory. The former can be efficiently solved even in high dimensions, while the latter is the opposite. However, we see in practice that many nonlinear problems can be solved using linear methods. EnKF is one good example. The reasons behind await our further exploration."

Explore further: Real time forecast of Hurricane Sandy had track and intensity accuracy

More information: Xin T Tong et al. Nonlinear stability and ergodicity of ensemble based Kalman filters, Nonlinearity (2016). DOI: 10.1088/0951-7715/29/2/657

David Kelly et al. Concrete ensemble Kalman filters with rigorous catastrophic filter divergence, Proceedings of the National Academy of Sciences (2015). DOI: 10.1073/pnas.1511063112

Majda AJ; Tong XT, "Performance of Ensemble Kalman filters in large dimensions" COMMUNICATIONS ON PURE AND APPLIED MATHEMATICS Accepted: 2017.

Related Stories

Towards a mathematical theory of PID control

April 4, 2017

The well-known and nearly hundred years old "proportional-integral- derivative" (PID) controller is a linear feedback control method that adjusts the input signal by using the regulation error of the systems only. Because ...

Going to extremes to predict natural disasters

July 10, 2017

Predicting natural disasters remains one of the most challenging problems in simulation science because not only are they rare but also because only few of the millions of entries in datasets relate to extreme events. A systematic ...

The mathematics of weather prediction

May 13, 2016

Dutifully processing 2.8 quadrillion mathematical calculations per second around the clock, these computers — each about the size of a school bus — are the nucleus of weather and climate forecasting in the United States ...

Model for 2-D materials-based RRAM found

May 31, 2017

Graphene and related two-dimensional (2-D) materials have raised massive interest and investment during the last years. However, the amount of 2-D materials-based commercial devices available in the market is still very low.

Recommended for you

When does one of the central ideas in economics work?

February 20, 2019

The concept of equilibrium is one of the most central ideas in economics. It is one of the core assumptions in the vast majority of economic models, including models used by policymakers on issues ranging from monetary policy ...

In colliding galaxies, a pipsqueak shines bright

February 20, 2019

In the nearby Whirlpool galaxy and its companion galaxy, M51b, two supermassive black holes heat up and devour surrounding material. These two monsters should be the most luminous X-ray sources in sight, but a new study using ...

Research reveals why the zebra got its stripes

February 20, 2019

Why do zebras have stripes? A study published in PLOS ONE today takes us another step closer to answering this puzzling question and to understanding how stripes actually work.

Correlated nucleons may solve 35-year-old mystery

February 20, 2019

A careful re-analysis of data taken at the Department of Energy's Thomas Jefferson National Accelerator Facility has revealed a possible link between correlated protons and neutrons in the nucleus and a 35-year-old mystery. ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.