Weather models developed through cloud computing - Weather models developed through cloud computing - @CSC
Weather models developed through cloud computing
Weather forecasting is based on mathematical atmospheric models which calculate forecasts by simulating atmospheric processes. While weather models always involve assumptions and approximations, they are becoming more accurate all the time. New research findings are helping us to understand and thereby to model atmospheric interactions better than before. Growing computing power is enabling faster and more detailed calculations and higher resolution with respect to time and space.
Weather models use classical fluid dynamics equations to describe changes in air pressure, wind, temperature and humidity. These equations are solved using numerical mathematical methods within the model's computational 3D grid. Computation begins with the initial state, based on meteorological observations, from which the model progresses in time steps of a few minutes towards the desired forecast time.
"Atmospheric processes omitted from this dynamic calculation, such as water phase changes, evaporation and precipitation, turbulent mixing and radiative heat transfer, are described parametrically. This means that the effects of these phenomena, which have a smaller size than the model's grid resolution, are added at each time step to the source and sink terms of the equations of the atmospheric dynamics. Parameterisations have a crucial effect on the practical forecasting capabilities of weather models, since they are mainly responsible for the description of the key elements of simulated weather conditions, such as clouds and precipitation, and temperature and wind," explains researcher Niko Sokka of the Finnish Meteorological Institute.
Cloud computing application developed through Nordic cooperation, based on HARMONIE model
The parametrisations, i.e the ‘physics' element, can be separated out from a full-scale, three-dimensional weather model to create a Single Column Model, or SCM. In principle, single column models contain the entire source code of the prediction model. This enables experiments focusing on atmospheric physics within a simplified environment, but using the same code used for actual weather predictions and research. In one-dimensional models, calculations are done in a single vertical "column".
Due to its limitations, a SCM is unsuitable for weather prediction but, being computationally lightweight and easily controllable, it is ideal for the development of weather models, the comparison of various initial values, and for sensitivity tests.
"In a simplified environment of this kind, high time and vertical resolutions are easy to implement i.e. calculations can be performed in high accuracy," Sokka says.
Due to their clarity, column models are also widely used for teaching and education. The 1D version of the 3D HARMONIE-weather prediction model used by the Finnish Meteorological Institute is known as MUSC (Modèle Unifie Simple Colonne). To be fully compatible with the corresponding, comprehensive HARMONIE model, MUSC software includes the entire source code of the 3D model - around two million lines of source code written in different programming languages. In addition, it is dependent on external scientific computing and system libraries.
HARMONIE is a regional atmosphere model system developed by several European countries and used for both weather forecasting and climate simulation. All of the Nordic countries are among the nations involved in the related development activities.
MUSC (Modèle Unifie Simple Colonne) is the column model of the 3D HARMONIE system; the first version of the model was developed by the French Meteorological Institute, Meteo France, during the 2000s. The MUSC model is now maintained through cooperation between the international HIRLAM and ALADIN consortia and is also available for general research purposes.
In many cases, installing the MUSC software in traditional desktops for research and educational use has proven to be fairly laborious and time-consuming:
"In a typical case of a researcher or university student installing the MUSC system into a desktop, too much time is spent on installation and configuration. MUSC-runs performed on different platforms have been challenging due to deviations created by the differences between the computing platforms. In many cases, laborious input and output file transfers are also needed in order to share the initial state and results between researchers," explains Sokka.
The idea of using cloud services as a platform for column model software grew out of these challenges faced by researchers and students. In autumn 2016, the MUSC system was installed on CSC's cPouta cloud service platform, in cooperation with the Norwegian and Swedish meteorological institutes. CloudMUSC was born.
CloudMUSC ready to use from the cloud
CloudMUSC software can be used directly via a cloud implementation service, avoiding laborious installation and the related testing. This spares both the time and nerves of installers.
"CloudMUSC was built as a virtualisation environment installation image - which can easily be transferred, in a functionally identical form, to another cloud service platform - on cPouta's OpenStack cloud platform. When researchers or students use CloudMUSC, installed on the basis of the same image, research runs performed on different cloud platforms are directly comparable," says a delighted Sokka.
NeIC (Nordic e-Infrastructure Collaboration) is an organisation that facilitates the development and operation of high-quality e-Infrastructure solutions in areas of joint Nordic interest. The NeIC is administered by NordForsk and funded by national research funding agencies (in Finland by the Academy of Finland). CSC represents Finland in the NeiC Board
The purpose of NeIC's Glenna Nordic Cloud project is to share knowledge and set best practices on managing cloud services and to create a Nordic federated cloud service, driven by the need of the Nordic researchers.
The project also involved the transfer of a CloudMUSC installation image implemented on cPouta to the corresponding OpenStack cloud platform of the Norwegian Meteorological Institute.
"The transferred installation image was installed and worked without a hitch on the new platform. The MUSC runtime environment is easy to replicate for large numbers of users such as training participants, when using a prepared installation image and the flexible and scalable computing resources of cloud service platforms," Sokka continues.
The source data typically required for a column model can be stored in the cloud in such a manner that it is easily and quickly available for CloudMUSC. A cloud environment also provides new opportunities for efficiently sharing research results between researchers.
MUSC was transferred to a cloud service as a case study for the Glenna project funded by NordForsk.
"The objective of the two-year Glenna project was to support and promote the Nordic countries' use of the cloud service research infrastructure across national boundaries, thereby improving opportunities for cooperation between researchers in the region," says Dan Still of CSC, who worked as a Project Manager on the Glenna project. The second stage of this project, which was begun in early 2017, will continue in the same manner, promoting the success of national cloud and data-intensive computing via Nordic cooperation.
What is a weather model?
The potential of using numerical methods for weather forecasting was identified in the 1920s, but the required computations could only be performed with sufficient speed following the development of computer-based computational power.
A weather model, i.e. a weather prediction model, describes the entire atmospheric system on the basis of mathematical equations derived from physical conservation laws. The model divides the Earth's atmosphere into 3D grid points consisting of several thousands of calculation points in both the horizontal and vertical directions. The model resolution is more accurate, the denser the grid's spacing is. Weather models can be global models which predict atmospheric state across large areas, or more detailed regional models.
In three-dimensional weather models, the Earth's atmosphere is divided into a three-dimensional grid. Picture: Indivisual/Finnish Meteorological Institute
- Forecast calculation begins with an initial state based on current weather observations and the known boundary conditions on the Earth's surface and at the atmosphere's upper boundary. Following this, the equations are solved for each time step at each point of the model's 3D grid until the forecast is ready, explains Laura Rontu, Senior Scientist at the Finnish Meteorological Institute.
Solving these equations is so heavy in computational terms that it is necessary to use the most powerful supercomputers available.