KNMI projects ECMWF MSCR-30 Highlights

On this page

MSCR-30

ECMWF

ECMWF

Highlights from MSCR, May 2018

This year's meeting of the ECMWF Member State Computing Representatives (MSCR-30) took place from May 16 – 18 at ECMWF in Reading. The meeting was attended by 21 Member States and Co-operating States, representatives from CTBTO and EUMETSAT, and some 22 people from ECMWF itself.
A short account on the meeting has also appeared in ECMWF Newsletter 156 (see the menu on the right).

On this page you will find some highlights which might be relevant or otherwise interesting for KNMI and other dutch users of ECMWF computing facilities. More details can be found in the presentations, which are available on the Internet (see the menu). There you can e.g. find some statistics, an account on the developments of various web services and examples of the possibilities of the new dissemination editor.

The new data centre

Although the migration to the new data centre in Bologna in 2020 featured at the background of the whole meeting, there was no separate presentation on this topic.
The procurement of the new HPC, that has to be operational by the fall of 2020 has started and the development of the ITT is underway.
ECMWF sees the migration as a good opportunity to end the support of superceded services and legacy software, asks us to clean up user accounts that are no longer needed, and will ask users to remove data that are no longer needed from the archive.

Copernicus

ECMWF hosts the Copernicus Climate Change Service (C3S) and the Atmospheric Monitoring Service (CAMS). An overview was given of the developments in the last year.
The Climate Data Store is being developed “as a one-stop shop to explore climate data”, see the link in the menu.

Cloud computing at ECMWF

With the ever growing demand of data, ECMWF is exploring how to make use of cloud services to provide these to various users all over the world. The Copernicus Climate Data Store is now implemented on a private cloud in the ECMWF data centre.
But more experiments are being carried out, also together with some of the Member States. For operational production they still have to investigate what the use of clould services would mean for availability and timeliness of the forecasts.

Software developments

The ECMWF software packages are constantly evolving, with the focus on performance.
If you have any questions, wishes or issues with the software, please provide your own examples, because the software can and will be used in many ways that ECMWF have not yet considered.

ecCodes

Once again, ECMWF stressed that the support for grib_api ends by the end of this year and that users are urged to migrate. New developments for GRIB and BUFR software are only available in the ecCodes library.
The performance of the library of great importance to the Centre. So, if you experience any issues, please report them. Currently, the improvement of threading for GRIB de/encoding is under investigation.

Python 3

An important target is to fully support Python3 by the end of 2018. Community channels like pypi, pip and conda are being used to distribute the software. Python2 should be abandoned completely once the data centre moves to Bologna.
ECMWF hosted last year a first Python framework workshop for the MetOcean community.

Magics

The major version number of Magics has been stepped up to 3 to indicate a mature version of the library. The biggest change is a new interface both for Python version 2 and 3. Furthermore, the automatic support for NetCDF-CF has been improved and predefined palettes can now be used.

Metview

The major version of Metview has also increased, to 5. It now uses the Qt5 interface only and no longer the Motif interface. That gives a slightly different look-and-feel.
New features include the interactive editing of plots (similar to what was previously possible in Metview 3), new color schemes and a new interface to Flexpart.

Metview 5 will be the first version to support a new Python interface. This new interface aims at combining the features of the Metview macro language with the Python eco-system, without the need to learn the Metview language. The Python code is very similar to the macros. In the Metview GUI python scripts can be created by drag-and-drop of Metview icons.
This summer, there will be a beta release of the interface and at the end of the year the first production release is planned.

MIR — the new interpolation package

MIR is a complete redesign of the ECMWF interpolation library, to replace emoslib. This new library will be much better suited for modern requirements and has been designed with scalability in mind to make optimal use of available hardware for optimal performance.
Presently, MIR is available as an option in MARS. Later this year it will become the default in MARS, and be the default in Metview. By the end of the year also the Product Generation should have completely moved to MIR and emoslib will be decommissioned.
Other interfaces to the library (command line tool, Python) will be developed later, in consultation with users.

Of course, MIR has been very thouroughly tested, also with other packages like CDO and Iris, to make sure the results are correct. This, however, does not mean that the results are exactly equal to the emoslib results.

eLearning

There are now several eLearning modules: on the ecgate batch system, ecCodes, ecFlow, MARS and Metview. Modules on compiling on ecgate and advanced use of ecCodes for GRIB decoding are being developed.
You can check it out through the link in the menu.
Hans de Vries, © KNMI, August 10, 2018