KNMI projects ECMWF MSCR-29 Highlights

On this page

MSCR-29

ECMWF

ECMWF

Highlights from MSCR, May 2017

Artists' impression of the new Bologna data centre.
Image: gmp von Gerkan, Marg & Partner
This year's meeting of the ECMWF Member State Computing Representatives (MSCR-29) took place from May 17 – 19 at ECMWF in Reading. The meeting was attended by 17 Member States and Co-operating States, another two remotely by Webex, and 15 people from ECMWF itself.

High on the agenda were the plans for a new data centre and the investigations that should lead to a new supercomputer in 2020. There was also a presentation on the status and plans for the next reanalysis (ERA5). And of course there was extensive attention for ECMWF's software and for developments in Member States.
Furthermore, there were presentations on web activities and services, security activities, and the re-engineering of product delivery.

On this page you will find some highlights which might be relevant or otherwise interesting for KNMI and other dutch users of ECMWF computing facilities. More details can be found in the presentations, which are available on the Internet (see the menu on the right).

A new data centre

As ECMWF's current data centre facility at their headquarters in Reading does not offer the required flexibility for future growth and changes in high-performance computing (HPC) technology, the Centre is looking for a new site to operate their HPC and other computing facilities. Earlier this year, the Council decided that the Italian proposal for a new ECMWF data centre in the Tecnopolo di Bologna best answered their requirements.
However, the final decision still has to be taken by the Council, later this year.
On 22 June, at the 90th session of the Centre's Council, the decision was taken to approve the proposal by the Italian Government and the Emilia Romagna Region to host ECMWF’s new data centre in Bologna.

Because the large volumes of data that go back and forth between the HPC and other computing environments at ECMWF the proposition is to relocate the entire data centre to the new site. As the contract for the current Cray XC40 ends at the end of September 2020, the new operational service is planned to be fully up and running by August 2020.
This means that in Reading only some network equipment will remain and the Data Centre will be decommissioned.

Some of the consequences of the choice are

The next HPC

ECMWF is currently investigating possibilities for the successor of the Cray XC40 in 2020. This should lead to an ITT in the second half of 2018 and a contract for the new HPC mid 2019.

Earlier this year, ECMWF invited 1714 external users to fill in a questionnaire on the current HPC and possible wishes for the next one. If you still have useful input for the process, please let ECMWF know.

ERA5 overview, status and release plans

Last year, the production of the next reanalysis (ERA5) has started. Compared to ERA-Interim, the resolution is higher, output is hourly and uncertainty estimates have been added.
In the 2nd or 3rd quarter of this year the period 2010 – near-real-time should become available. Next, 1979 – 2009 should be ready in the first half of 2018 and by the end of 2018 the full period 1950 – near-real-time is expected to be ready.

Software developments

ecCodes

For GRIB handling ecCodes has now officially replaced GRIB_API. There is still limited support for GRIB_API, e.g. tables and urgent bug fixes, but this will end in the autumn of 2018.

Migration of the GRIB handling should be completely transparent. The ecCodes library supplies the same routines and utilities as GRIB_API.
Migration of BUFR handling from BUFRDC will require more work. The BUFR handling in ecCodes works similar to GRIB handling.

If you like to discuss with other users of the software, ECMWF is happy to create a platform on their web services to share your experience.

MIR — the new interpolation package

The new interpolation package MIR is now almost feature-complete with Emoslib and is undergoing very thorough validation and testing.
The first priority is now to replace Emoslib, but suggestions on enhancements from users are also welcome.

ECMWF invites alpha-users, especially specialists with specific requirements (e.g. Hirlam rotated lat/lon grids), to contact them and help with testing.

Metview

Within a few months Metview 5 will be released. The differences with Metview 4 will not be as large as between Metview 3 and 4, but new features warrant a new major version number.
These new features include a new layer management and more control over the different layers; support for colour gradients; support for Flexpart (a Lagrangian transport and dispersion model suitable for the simulation of a large range of atmospheric transport processes).

A new E-learning module on Metview is coming soon.

The Pythonic future

Software Support is aware of the growing popularity of Python among researchers. Python (2) interfaces are already provided for individual packages, but ECMWF is actively looking for ways to build a better Python experience, with external help.
The aim right now is a high level interface implemented as a Python version of the Metview Macro, that is easily installable (e.g. through pip or anaconda).

Survey

At the end of last year, ECMWF did a survey on the use of their software packages. They contacted more than 4000 users of whom 351 responded.
In general, ECMWF are happy with the results, but some areas have been identified where progress can be made, e.g. creating awareness for ecCodes, and documentation. And they will try to follow up with individual users who left interesting input and contact details.
Full details on the survey are available in the presentation.

If you feel something is missing in the survey, ECMWF welcomes your suggestions for extra questions in the next edition.

User Services

Software updates

ECMWF software packages are being updated twice per year (May, November).
The aim is to update compilers once per year, in November.
After the upgrade of the OS on ecgate from RHEL6.4 to RHEL6.8 a decrease in perfomance of MARS was noticed, which eventually was traced back to this upgrade. Therefore, in June, ecgate will move to RHEL6.9 which should resolve these performance issues.

HPC OS upgrade

Early 2018 the OS of the Cray XC40 will be upgraded because Cray stops active development of the current version (CLE 5.2). The new version will also no longer support the current versions of the Cray compilers. This upgrade might have similar impact on the software as a whole new HPC.
The compilers will be upgraded first, while the old versions are still available. The OS will follow later. More detailed plans on the upgrade will be annouced later this year.

MARS

The MARS service has known several performance issues lately. ECMWF is aware of this. It should improve with the new OS on ecgate, but also new servers have been added to the HPC to deal with the requests and extra storage is underway.
And it might help to investigate whether you can optimise your requests. User Support will be happy to assist.

Web API keys

Web API keys used to be valid indefinitely, but from now on, an expiration date of one year has been added. You will be notified one month in advance.

E-learning

ECMWF has started a project to produce e-learning modules. At the end of the summer, 12 modules from various training courses will be publicly available, including several major software packages like Metview, MARS and ecCodes.

Developments in Member States

Some of the topics that are under investigation or development in several Member States
Hans de Vries, © KNMI, June 23, 2017