This year's meeting of the ECMWF Member State Computing Representatives (MSCR-31) took
place from October 23 – 25 at ECMWF in Reading. The meeting was attended by 21 Member
States and Co-operating States, representatives from CTBTO and EUMETSAT, and some 24 people
from ECMWF itself.
On this page you will find some highlights which might be relevant or otherwise
interesting for KNMI and other Dutch users of ECMWF computing facilities. More details can
be found in the presentations, which are available on the Internet (see the menu). There
you can e.g. find some statistics, an account on the developments of various web services
and examples of the possibilities of the new dissemination editor.
BOND (Bologna Our New Datacentre)
Work on ECMWF's new data centre in Bologna is well on its way, although unforseen ground
conditions have caused delays. A total of 15 ITTs are needed to procure all necessary
By September 2020 the Centre will have 25 staff working in Bologna.
For users of the ECMWF facilities, the move to the new data centre will not go unnoticed:
HPC and ecgate services
The HPC and ecgate services will be merged into one new HPC. That means that everyone will
have access to only one machine in future — with one batch system and one software
stack. There will be some separation between scalar and parallel work, but how that is
going to look and what they will do with accounting is still open.
A general user migration is expected 3 – 4 months before the operational switch-over,
so towards the end of 2020.
MARS and ECFS
Hardware and tape media for MARS and ECFS will be physically moved to Bologna ahead of the
There will be extra disk space available in Reading during the move to cache data that is
frequently used, but that can obviously not hold everything.
Data on tape will be unavailable for weeks or even a few months until all tapes have
been re-loaded in the new data centre. Moreover, it might be expected that some tapes get
damaged during transport and that some data will be lost.
For 2 – 4 days there will be no MARS or ECFS service at all and the bandwidth for
data retrievals will be reduced for several months.
Actions requested from us
Delete data that is no longer needed from archive (ECFS and MARS).
Ask ECMWF to copy required essential data to disk cache before the move.
Reduce use of the archive, expect slower response during transition months.
Share your concerns with ECMWF!
Not all of the (older) software is going to be available on the new servers and
the plan is to maintain just a few recent versions of all of the software packages
there. Check ECMWF Confluence for a list of software that is going to be
Copernicus Climate Change Service
ECMWF hosts the Copernicus Climate Change Service (C3S). C3S supports society by providing
authoritative information about the past, present and future climate in Europe and the
rest of the World.
Climate Data Store (CDS) and Application Data Store (ADS)
CDS is a data store that brokers access to datasets maintained by third parties. It offers
data retieval, processing and visualisation through a web platform and CDS API.
Datasets are assigned a DOI to offer an easy way to reference.
ERA5 was one of the first data sets to become available in CDS. For public users CDS is
now the only access point for ERA5 data. Web API access has been closed. Member state
users can still access ERA5 data through MARS as well.
Surface and pressure level data are stored on disk in CDS, but model level data have to be
European Weather Cloud
ECMWF and EUMETSAT are developing the European Weather Cloud (confusingly for KNMI:
EWC) to enhance the use of their data, increase the socio-economic benefits, and exploit
opportunities and address challenges linked to open policies and market changes. The value
lies in fast access to data, which seems to be only possible realistically and sustainable
with cloud technology.
In Januari 2019 a pilot project started to build a private cloud, and gain experience with
use cases. The coming year they will continue with the implementation. New use cases are
still welcomed, they will help in define what is needed on the European Weather Cloud.
ECMWF software (ecFlow, ecCodes, Magics, Metview) is now available for Python 3. The aim
is to have ECMWF software available in all mainstream package managers.
The interactive web application for forecasters to access the latest ECMWF forecasts,
ecCharts, has undergone a major upgrade. The new version, which became default in November,
is significantly faster than before.
Github and code contributions
ECMWF has moved its source code to GitHub now. Apart from all the usual advantages, this
makes it also easier for others to contribute.
So, if you have anything to contribute, code, documentation, tests, examples, etc, please
do not hestitate to. See the link in the menu on the right.
The future of Numerical Weather Prediction
To improve forecast skill as before, the solutions from the past do not work anymore.
We can not pack more transistors on a chip, and clock-speed does not increase
anymore. Moore's law and Dennard scaling have come to an end. Furthermore, NWP codes
have become increasingly inefficient on HPC systems, because of increasing forecasting
system and code complexity.
Bigger computers, the solution of the past, will be simply unaffordable to buy and
unaffordable to power. Handing the computer codes to software people to make them faster
will not work because the codes lack in flexibility and significant scientific judgement
will still be needed. Also data sizes and hence IO is growing exponentially.
To meet these challenges, ECMWF set up the Scalability Programme several years back. With
new technologies for processors, IO and integration they now feel well on their way to