Climate Science Data Enhanced Virtual Laboratory

Bringing together international reference datasets and observations in a supercomputing environment capable of the intensive data analysis and computationally-demanding simulations that climate research requires.

A major focus of the Australian climate research community currently is their contribution to the World Climate Research Programme (WCRP) Coupled Model Intercomparison Project phase 6 (CMIP6). This work underpins research into historical climate variability and future projections which can assist Australian government, business, agriculture and industry to manage climate risks and opportunities related to climate variability, change and extremes. Approximately 20 petabytes (PB) of CMIP6 data are expected globally, the largest collection of climate data ever produced, of which a substantial portion will be made available and analysed at the National Computational Infrastructure (NCI) Australia. To prepare for this, the Climate DeVL project has established the centrally-managed systems and procedures for organising the massive CMIP6 data archive as well as the data services, data analysis tools and user support that make it accessible to a broad range of users in the Australian region.

Start date 2 July 2018

Expected completion date 16 January 2020

Investment by ARDC $625,000

Co-investment partners

Bureau of Meteorology (BoM)

ARC Centre of Excellence for Climate Extremes (CLEX)

CSIRO

National Environmental Science Programme Earth Systems and Climate Change Hub (ESCC)

National Computational Infrastructure (NCI)

Lead node

1 Production release of new ESGF (Earth System Grid Federation) and CMIP data replication systems

The release includes local improvements through integration with NCI infrastructure and additional services to meet community requirements. Further aspects involve the establishment of the local procedures for publishing Australian datasets, replicating international data, and republishing the replicated data for user access via NCI data services, CMIP5 and CMIP6 variable replication for priority datasets, and coordination with international ESGF community for management of the local replicated data collections.

2 Improved data FAIRness and local search capabilities

Integration of the CleF API with the NCI Metadata Attribute Service (MAS) provides access to the detailed metadata information within the millions of files that constitute the CMIP data collection and simplifies the process for additional data requests. Availablility of more detailed NCI data catalogue information for CMIP datasets, and the NCI ESGF node includes overseas EGSF data indexes for faster local search.

3 Additional data analysis tools and environments

This includes the installation on NCI of prototype Jupyter-hub service for Climate and evaluate resources to run this as an ongoing managed service. Scoping of requirements to establish a suite of R tools and workflows for climate data analysis was undertaken. When they are released, there will be local installations and evaluation of the publicly available versions of the internationally maintained tools promoted by the ESGF (ESMValTool and the PCMDI Metrics Package). A review was undertaken of the current data analysis pipelines to determine scope of work to update for new CMIP datasets.

4 User support, training and outreach

Extensive review and update of the CMIP Community website incorporating new features and tools such as the data search table where users may view and search through the available CMIP data at NCI, as well as discover what data is planned to be downloaded. Preparation of new material for online user support and targeted training events completed.

Core features

Data repository

An official and centrally-managed CMIP data repository providing consistent data management practices to ensure international standards are rigorously maintained and data is replicated locally in line with community priorities and as it becomes available internationally.

Data-intensive research platform

Data is available for use within NCI’s integrated data storage, supercomputing and data services environment which provides the only platform in Australia capable of supporting the intensive data analysis and simulation that CMIP6 requires.

Advanced search

Improved FAIRness of data for a broad range of users including advanced search capabilities and tools which streamline access and use of data.

User support

Coordination and development of community website and training materials to improve user experience and utilisation of the range of tools and resources.

Who is this project for?

Climate scientists

What does this project enable?

Australian climate researchers are able to undertake cutting-edge science climate research which underpins all climate projects and policy for Australia. The Climate Science DeVL project provides the data management, services and tools to permit a diverse range of innovative research to be undertaken with CMIP5/6 data.