Monthly ESMValTool community meeting - February 2024 - DATE SET!

[paigem]

Hi everyone,

The ACCESS NRI team is pleased to announce the day and time of the February 2024 ESMValTool Community Meeting:

Tuesday, February 27th @ 9am (Australian Eastern Time), which is Monday, February 26th @ 10pm UTC/11pm CET

The duration will be one hour. Join the meeting via Zoom

Calendar invitations to the meeting are sent to the mailing list, join it here.

Note: This is the first ESMValTool meeting to be organized by the Australian-based ACCESS NRI team in a Pacific-friendly timezone. The discussion will be led by @rbeucher and we will aim to follow a similar format to previous community meetings, as outlined below. We look forward to seeing you all there!

Format and agenda

Anyone with an interest in ESMValTool is welcome to join and ask questions.

We start the meeting by doing a round where everyone gets one minute to introduce themselves and tell what they will be working on in the next month or what they would like help with.

Using the input from the introduction round, we choose some topics to talk about in more detail.

If you already know that you would like to bring up a particular topic in the meeting, you can mention the GitHub issue on the topic in a comment below. That will make it easy to find for the other people in the meeting and we can already start discussing there before the meeting. Just create a new GitHub issue about your topic if there is no open issue yet.

GitHub issues are our preferred way to communicate outside the meeting and keep track of work, so any feature requests/action items/etc discussed in the meeting should ideally result in a GitHub issue.

Notes from the previous meeting are available in #142

[paigem]

@paultgriffiths Could you please make sure the date and time of the February community meeting gets sent out through the ESMValTool email list? Thanks and we look forward to running the meeting!

[paultgriffiths]

✅

[paigem]

A more detailed agenda for the upcoming meeting is included below. We look forward to seeing you many of there!

---

Meeting host - Romain Beucher
Note taker - Paige Martin

Agenda

• (5 min) Welcome & introduction to ACCESS NRI, and background for hosting this meeting in a Pacific-friendly timezone
• (15-20 min) 1-minute introductions and updates from attendees
• (remainder of time) Open discussion:
  • What does the Pacific region want out of these meetings?
  • Any other follow-up discussions from the community updates
  • Any other topics of interest to the community

[paigem]

Thanks to everyone who attended! It was a great first meeting in a Pacific-friendly time. We hope to continue hosting these in the future!

For any of the attendees, please feel free to edit my notes below if you feel I did not aptly represent what was said!

Meeting Notes

Attendee introductions

• Romain Beucher (host)

  • Team Leader, Model Evaluation and Diagnostics (MED) at ACCESS NRI
  • ACCESS NRI priorities: ACCESS model outputs, obs datasets, training (ESMValTool has a steep learning curve!), live CMORizing

• Paige Martin

  • Member of the MED team at ACCESS NRI

• Jerry Potter

  • Interest in CMORizing reanalysis datasets
  • Need somewhere to put the CMORized data - a data archive
    • NASA lost interest in CMORizing, because it’s often not NASA data
    • COPERNICUS is interested, but only interested in having reanalysis data not in the CMORizing.
  • Used CMOR itself previously, but ESMValTool is much easier - one step to download the data and one step to CMORize

• Yue Sun

  • NCI - national infrastructure for computing
  • Attended the ESMValTool workshop last (Australian) winter and wants to use it for training deep learning models

• Mike Tetley

  • At ACCESS NRI, located in the UK
  • Trying to integrate ESMValTool recipes in a live setting, i.e. shoe-horn ESMValTool to be able to do quick diagnostics on the fly while models are running

• Rheagar

  • At ACCESS NRI
  • Building a CMORizer for ACCESS data

• Felicity Chun

  • At ACCESS NRI, MED team
  • Been playing with ESMValTool for a few months, running existing recipes, then working on CMORizing observational datasets

• Claire T.

  • At CSIRO climate science center, coastal/wave modeling
  • Did some initial work on ESMValTool a couple years ago
  • Primary interest is to do assessment of downscaled, regional models
  • Keen for CMORized data on the fly to avoid duplicate versions of datasets

• Kristi

  • At Environment Canada
  • Working on model development, creating a set of standardized diagnostics that people can programmatically write recipes from
  • Trying to organize a framework for writing and updating recipes

• Rui Yang

  • At NCI
  • Wants to learn more about ESMValTool to help support the NCI community

• Ben Evans

  • NCI Deputy Director

Discussion Notes

• There will be an ESMValTool workshop in May in Germany

Live CMORization

• NCI has set up a collection for CMORized data
• But live CMORization is the best path forward so as not to store multiple versions of all the data
• Jerry wants to CMORize all reanalysis datasets
  • COPERNICUS is doing a big storage of obs data, but not currently focused on CMORizing
• there is talk about making it easier to submit CMORized ObsMIP(?), but Jerry points out that there are a lot of restrictions
  • it is not always straight-forward to keep datasets up to date, and CMORizing is yet another thing to keep up to date. Another reason to promote live CMORizing.
• When CMORizing live, it’s all through Dask - in memory but in a lazy way
• There was talk about a separate set of tools for CMORizing reanalysis datasets specifically - we aren’t sure what happened to that

Live diagnostics and the idea of making an ESMValTool API

• Mike: overview of work on live diagnostics
• Idea: someone has a climate modeling running at NCI, and wants to know that things aren’t going crazy while it’s running. Eg making basic plots while running.
• Would be nice to do some preliminary analysis, some metrics, comparisons with other model runs while on the fly.
• Integrating ESMValTool is tricky due to CMORizing. We might be able to cherry pick variables of interest and CMORize those so don’t need to do entire dataset.
• Difficulty: ESMValTool has one entrance and one exit, rather than an API. He wants to cherry pick functions of interest and run them in a more programmatic way, more like how APIs work. Is this doable at all? It is a different framework to how it currently works.
• Want to use this tool for more light-weight and fast recipe calculations.
• Research area for ACCESS NRI: are there other ways ESMValTool can be used/extended. Can we wrap some of these functions and use them in other ways? Would be great to tap the power of ESMValTool by extending its flexibility.
• Oftentimes treated as a mass conversion tool, rather than something that can be used on the spot.
• People want to use ESMValTool in Jupyter notebooks, but that implies a completely different way to do things.
• We also may not have the time at ACCESS NRI to do such an extension, but it’s something that might be very useful to a lot of people
• COSIMA: in space of Python and Jupyter notebooks
  • We are thinking of porting their recipes into ESMValTool, but difficult since heavily based on xarray rather than iris
• Also want to develop recipes around ENSO, particularly in the Pacific

How to make ESMValTool more useful without everyone having to learn it in depth?

• Kristi: working on an organizational problem. Given that most of the research group doesn’t use ESMValTool, how can they get everyone to use it without having to train the all in all the nuances.
• Trying to create a surface-level page to allow users to click a button to run relevant recipes
• Running efficiently and writing recipes, often written by community and not written to be run programmatically.
• Trying to come up with a simple platform so folks can view all the plots as new ones are generated
• Does anyone know how to use the provenance files to be helpful, in terms of using Python to extract the data back?
• If people want to make paper-friendly plots, would be nice to have a diagnostic script to be workable on the fly. Hard to do that with ESMValTool at the moment.

[rbeucher]

Thanks @paigem !

[bouweandela]

Difficulty: ESMValTool has one entrance and one exit, rather than an API. He wants to cherry pick functions of interest and run them in a more programmatic way, more like how APIs work. Is this doable at all? It is a different framework to how it currently works.

Which functions specifically would you be interested in?

• ESMValCore has an API docs and this also allows running (parts of) recipes, see here.
• ESMValTool also has an API, but it covers a very limited part of the tool docs.

[rbeucher]

I think you will have to ping @headmetal
Happy to discuss at our next meeting. I think the ESMValCore API can do quite a lot already.

[rbeucher]

@paigem could you add the GitHub handles next to the participant names?

[headmetal]

Hi @bouweandela

I'm not sure exactly yet, but will have a much clearer picture over the coming months as the project develops. Will have a deeper dive into the ESMValCore API in the meantime. Happy to have a chat/learn more about this in our next meeting :)

[bouweandela]

Integrating ESMValTool is tricky due to CMORizing. We might be able to cherry pick variables of interest and CMORize those so don’t need to do entire dataset.

This could be achieved by adding support for the model in its native format, as has already been done for several other climate models. See https://docs.esmvaltool.org/projects/ESMValCore/en/latest/develop/fixing_data.html#add-support-for-new-native-datasets and https://doi.org/10.5194/gmd-16-315-2023

[bouweandela]

People want to use ESMValTool in Jupyter notebooks, but that implies a completely different way to do things.
We also may not have the time at ACCESS NRI to do such an extension, but it’s something that might be very useful to a lot of people

This is already possible: see e.g. https://docs.esmvaltool.org/projects/ESMValCore/en/latest/api/esmvalcore.experimental.recipe.html#running-a-recipe and https://github.com/ESMValGroup/EGU22-short-course/blob/main/notebooks/Introduction_to_ESMValTool_with_output.ipynb

[bouweandela]

Trying to come up with a simple platform so folks can view all the plots as new ones are generated

I know that something like this was developed at BSC for monitoring EC-Earth runs. Maybe @sloosvel could tell you more about it?

[sloosvel]

We use an R shiny app to showcase the plots.

[bouweandela]

Does anyone know how to use the provenance files to be helpful, in terms of using Python to extract the data back?

There is an example script available here: https://docs.esmvaltool.org/en/latest/utils.html#extracting-a-list-of-input-files-from-the-provenance

[bouweandela]

If people want to make paper-friendly plots, would be nice to have a diagnostic script to be workable on the fly. Hard to do that with ESMValTool at the moment.

Most recipes generate NetCDF output of the plotted data. That can be conveniently loaded in a Jupyter notebook using the Python API. That might be a good starting point for tweaking plots. Also note the resume_from configuration option, which allows you to re-use preprocessed data.