diff --git a/ocean/AMOC/watermassanalysis/examples/basic_surface_wmt/README.md b/ocean/AMOC/watermassanalysis/examples/basic_surface_wmt/README.md new file mode 100644 index 00000000..c7acf570 --- /dev/null +++ b/ocean/AMOC/watermassanalysis/examples/basic_surface_wmt/README.md @@ -0,0 +1,64 @@ +# Basic surface water mass transformation for MPAS runs + +The core surface water mass transformation (WMT) calculations are located in [`watermasstools.py`](https://github.com/MPAS-Dev/MPAS-QuickViz/blob/master/ocean/AMOC/watermassanalysis/modules/watermasstools.py). These calculations contain one additional dependency beyond the `e3sm_unified` environment + + - [`fastjmd95`](https://github.com/xgcm/fastjmd95) -- A numba-accelerated package for the Jackett and McDougall (1995) equation of state + +### Serial usage + +The command line executable module `basic_surface_wmt.py` is a postprocessing wrapper around the core WMT functions. This module is set up to accomplish two tasks + + 1. Build a basic coordinates file `python basic_surface_wmt.py -c [MESHNAME]` + 2. Process a single monthly results file `python basic_surface_wmt.py -f [FILENUMBER] [MESHNAME]` + +Here `MESHNAME` is either `LR` (`EC30to60E2r2`) or `HR` (`oRRS18to6v3`). Both options use the CORE-II E3SMv2 G-cases with the `20210421_sim7` tag. Different runs can be specified in `parameters.yaml` (small changes to `python basic_surface_wmt.py` will probably also be required). + +### Parallel usage + +The workflow is set up to process single monthly files such that each serial task can be distributed across multiple cpus on a single node. I use GNU Parallel for this. The general workflow is as follows + +``` +#!/bin/bash +#SBATCH --job-name=JOB_NAME +#SBATCH --qos=regular +#SBATCH --nodes=1 +#SBATCH --ntasks-per-node=64 +#SBATCH --cpus-per-task=1 +#SBATCH --constraint=cpu +#SBATCH --exclusive +#SBATCH --output=JOB_NAME.o%j +#SBATCH --time=1:00:00 +#SBATCH --account=ACCOUNT + +source $HOME/.bashrc +module load parallel +conda activate ENV_NAME + +meshName=LR +savePath="/path/to/${meshName}" +mkdir -p "${savePath}/monthly_files" +mkdir -p "${savePath}/concatenated" + +# Calculate coords file (-c loads the coords file) +python basic_surface_wmt.py -p "${savePath}" -c "${meshName}" + +# We use GNU Parallel to run the serial monthly processing across all cpus on our node +PARALLEL_OPTS="-N 1 --delay .2 -j $SLURM_NTASKS --joblog parallel-${SLURM_JOBID}.log" + +# Here GNU Parallel distributes the individual SRUN tasks, so for SRUN +SRUN_OPTS="-N 1 -n 1 --exclusive" + +# Process the monthly files, $(seq 0 119) does the first 10 years +parallel $PARALLEL_OPTS srun $SRUN_OPTS \ + python basic_surface_wmt.py -p "${savePath}" -f {} "${meshName}" ::: $(seq 0 119) + +# Concatenate monthly files +ncrcat -h ${savePath}/monthly_files/${meshName}_WMT1D* \ + ${savePath}/concatenated/${meshName}_WMT1D_years1-10.nc +ncrcat -h ${savePath}/monthly_files/${meshName}_WMT2D* \ + ${savePath}/concatenated/${meshName}_WMT2D_years1-10.nc +``` + +### Visualization + +A basic visualization example can be found in `basic_surface_wmt.ipynb`. \ No newline at end of file diff --git a/ocean/AMOC/watermassanalysis/examples/basic_surface_wmt/basic_surface_wmt.ipynb b/ocean/AMOC/watermassanalysis/examples/basic_surface_wmt/basic_surface_wmt.ipynb new file mode 100644 index 00000000..532e4f34 --- /dev/null +++ b/ocean/AMOC/watermassanalysis/examples/basic_surface_wmt/basic_surface_wmt.ipynb @@ -0,0 +1,1341 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "e782b890-153d-4122-86a6-ac4929b7544b", + "metadata": {}, + "source": [ + "# Basic surface water mass transformation for MPAS runs\n", + "\n", + "Basic surface water mass visualization of the calculations based on `basic_surface_wmt.py`. For generating the results, see README." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "f54b258c-19da-4b80-abba-11bc6c5cf98c", + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import xarray as xr\n", + "from matplotlib import pyplot as plt\n", + "\n", + "plt.rcParams.update({'font.size': 12, 'axes.titlesize': 'medium'})" + ] + }, + { + "cell_type": "markdown", + "id": "aa57ef90-3c97-4271-a85f-76bcc2912a6b", + "metadata": {}, + "source": [ + "***\n", + "\n", + "### Load postprocessed results for CORE-II LR and HR G-cases\n", + "\n", + "Generated with `basic_surface_wmt.py` and `ncrcat` in GNU Parallel. See README." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "ba04c82d-868f-4247-ad0a-a1fdd8fb72c4", + "metadata": {}, + "outputs": [], + "source": [ + "# Load postprocessed WMT files\n", + "savepath = '/pscratch/sd/b/bmoorema/wmt/basic_example'\n", + "wmt = {}\n", + "for meshName in ['LR', 'HR']:\n", + " wmt[meshName] = {}\n", + " for ctgy in ['1D', '2D']:\n", + " wmt[meshName][ctgy] = xr.open_dataset(f'{savepath}/{meshName}_WMT{ctgy}_years1-10.nc')" + ] + }, + { + "cell_type": "markdown", + "id": "532ba700-858b-481b-84a4-0a7e6dd8b04b", + "metadata": {}, + "source": [ + "***\n", + "\n", + "### Output data structures\n", + "\n", + "Here are previews of the `xarray.Dataset` structures for the LR results for 1D and 2D." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "f015f377-45a6-40f3-8c14-4afd0396e704", + "metadata": { + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "
<xarray.Dataset> Size: 5MB\n", + "Dimensions: (time: 120, regionNames: 11, densityBins: 171)\n", + "Coordinates:\n", + " * time (time) datetime64[ns] 960B 1948-01-01 ... 195...\n", + " * regionNames (regionNames) <U23 1kB 'Greenland Sea' ... 'R...\n", + " * densityBins (densityBins) float64 1kB 26.5 26.51 ... 28.2\n", + "Data variables:\n", + " densityHeatTransformation (time, regionNames, densityBins) float64 2MB ...\n", + " densitySaltTransformation (time, regionNames, densityBins) float64 2MB ...\n", + " densityTotalTransformation (time, regionNames, densityBins) float64 2MB ...
<xarray.Dataset> Size: 16GB\n", + "Dimensions: (time: 120, lat: 170, lon: 280, densityBins: 171)\n", + "Coordinates:\n", + " * time (time) datetime64[ns] 960B 1948-01-01 ... 1957...\n", + " * lat (lat) float64 1kB 0.25 0.75 1.25 ... 84.25 84.75\n", + " * lon (lon) float64 2kB -99.75 -99.25 ... 39.25 39.75\n", + " * densityBins (densityBins) float64 1kB 26.5 26.51 ... 28.2\n", + "Data variables:\n", + " densityHeatTransformation (time, densityBins, lat, lon) float64 8GB ...\n", + " densitySaltTransformation (time, densityBins, lat, lon) float64 8GB ..." + ], + "text/plain": [ + "