Supporting time-dependent geom (and related) in MAPL3

[tclune]

In MAPL3 we intend to allow the generic layer to introduce couplers to compensate for certain types of mismatches between source and destination fields. At a minimum we expect to support couplers to adapt:

• Horizontal geometry
• Vertical grid
• typekind
• units
• frequency (time average/accumulation)

In most cases the needed coupler will itself be time invariant. E.g., horizontal geometry is expected to usually be constant, but with important exceptions for instrument samplers and ExtData collections that have windows of time validity. More generally most of the above could change during the course of a run - almost exclusively due to the flexibility of the ExtData layer. Note: it probably does not make sense to allow the frequency of a component to change over time, so we may not address that.

For this to work, there are a few things that must happen:

• The generic layer must interpose a coupler even if the initial state of the model does not require a coupler. E.g., if an ExtData collection starts on the native model grid but later becomes latlon, it is "too late" to introduce a coupler into the hierarchical structure of the model. (Not an insolvable problem but would pose a major design challenge compared to the assumptions baked into the current structural analysis.)
• Couplers must be able to detect when they need to update themselves due to a change in the actual aspect of either their input or output quantities. At this level communication is solely by ESMF state items, which means the detection has to be via accessors on those items. E.g., a regrid coupler could extract the geom from input and output fields and check to see if it is different than previously and if so get a new regridder from the regridder registry. In some cases, the information could simply be a string in the ESMF_Info object.

Here is what I think can be done for each case:

• geom: use geom info object to obtain geom id and compare to saved ID
  • comparing the geom directly is possible but very expensive: have to check every latitude, longitude, ... on all processes.
• vertical grid: use info object on the field. (No ESMF representation)
• units: use info object on the field
• typekind: direct introspection of the field

A related challenge is that a change in the original export may need to propagate "downstream" through other state items. E.g., consider an ExtData item that needs a coupler both for geom and units and further that halfway through the run the units in the source file change. Because MAPL3 adapts units after (downstream) the geom, the intermediate field (the one between the regrid coupler and the unit converter) must update its units when the original export updates its units. This might get expensive to constantly copy info data just in case something has changed. Needs further thought.

[tclune]

This might get expensive to constantly copy info data just in case something has changed.

And of course copying data from an info object is the lightweight use case. Suppose ExtData updates a field to be real4 rather than real8? Then intermediate fields must actually be destroyed and recreated.

Side note: because MAPL3 shares fields between exports and imports (except when a coupler intervenes) we cannot actually create/destroy fields. Instead we "hollow-out" the field. The public interfaces for ESMF do not support this but essentially one can use private access to the Field components to leave the object in place, but with the data storage deallocated. and then (re) complete the field. And this in turn presupposes that we have the necessary information to do this correctly. E.g., consider ungridded dimensions.

[tclune]

Yet another issue arises when an import changes an aspect that had previously been the same as that of some other likewise connected import. Initially the two imports can share the same export extension, but if one changes that sharing is no longer tenable.

Conclusion: If an import spec is time dependent all extensions in its connection chain must not be shared.

Possibly this is a non-issue in practice as, e.g., when a sampler changes geometry we do not expect any other components to be on the same geometry.

[tclune]

I am trying to think of a clean mechanism to ensure that 2 imports do not share a time-dependent export extension. Basically we always want extensions to start from the actual source spec and work out. This means that the find_closest_extension() should always return the primary (unextended) item in an extension family.

Currently find_closest_extension() applies a series of filters (match() method on Adapter's) and terminates when the number goes to 0. So maybe this is as simple as saying that the match() is always false for time dependent export specs.

[tclune]

While prototyping the logic to handle time-dependence I ran into a new difficulty. A vertical regridder depends not only on its formal input field from the import state, but also on the two embedded pressure fields. Those may change the values of the vertical coordinates without changing the structure of the actual import/export fields.

Some related complications:

• We expect model pressure levels to change every timestep. I.e., this is not a rare corner case.
• We expect to be able to share regridder weights across a large number of couplers; common use case in history and maybe extdat.
• We want the "first" field for a given set of target pressure levels in each timestep to trigger the recalculation. At the same time we want to reuse the weights for other fields that do share the same target.

If we set some flag (info) on the input field saying that the weights must be recalculated, when do we clear the flag? If we clear on the 1st then variant regrid weights are not computed. But if we don't the related weights are recalculated.

[tclune]

A possible approach is that each VGrid object will have a counter. VGrid couplers will compare the counter to a counter in the weight tables. When they are different, the weights are recomputed and the counter in the table is updated. Subsequent regrid couplers in the same family will see agreement and do nothing.

The owner of the grid must update the counter when the vertical changes. Would be nice if this was fully automatic, but probably not realistic. One could assume that if a component is a provider (and vgrid is model grid) then the counter should be updated after component runs.