Enable MEMFS (memory based access to definition/sample files)

[kynan]

As of 0.16.0 ecCodes supports memory based access to definition/sample files:

From this version the definition and sample files used by ecCodes engine to decode all kind of messages can be built in to the library (embedded) providing a performance improvement by significantly reducing access to the filesystem.

This feature is called MEMFS (MEMory FileSystem) and can be enabled at configuration time (when you invoke cmake) by:

cmake /path/to/src -DENABLE_MEMFS=ON

This will take longer to compile and the resulting library will be larger but whenever a definition file (*.def or .table) or a sample file (.tmpl) is accessed, its contents will be fetched from memory and no access to the file system will be made.

If you invoke the codes_info tool with this configuration, the paths reported will contain the prefix "/MEMFS/".

Please note that once ecCodes is built in this mode, any subsequent changes to the installed definition/sample files will NOT be picked up

How do does everyone feel about enabling this? @kmuehlbauer @pelson @ocefpaf

[kmuehlbauer]

Please note that once ecCodes is built in this mode, any subsequent changes to the installed definition/sample files will NOT be picked up

How do does everyone feel about enabling this? @kmuehlbauer @pelson @ocefpaf

I'm very much open, if this improves speed significantly. But there is a drawback. Users won't be able to add/change definitions and tables in their installations. This could turn the package useless in some environments.

[kynan]

True, should this be a dedicated feedstock then?

[kmuehlbauer]

Maybe we can use a dedicated branch? Maybe @ocefpaf has an idea.

[ocefpaf]

Maybe @ocefpaf has an idea.

Branches are probably better, but we would need to disambiguate the name with a prefix or something similar.

[kynan]

variant/memfs? @ocefpaf

[kynan]

@kmuehlbauer @ocefpaf I actually want to try this to work around a runtime failure I'm currently trying to debug rather than for performance. If we can agree on a branch naming convention I'll give it a go.

cc @shahramn

[kmuehlbauer]

@kynan

You could try with a PR first, to see if this will build correctly. Then @ocefpaf might help us to get the package prefixing correct. Then the PR can be diverted to the new branch.

[ocefpaf]

You could try with a PR first, to see if this will build correctly. Then @ocefpaf might help us to get the package prefixing correct. Then the PR can be diverted to the new branch.

Sounds like a plan.

[kynan]

Done! #28

[kmuehlbauer]

With #127 it is available on windows. Users can add local tables/definitions. Question: Is it only relevant for windows or does also linux/mac builds profit, @shahramn?