Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fmuobs] Add naive support for GENERAL_OBSERVATIONs in yaml #352

Closed
wants to merge 2 commits into from

Conversation

berland
Copy link
Collaborator

@berland berland commented Apr 28, 2021

Implements the most obvious/naive way of converting ert observations to the yaml format, meaning that

GENERAL_OBSERVATION GEN_OBS1{
   DATA     = SOME_FIELD;
   RESTART  = 20;
   OBS_FILE = some_file.txt;
};

GENERAL_OBSERVATION GEN_OBS2{
   DATA       = SOME_FIELD;
   INDEX_LIST = 0,3,9;
   RESTART    = 20;
   OBS_FILE   = some_file.txt;
};

is translated (in both directions) to:

general:
 - label: GEN_OBS1
   data: SOME_FIELD
   restart: 20.0
   obs_file: some_file.txt
 - label: GEN_OBS2
   data: SOME_FIELD
   index_list: "0,3,9"
   restart: 20.0
   obs_file: some_file.txt

@codecov-commenter
Copy link

Codecov Report

Merging #352 (1654cff) into master (9501b7c) will increase coverage by 0.03%.
The diff coverage is 100.00%.

❗ Current head 1654cff differs from pull request most recent head ab77c75. Consider uploading reports for the commit ab77c75 to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##           master     #352      +/-   ##
==========================================
+ Coverage   87.27%   87.30%   +0.03%     
==========================================
  Files          41       41              
  Lines        5657     5673      +16     
==========================================
+ Hits         4937     4953      +16     
  Misses        720      720              
Impacted Files Coverage Δ
src/subscript/fmuobs/parsers.py 95.63% <100.00%> (+0.17%) ⬆️
src/subscript/fmuobs/writers.py 96.21% <100.00%> (+0.14%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9501b7c...ab77c75. Read the comment docs.

@berland berland linked an issue Apr 28, 2021 that may be closed by this pull request
Also ensure that integer values are printed as integes in ert-obs
output
Copy link
Contributor

@eivindsm eivindsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would expect the parser to read the obsfile, fetch the observation values and stds corrending to indexlist, and internalize these as two lists. Leaving behind a reference to a file does not serve much purpose. The format for these files is well defined, so it seems realistic to fetch the numbers. Sorry, if I missed something obvious :)

@berland
Copy link
Collaborator Author

berland commented Apr 28, 2021

Good point, parsing that text file is the original request. I am not yet sure if such parsing can be done in a reversible manner and it at least requires more thinking for how to convert back to ERT observation format. Should it create its own OBS_FILE on disk if it is converting from yaml back to ert-obs?

The current implementation is the obvious implementation by naivety. Parsing the OBS_FILE should be an extension of this.

Copy link
Contributor

@eivindsm eivindsm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no objections to the code, and can approve. However, I am unsure if I like the proposed yml- data structure;

  • The restart entry in ERT does really refer to restart numbering, but a required identifier that in may refer to restart (I think it did in good old EnKF days). Should we consider a generic name for this to avoid confusion, or will this only add on to the mess?
  • Should the index list be a string and not a list?
  • Will we have to change the structure if functionality to parse numbers are added?
    Another thought, is it feasible to output this into a format that can be used with eg WebViz or Resinsight?

@berland berland changed the title Add support GENERAL_OBSERVATIONs in fmuobs yaml Add naive support GENERAL_OBSERVATIONs in fmuobs yaml Apr 29, 2021
@berland berland marked this pull request as draft April 29, 2021 16:19
@berland
Copy link
Collaborator Author

berland commented Apr 29, 2021

Converted to draft PR. It is not important to merge this as it would be of no use currently. We should think through what we want from this, and what is possible to achieve from a format converter.

@berland berland changed the title Add naive support GENERAL_OBSERVATIONs in fmuobs yaml [fmuobs] Add naive support GENERAL_OBSERVATIONs in yaml Jun 2, 2021
@berland berland changed the title [fmuobs] Add naive support GENERAL_OBSERVATIONs in yaml [fmuobs] Add naive support for GENERAL_OBSERVATIONs in yaml Jun 3, 2021
@rnyb rnyb closed this Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[fmuobs] Write GENERAL_OBSERVATION to yaml
4 participants