You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current storage for everest optimization results should be ported away from the seba-sqlite to something as close to the ERT storage as possible. The results generated by Everest can be seen as:
(1) forward model runs w/ inputs (Everest controls / ERT parameters) and outputs (Everest objectives / ERT GEN_DATA responses)
(2) optimization results computed by ROPT, where the forward model responses are used as inputs. Some of these are not explicitly existing in ERT storage
All things computed by ROPT cannot be retrieved from ERT storage, this includes constraint violations, and basically any dataframe starting with batch_, meaning various aggregations of results per-realization. Recomputation of these should be very fast if we resolve #9764
Issue
Optimization results are currently stored in an SQLite database, which is used by Everest via a snapshot object to report and store results. This approach is not directly supported anymore by the ropt which is used by Everest to run the optimization. Everest uses the seba-sqlite package which provides the original code from Seba to support the SQLite database.
This code should be replaced in Everest for the following reasons:
Reliability issues with the SQLite database, in particular on NFS files systems.
Performance is poor.
The code has not kept up with new features that have added in recent years, impeding proper monitoring and storage of results. For instance, when auto-normalization is used, the console output does not handle scaled results properly due to missing information.
The seba-sqlite code is deprecated, TNO will not continue to support it.
Instead of using the database, Everest should receive optimization results directly from ropt via the provided callback mechanism, and take appropriate actions to report and store results.
Definition of done
When the SQLite code has been removed from Everest and all current reporting and storage functionality is available.
The current storage for everest optimization results should be ported away from the seba-sqlite to something as close to the ERT storage as possible. The results generated by Everest can be seen as:
(1) forward model runs w/ inputs (Everest controls / ERT parameters) and outputs (Everest objectives / ERT GEN_DATA responses)
(2) optimization results computed by ROPT, where the forward model responses are used as inputs. Some of these are not explicitly existing in ERT storage
Course of action:
Considerations/notes:
batch_
, meaning various aggregations of results per-realization. Recomputation of these should be very fast if we resolve #9764Issue
Optimization results are currently stored in an SQLite database, which is used by Everest via a snapshot object to report and store results. This approach is not directly supported anymore by the
ropt
which is used by Everest to run the optimization. Everest uses theseba-sqlite
package which provides the original code from Seba to support the SQLite database.This code should be replaced in Everest for the following reasons:
seba-sqlite
code is deprecated, TNO will not continue to support it.Instead of using the database, Everest should receive optimization results directly from
ropt
via the provided callback mechanism, and take appropriate actions to report and store results.Definition of done
When the SQLite code has been removed from Everest and all current reporting and storage functionality is available.
Blocked by equinor/everest#118The text was updated successfully, but these errors were encountered: