Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Metadata support when using aggregated time series data #402

Open
asnyv opened this issue Aug 12, 2020 · 1 comment
Open

Metadata support when using aggregated time series data #402

asnyv opened this issue Aug 12, 2020 · 1 comment

Comments

@asnyv
Copy link
Collaborator

asnyv commented Aug 12, 2020

Simulation time series metadata is used for several functionalities, including the coming average rates and interval cumulatives.

#323 added support to utilize fmu-ensemble / libecl for metadata of simulated time series data (Eclipse format), but did not handle the scenario of aggregated/csv exported data.

Necessary to make the assumption that all ensembles and realizations share the same metadata (e.g. that there are not inconsistencies in units and etc, was also done in #323, and it is something we do when we allow co-plotting of ensembles anyways.)

fmu-ensemble stores the metadata as a dict when running ScratchEnsemble.get_smry_meta() (unfortunately no support for EnsembleSet as far as I know), which would be the natural way to make this for a user. In Webviz we currently iterate over these dicts (one for each ensemble) and store the union of them in a multiindex pandas.DataFrame, that is storable as a parquet file (for portable), but is not ideal for csv as the multiindex behavior is lost.

Alternatives:

  • To have a standard where fmu-ensemble's output dicts are stored as json-files, and where webviz supports reading and converting the json input to the current multiindex DataFrame. In this case, it could be preferable that there exists a standard way of converting the fmu-ensemble (even though saving a dict as a json file is straight forward).

  • Again using json's from fmu-ensemble, but also switch out the current multiindex DataFrame in webviz and use dicts stored as BytesIO/json for portable (probably what should have been done in the first place). Will risk breaking current portable apps and require rebuild unless we handle it with care... Not really necessary, just cleaning it up a bit. Might make users "angry", so probably avoid this.

  • Add an export to some standard csv format to fmu-ensemble

In all cases I think it would be useful to have a similar get_smry_meta() functionality for EnsembleSet's in fmu-ensemble, so that the behavior is well defined outside upstream of Webviz (e.g. a user might have different wells/wellnames in different ensembles, but we would like a unified metadata set).

@jorgenherje
Copy link
Collaborator

jorgenherje commented Feb 11, 2022

@asnyv and @anders-kiaer

Is this covered by vector_metadata() in EnsembleSummaryProvider, or is there a need for special handling of metadata for a ProviderSet - equivalent to EnsembleSet mentioned above?

Example of a ProviderSet in SimulationTimeSeries-plugin: https://github.com/equinor/webviz-subsurface/blob/master/webviz_subsurface/plugins/_simulation_time_series/types/provider_set.py

ProviderSet could be of interest for more developers and can be revisited and moved to a common utils-folder if needed.

VincentNevermore pushed a commit to VincentNevermore/webviz-subsurface that referenced this issue Jul 19, 2022
@anders-kiaer anders-kiaer moved this to Backlog 📝 in Webviz Jan 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Backlog 📝
Development

No branches or pull requests

2 participants