Fix precision loss issue in calculation of standard deviation. #9625
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue
Resolves #9301
Approach
The problem was due to catastrophic precision loss in the standard deviation calculation.
This PR changes the update function so that standard deviation is calculated in 64 bit float
precision. It still remains to validate that this does not use too much memory.
src/ert/analysis/_es_update.py
: Changed the standard deviation calculation to 64-bit precision.tests/ert/ui_tests/cli/test_update.py
: Added a property test that was used to find the bug, and several others: Warning of ill-conditioned matrix only showed in terminal #9590, uninformative error message when writing updated zero values of logarithmic data #9585, Uninformative error message in update: "Matrix is singular." #9581, Invalid values for DERRF distribution leads to "float division by zero" in update #9523, & Parameters for triangular distribution is not validated #9520 .tests/ert/unit_tests/analysis/test_es_update.py
: Added a new test to handle precision loss in standard deviation calculation.git rebase -i main --exec 'pytest tests/ert/unit_tests -n logical -m "not integration_test"'
)When applicable