Improved QA4SM metrics calculators #219
Conversation
s-scherrer
commented
Mar 12, 2021
•
s-scherrer
commented
- new matcher that makes sure all dataframe combinations only have common timestamps
- new implementation of QA4SM metrics calculators that take only 2 or 3 dataframes
…nto metrics_calculators
This adds a test to check whether the new matching in BasicTemporalMatching works as the old one in case there are no duplicates in the reference frame. It replaced the matcher in the other tests by a dummy that still uses the old matching implementation. The following commit will remove the dummy and set the new expected values.
|
The previous commit compared the old and the new temporal matching implementation when dropping duplicates before matching (see also #218). The CI failed only due to coverage, the tests are running. The next commit will remove the comparison test and update the target values in the tests affected by removing duplicates in the reference. |
One test still fails due to difference in scaling before calculating ubRMSD.
This adds a test to check whether the new matching in BasicTemporalMatching works as the old one in case there are no duplicates in the reference frame. It replaced the matcher in the other tests by a dummy that still uses the old matching implementation. The following commit will remove the dummy and set the new expected values.
One test still fails due to difference in scaling before calculating ubRMSD.
s-scherrer
force-pushed
the
metrics_calculators
branch
from
dee80d6
to
123061c
Compare
- bug in validation.get_data_for_result_tuple - no segfault when running tests anymore
…nto metrics_calculators
There was a problem hiding this comment.
@s-scherrer Looks good to me, is this done from your side? Im still a bit worried about #220, but it seems that the current solution is ok for now.
|
I changed the tests a bit to also have a case where the reference name and the other name have a different alphabetical order. The results don't change much, it's just a bit hacky to figure out which one was the reference. For QA4SM this shouldn't be a problem, because the dataset names are numbered, e.g. In other cases it's only a usability issue - it's not as easy to tell which one was the reference, but the results are still okay. I would propose to solve #220 in a separate PR. |
