Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test against original Matlab code #4

Open
greglucas opened this issue Nov 19, 2019 · 5 comments
Open

Test against original Matlab code #4

greglucas opened this issue Nov 19, 2019 · 5 comments
Milestone

Comments

@greglucas
Copy link
Owner

@bmurphy-usgs, @erigler-usgs
Starting a new issue spawned from the discussion in #1.

We need to validate this code against the original Matlab code for some explicit geometries. I'm not sure if we validate against the geomag-imp code base, or actually go back to the original Matlab for this. I'll leave that up to you all. But, it would be great to get some actual values for specified cases and directly test against those in the test suite.

#2 laid the foundation for the test-suite with some initial basic tests for the code. It should be easy to expand and put the new test cases in there with references to where original values are taken from when needed in comments.

@greglucas
Copy link
Owner Author

I was just going through this and wondering if anything has transpired with the validation, or if I needed to do anything for it on my end?

@bmurphy-usgs
Copy link
Contributor

Sorry, haven't worked on this at all, got distracted by other things... If you can make some progress on this that would be great; I can still try to help, although not sure when I'll have a chance to get to this...

@greglucas
Copy link
Owner Author

I don't have any original Matlab code, so I am unable to do the direct comparison... I did incorporate some basic tests for sanity though when you found the previous error. So, everything is passing the tests now, but that could still benefit from a little more rigor.

@greglucas greglucas mentioned this issue Mar 11, 2020
7 tasks
@greglucas greglucas added this to the v0.1 milestone Mar 11, 2020
@erigler-usgs
Copy link
Collaborator

Hi guys, sorry I'm only chiming in now. I'm happy to help with this, although I probably can't do much for at least a week or two. If @bmurphy-usgs wanted to take a stab, he should be able to find the original Matlab code here: https://code.chs.usgs.gov/erigler/geomag-conus-secs. @greglucas , if you want to take a stab, let me know, and I'll zip up the files and email to you.

To be clear, we cannot make this Matlab code accessible to the public, given that it does not belong to us. Therefore, I'm not sure how best to incorporate it into a validation suite. We might document a comparison of the original Geomag-IMP and the ML code better than I did, then set aside the original Geomag-IMP code for future comparisons. Or maybe we just use the original ML code to generate a suite of results files against which we validate moving forward.

I'm not sure which approach is best.

@greglucas
Copy link
Owner Author

No problem at all, I haven't exactly been keeping up on this over the past several months either. I'm actually just beginning to pick this up again to try and use it for some other work which is why I brought this up again.

My real desire is not necessarily to make it into a full-blown validation to be run every time. I'm really more interested in a sanity check because Ben already found one error :) This could involve a set of points that we run through each routine to test. A mix of SECs below/above, near/far from observations. We could easily drop the expected output arrays into the test suite too without needing to run the Matlab code every time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants