Infrastructure for memory usage tests

[emolter]

This PR adds a context manager to more easily check the peak memory usage of a code block and raise an error if a memory limit is exceeded. A unit test of the memory usage in the median calculation in outlier detection provides an example use-case.

Relates to JP-3775 but only fixes one small part of that ticket.

Tasks

• update or add relevant tests
• update relevant docstrings and / or docs/ page
• Does this PR change any API used downstream? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • run regression tests with this branch installed ("git+https://github.com/<fork>/stcal@<branch>")
    • jwst regression test
    • romancal regression test

news fragment change types...

• changes/<PR#>.apichange.rst: change to public API
• changes/<PR#>.bugfix.rst: fixes an issue
• changes/<PR#>.general.rst: infrastructure or miscellaneous change

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.30%. Comparing base (60bd3b8) to head (12c5886).
Report is 4 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #299      +/-   ##
==========================================
+ Coverage   86.21%   86.30%   +0.09%     
==========================================
  Files          47       49       +2     
  Lines        8812     8870      +58     
==========================================
+ Hits         7597     7655      +58     
  Misses       1215     1215              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[braingram]

Since the parent ticket JP-3775 is for building out infrastructure for tests like this, what do you think about building out that infrastructure and using it here? Something like:

with memory_threshold(expected_mem):
    computer = MedianComputer...

I can understand that the infrastructure is beyond the scope of this PR but I think an argument can be made that building the infrastructure first (before this PR) makes sense since:

• this is a useful test for the infrastructure
• if we merge this PR first a follow-up PR will modify this test again when we have the new infrastructure

[emolter]

I can understand that the infrastructure is beyond the scope of this PR but I think an argument can be made that building the infrastructure first (before this PR) makes sense

I completely agree. I'll work on making a context manager that is called like you show

[emolter]

@braingram I added the context manager and a fixture that calls it. I wasn't sure exactly how to structure this, because based on what I read it's not great practice to import from conftest.py directly. Let me know if the proposed helper.py and conftest.py look right to you.

Also, do you think it's necessary/useful to add infrastructure for testing the fixture itself like in this StackOverflow answer?

[braingram]

@braingram I added the context manager and a fixture that calls it. I wasn't sure exactly how to structure this, because based on what I read it's not great practice to import from conftest.py directly. Let me know if the proposed helper.py and conftest.py look right to you.

Also, do you think it's necessary/useful to add infrastructure for testing the fixture itself like in this StackOverflow answer?

Since the fixture isn't needed for this test I'd say we leave it out of this PR. The class you added in tests.helpers seems to cover all the needs for the added test. I'll leave a few other comments inline.

[braingram]

Thanks! LGTM.

[emolter]

status update: @penaguerrero is using this to write a test for the jump step, and in so doing is evaluating this PR. So this is awaiting her review

[penaguerrero]

I tried the code in writing a test for jwst, and it works as intended. Other than suggesting to allow to present the result in other units I have no further comments. LGTM

[emolter]

@penaguerrero can you test the units option I added and see if it covers your use-case?

[penaguerrero]

I just tested again with the changes. It works as expected. LGTM!