JP-3708, JP-3771, JP-3791

[kmacdonald-stsci]

Resolves JP-3708
Resolves JP-3771
Resolves JP-3791

JP-3708: Add the CRMAG parameter to the optional results product in the C-extension. This wasn't implemented with the C-extension, nor was it properly handled for multiprocessing.

JP-3771: Corrected bugs exposed by adding multiprocessing regression tests to the JWST regression testing suite. The primary bug found was incorrect dimension sizes for internal arrays used for the optional results product. The internal arrays should have dimensions of (nints, nrows, ncols), but they had dimensions (ngroups, nrows, ncols). These incorrect dimensions caused memory corruption causing processes to freeze or segfault.

JP-3791: Corrected the CHARGELOSS recomputation of the read noise variance. For multiprocessing the DQ flags were not being properly sliced and passed to the C-extension for recomputation.

Tasks

• update or add relevant tests
• update relevant docstrings and / or docs/ page
• Does this PR change any API used downstream? (if not, label with no-changelog-entry-needed)
  • write news fragment(s) in changes/: echo "changed something" > changes/<PR#>.<changetype>.rst (see below for change types)
  • run regression tests with this branch installed ("git+https://github.com/<fork>/stcal@<branch>")
    • jwst regression test
    • romancal regression test

news fragment change types...

• changes/<PR#>.apichange.rst: change to public API
• changes/<PR#>.bugfix.rst: fixes an issue
• changes/<PR#>.general.rst: infrastructure or miscellaneous change

[codecov]

Codecov Report

Attention: Patch coverage is 91.42857% with 3 lines in your changes missing coverage. Please review.

Project coverage is 86.66%. Comparing base (d420770) to head (104b8d8).
Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
src/stcal/ramp_fitting/ols_fit.py	66.66%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #318      +/-   ##
==========================================
+ Coverage   86.38%   86.66%   +0.28%     
==========================================
  Files          49       49              
  Lines        8899     8934      +35     
==========================================
+ Hits         7687     7743      +56     
+ Misses       1212     1191      -21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[melanieclarke]

Looking good!

Testing on the sample data that failed in #304, I no longer see the processing hang with the optional output + multiprocessing. Optional output looks the same with and without multiprocessing now, and also shows only minor differences from output with the main branch + algorithm = 'OLS'.

Did you run regression tests for jwst and romancal? If so, can you please link them here?

[kmacdonald-stsci]

Looking good!

Testing on the sample data that failed in #304, I no longer see the processing hang with the optional output + multiprocessing. Optional output looks the same with and without multiprocessing now, and also shows only minor differences from output with the main branch + algorithm = 'OLS'.

Did you run regression tests for jwst and romancal? If so, can you please link them here?

Here is a link to the regression tests I ran on these PRs:

https://github.com/spacetelescope/RegressionTests/actions/runs/11820513473

These changes don't effect RomanCal, but I will ask a RomanCal maintainer to run regression tests against these changes.

[melanieclarke]

Regression tests for jwst look good. Docs are building cleanly now -- thanks for the change notes fix!

I have a couple more minor copy editing suggestions for the change note, but I will approve now, from my perspective. We should still wait to make sure romancal tests are clean.

[ddavis-stsci]

The romancal regression tests pass and are at
https://github.com/spacetelescope/RegressionTests/actions/runs/11895485690

[jemorrison]

I left a few comments - all minor some just for my benefit.
It all seems reasonable to me. There are a lot of changes in the PR.