Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dask statistics #922

Merged
merged 1 commit into from
Oct 18, 2024
Merged

Conversation

keflavich
Copy link
Contributor

Dask statistics were returning nans if any chunk was all-nan. This fixes that issue, at a slight computational cost (I hope slight, anyway).

Example that I put in a regression test (approximately):

>>> cube = SpectralCube.read('tests/data/example_cube.fits', use_dask=True)
>>> cube._data[:,:,:2] = np.nan
>>> cube.rechunk((1,2,2)).statistics()
{'npts': 28.0,
 'min': <Quantity nan Jy / beam>,
 'max': <Quantity nan Jy / beam>,
 'sum': <Quantity 0.09592362 Jy / beam>,
 'sumsq': <Quantity 0.00211593 Jy2 / beam2>,
 'mean': <Quantity 0.00342584 Jy / beam>,
 'sigma': <Quantity 0.00813614 Jy / beam>,
 'rms': <Quantity 0.00869304 Jy / beam>}

@keflavich keflavich force-pushed the fix_dask_statistics branch from 7f61b25 to f238c7f Compare October 13, 2024 00:24
@e-koch
Copy link
Contributor

e-koch commented Oct 18, 2024

Merging. Tests failures are unrelated.

@e-koch e-koch merged commit 7932645 into radio-astro-tools:master Oct 18, 2024
4 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants