Produce all required quantities for vectorized source measurements #87
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…uld be preferred, here. The message is needed to indicate the nature of the error.
This is some reformatting happening accidentally after trying to add the computation of the signal-to-noise ratios to this function. It was removed since it seems more efficient to add this computation to the 'insert_island_data' function.
This is for VECTORIZED=True. This was still missing from the vectorized part of PySE. It seemed easiest to add this to 'insert_island_data', but requires the noise map as an extra input argument. The dummy array can now actually be used when implemented as float32 instead of int32. The significance (sig) of the detection is defined as the maximum signal-to-noise ratio of the spectral brightness to the local noise, i.e. the local standard deviation of the background pixel values, across all island pixels. Added some clarification here and there.
After some considerations it turns out that chisq and reduced_chisq are best calculated in 'fitting.moments_enhanced'. That, however, also requires the availability of 'noise_islands', i.e. the rms noise values at the positions of the sources, i.e. at the positions of the pixels comprising 'island_data', in that function. In turn that meant that we need some array with rms noise values similar to 'islands', which contains spectral brightnesses at the positions of the source. To fill 'noise_islands', 'insert_island_data' was expanded and the calculation of the significance of the detection was removed, since this is also best calculated in 'fitting.moments_enhanced'. In affectuating this change, it turns out that 'islands', 'island_data' and 'insert_island_data' are no longer accurate terms since an island can be an island of spectral brightnesses of a source or an island of rms noise values. Hence, the changes in variable and function names. The actual computations of sig, chisq and reduced_chisq will have to covered in upcoming commits. The current commit also includes some corrections and clarifications in comments. Some arrays were initialized later, i.e. closer to the function call. The 'noises' array has the same shape as the 'sources' array, which was formerly called the 'islands' array.
'sig' should not be filled with values through 'insert_sources_and_noise', i.e. should no longer be an input argument of that function. It has to be filled in 'fitting.moments_enhanced' in an upcoming commit.
In a previous commit, it was only removed from the arguments.
of each source detection. This was formerly computed in 'insert_island_data' for the vectorized part of PySE.
for 'moments'. The 'moments_enhanced' docstring was taking too many lines, so also reformatted to fill out more blank space.
code, therefore it needs a njit decorator from Numba. Also, allow for two blank lines before a function.
Equivalent to the code in 'goodness_of_fit'. Needed 'indep_pixels' to be njitted. Also, replace numpy. by np. because it's shorter.
and comment within 80 characters per line.
the corresponding statistics for each detection. Both arrays are 1D single precision floats.
why. And preparing for the Gaussian_islands and Gaussian_residuals arrays to be filled. This is just initialisation.
Instead, use of MaskedArray will ensure that masks will be propagated.
although we started off with 'self.rawdata = numpy.ascontiguousarray(data, dtype=numpy.float32)'. However, the function 'data_bgsubbed(self)' returns 'self.data - self.backmap' and the latter is numpy.float64 since we had 'my_map = numpy.ma.MaskedArray(numpy.zeros(self.data.shape), mask=self.data.mask)' which creates a numpy.float64 array. Also, 'mode_and_rms = useful_data.map_blocks(..,..,dtype=numpy.complex64,..)' oddly enough returns numpy.complex128! For that reason 'mode_grid' and 'rms_grid' were converted to four byte floats.
is now computed within 'fitting.moments_enhanced'. After reconsideration, this function turned out to be the best candidate to compute those arrays. It did require adding two array arguments to this function, i.e. the zeroed 'Gaussian_islands' and 'Gaussian_residuals' arrays. These arrays can later be saved as maps (FITS files).
since they will not be used. Instead, return 'Gaussian_islands' and 'Gaussian_residuals'. 'image.py' has been adapted such that 'data_bgsubbeddata' will always be a np.float32 array, i.e. not np.float64. 'extract.calculate_Gaussian_islands' is no longer needed.
'source_measurements_pixels_and_celestial_vectorised', since we are returning five more arrays.
and 'VECTORIZED = False' is required to pass all unit tests - the two that are not passed will be fixed later. 'Gaussian_islands', 'Gaussian_residuals', 'sig', 'chisq' and reduced_chisq are now returned. Calling 'extract.calculate_Gaussian_islands' is no longer needed and this function has been removed in a previous commit.
Gaussian_residuals_map.
and an extended docstring.
brevity. This also affected indentation for multiline commands here and there, indentation adapted.
HannoSpreeuw
deleted the
produce_all_required_quantities_for_vectorized_source_measurements
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #78