Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Produce all required quantities for vectorized source measurements #87

Commits on Oct 14, 2024

  1. Because of Numba limitations, one cannot raise a ValueError, which wo…

    …uld be preferred, here. The message is needed to indicate the nature of the error.
    HannoSpreeuw committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    eddcc25 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    cdde415 View commit details
    Browse the repository at this point in the history
  3. More blank lines

    HannoSpreeuw committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    9c76e3c View commit details
    Browse the repository at this point in the history

Commits on Oct 15, 2024

  1. Configuration menu
    Copy the full SHA
    fe88cbb View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. Configuration menu
    Copy the full SHA
    2b3429f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8fb75ea View commit details
    Browse the repository at this point in the history
  3. Reformatting

    This is some reformatting happening accidentally after trying to add the computation of the signal-to-noise ratios to this function. It was removed since it seems more efficient to add this computation to the 'insert_island_data' function.
    HannoSpreeuw committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    5d1c276 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. Calculate signal-to-noise ratios of detections

    This is for VECTORIZED=True. This was still missing from the vectorized part of PySE.
    It seemed easiest to add this to 'insert_island_data', but requires the noise map as
    an extra input argument. The dummy array can now actually be used when implemented as
    float32 instead of int32. The significance (sig) of the detection is defined as the
    maximum signal-to-noise ratio of the spectral brightness to the local noise, i.e. the
    local standard deviation of the background pixel values, across all island pixels.
    Added some clarification here and there.
    HannoSpreeuw committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    a48f73f View commit details
    Browse the repository at this point in the history
  2. More clarifications.

    HannoSpreeuw committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    d896e9a View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2024

  1. First step in computing chisq and reduced_chisq vectorized

    After some considerations it turns out that chisq and reduced_chisq are best calculated in 'fitting.moments_enhanced'. That, however, also requires the availability of 'noise_islands', i.e. the rms noise values at the positions of the sources, i.e. at the positions of the pixels comprising 'island_data', in that function. In turn that meant that we need some array with rms noise values similar to 'islands', which contains spectral brightnesses at the positions of the source. To fill 'noise_islands', 'insert_island_data' was expanded and the calculation of the significance of the detection was removed, since this is also best calculated in 'fitting.moments_enhanced'. In affectuating this change, it turns out that 'islands', 'island_data' and 'insert_island_data' are no longer accurate terms since an island can be an island of spectral brightnesses of a source or an island of rms noise values. Hence, the changes in variable and function names. The actual computations of sig, chisq and reduced_chisq will have to covered in upcoming commits. The current commit also includes some corrections and clarifications in comments. Some arrays were initialized later, i.e. closer to the function call. The 'noises' array has the same shape as the 'sources' array, which was formerly called the 'islands' array.
    HannoSpreeuw committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    7ba044a View commit details
    Browse the repository at this point in the history
  2. 'sig' is no longer an array to be filled

    'sig' should not be filled with values through 'insert_sources_and_noise', i.e. should no longer be an input argument of that function.
    It has to be filled in 'fitting.moments_enhanced' in an upcoming commit.
    HannoSpreeuw committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    02dca2c View commit details
    Browse the repository at this point in the history
  3. Remove 'sig' also from signature.

    In a previous commit, it was only removed from the arguments.
    HannoSpreeuw committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    f7d69e5 View commit details
    Browse the repository at this point in the history
  4. 'fitting.moments_enhanced' computes significances

    of each source detection. This was formerly computed in 'insert_island_data' for the vectorized part of PySE.
    HannoSpreeuw committed Oct 19, 2024
    Configuration menu
    Copy the full SHA
    2b8bc04 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. Reformatting to 80 characters per line

    for 'moments'. The 'moments_enhanced' docstring was taking too many lines, so also reformatted to fill out more blank space.
    HannoSpreeuw committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    84cd03b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a9980c8 View commit details
    Browse the repository at this point in the history
  3. Redundant parenthesis

    HannoSpreeuw committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    7a066bc View commit details
    Browse the repository at this point in the history
  4. 'indep_pixels' is to be called from guvectorized

    code, therefore it needs a njit decorator from Numba. Also, allow for two blank lines before a function.
    HannoSpreeuw committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    df7ebee View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    1a7d120 View commit details
    Browse the repository at this point in the history
  6. Calc. chisq + reduced_chisq in 'moments_enhanced'

    Equivalent to the code in 'goodness_of_fit'. Needed 'indep_pixels' to be njitted. Also, replace numpy. by np. because it's shorter.
    HannoSpreeuw committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    4d334b6 View commit details
    Browse the repository at this point in the history
  7. Indendation corrections from numpy. --> np.

    and comment within 80 characters per line.
    HannoSpreeuw committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    fed0b07 View commit details
    Browse the repository at this point in the history
  8. chisq and reduced_chisq to be filled with

    the corresponding statistics for each detection. Both arrays are 1D single precision floats.
    HannoSpreeuw committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    15d1aae View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. 1.05 was apparently not chosen, cannot remember

    why. And preparing for the Gaussian_islands and Gaussian_residuals arrays to be filled. This is just initialisation.
    HannoSpreeuw committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    e43be70 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2024

  1. The 'data' attribute should not be used.

    Instead, use of MaskedArray will ensure that masks will be propagated.
    HannoSpreeuw committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    3a7919a View commit details
    Browse the repository at this point in the history
  2. 'self.data_bgsubbed' became numpy.float64

    although we started off with 'self.rawdata = numpy.ascontiguousarray(data, dtype=numpy.float32)'. However, the function 'data_bgsubbed(self)' returns 'self.data - self.backmap' and the latter is numpy.float64 since we had 'my_map = numpy.ma.MaskedArray(numpy.zeros(self.data.shape), mask=self.data.mask)' which creates a numpy.float64 array. Also, 'mode_and_rms = useful_data.map_blocks(..,..,dtype=numpy.complex64,..)' oddly enough returns numpy.complex128! For that reason 'mode_grid' and 'rms_grid' were converted to four byte floats.
    HannoSpreeuw committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    1af20ab View commit details
    Browse the repository at this point in the history
  3. The map of Gaussian reconstructions and residuals

    is now computed within 'fitting.moments_enhanced'. After reconsideration, this function turned out to be the best candidate to compute those arrays. It did require adding two array arguments to this function, i.e. the zeroed 'Gaussian_islands' and 'Gaussian_residuals' arrays. These arrays can later be saved as maps (FITS files).
    HannoSpreeuw committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    043673a View commit details
    Browse the repository at this point in the history
  4. No longer return 'x_positions' and 'y_positions'

    since they will not be used. Instead, return 'Gaussian_islands' and 'Gaussian_residuals'. 'image.py' has been adapted such that 'data_bgsubbeddata' will always be a np.float32 array, i.e. not np.float64. 'extract.calculate_Gaussian_islands' is no longer needed.
    HannoSpreeuw committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    a0a86dc View commit details
    Browse the repository at this point in the history
  5. Updated docstring for

    'source_measurements_pixels_and_celestial_vectorised', since we are returning five more arrays.
    HannoSpreeuw committed Oct 28, 2024
    Configuration menu
    Copy the full SHA
    6b23f2c View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2024

  1. Ensure that ImageData.data_bgsubbed is np.float32

    and 'VECTORIZED = False' is required to pass all unit tests - the two that are not passed will be fixed later. 'Gaussian_islands', 'Gaussian_residuals', 'sig', 'chisq' and reduced_chisq are now returned. Calling 'extract.calculate_Gaussian_islands' is no longer needed and this function has been removed in a previous commit.
    HannoSpreeuw committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    7655151 View commit details
    Browse the repository at this point in the history
  2. Clarify the use of Gaussian_islands_map and

    Gaussian_residuals_map.
    HannoSpreeuw committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    66afb02 View commit details
    Browse the repository at this point in the history
  3. Replace 'numpy.' by 'np.' for reasons of brevity

    and an extended docstring.
    HannoSpreeuw committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    aef50af View commit details
    Browse the repository at this point in the history
  4. Again, replace "numpy." by "np.", for reasons of

    brevity. This also affected indentation for multiline commands here and there, indentation adapted.
    HannoSpreeuw committed Oct 29, 2024
    Configuration menu
    Copy the full SHA
    fc951bd View commit details
    Browse the repository at this point in the history