MIRI and NIRcam PSF photometry with spacephot #173
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:25:59Z The notebook needs many more comments. It is not clear what has been done and why in many parts.
WebbPSF PSFs are made but no details are provided, so it is hard to evaluate what happened. WebbPSF for MIRI have various issues and they are ignored in the analysis below, making the results not very trustworthy.
There is no indicator of the PSF fit itself. How can a user objectively evaluate the results? Looking at the residuals or the corner plots is only partially useful. orifox commented on 2023-11-26T18:29:10Z Thanks. This was an excellent review. Very thorough and you understood a lot of the intricacies. I've tried to address all of your comments, and I think I have to within the scope of this notebook. Particularly how to use the results to determine the goodness of fit. Remaining are a number of Developer Notes (Action Items) to significantly improve the documentation for WebbPSF, Photutils, JDox, and of course, Space_Phot. We'll need to add a lot of that, but I think that is a much larger effort beyond the scope of this notebook. At the current time, users have no examples on how to do PSF photometry. Particularly no examples on how to generate even the most basic JWST PSFs and use them to obtain PSF photometry. So I think if we can confirm the photometry results here are indeed correct and have reliable error bars, we can move forward. Particulary given the PSF notebooks that are currently available. What would be good is to understand within the scope of all of these limitations, what you think the best way to move forward is.
Please remember, the only PSF photometry that users are currently being directed to is by Matteo from 3 years ago: |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:00Z Please provide clear instructions about how to install the packages. If you install only space_phot, it retrieves the latest versions of all packages but new changes break some parts. The packages I had to change are the following:
orifox commented on 2023-11-21T16:51:18Z Fixed. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:01Z I would add a variable for the folder given that it might be different from "mast" orifox commented on 2023-11-22T20:11:28Z Fixed and hard-coded based on download script now included. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:02Z Using jwst_download, the name of the json file includes the date. Add a comment to remember to change it orifox commented on 2023-11-23T15:43:28Z Good catch. Now automated and fixed. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:03Z It should be explained what a DQ of 1 means. Also, the DQ flags that should not be used are many more than that. For example, it might be worth to exclude all pixels with a DQ>10, or at least those with a very large number.
I would also mask all pixels related to the 4QPM regions because we want to avoid users to make any science with them. orifox commented on 2023-11-21T20:43:34Z Done. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:05Z Line #4. ref_fits = fits.open(ref_image)
Why not using the jwst datamodels? Besides that usually we want to push to use products made "in the house", the jwst datamodels offer some advantages, like an easiest access to the header keywords and built-in functions. orifox commented on 2023-11-23T15:46:06Z jwst datamodels now used. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:06Z Line #6. norm1 = simple_norm(ref_data,stretch='linear',min_cut=-1,max_cut=10)
The scale should highlight the background noise so it is possible to see all faint sources. I would add a comment and change the values between 4 and 5, and/or use a log scale. orifox commented on 2023-11-23T15:46:26Z Fixed. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:07Z Line #10. plt.show()
I would make the image larger, it is hard to see the details.
This is a "scientific" notebook, so we should make it as such. I would add axis labels and a colorbar. I would add the units (both pixels for the x/y axes and MJy/sr for the colorbar). orifox commented on 2023-11-21T21:53:41Z Done. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:08Z Line #2. source_location = SkyCoord('17:43:04.4879','+66:55:01.837',unit=(u.hourangle,u.deg))
There is no explanation about this source. Why was it chosen among the rest in the field? Does it have an ID in some catalogs like Gaia? orifox commented on 2023-11-23T15:55:00Z Ok, updated. We are using Visit 006, which targets the A5V dwarf 2MASSJ17430448+6655015. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:09Z Line #10. plt.show()
Same comments as before: labels, units, scale. I would zoom out a bit to show the wings of the PSF. orifox commented on 2023-11-21T22:00:28Z Done. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:10Z Besides that units and labels are missing, there is no explanation of what we are seeing. This is the PSF made by WebbPSF. How was it made? Did you use a monochromatic PSF or some wavelengths? Was it generated with the spectrum of the source? Is it oversampled and, if so, by how much?
You are analyzing a LVL-2 cal image. Are you using the distorted or the "undistorted" PSF from WebbPSF? It is critical to use the correct one.
WebbPSF PSFs do not include many detector effects. Depending on which version of WebbPSF you are using, there might be only the IPC effect. Also, the cruciform model in WebbPSF is wrong. It has been adjusted in the latest version to match that of my PSFs. Regardless, WebbPSF includes the cruciform artifact ONLY in the distorted PSF. orifox commented on 2023-11-22T17:09:27Z # Get PSF from WebbPSF # The function get_jwst_psf is a space_phot wrapper for the WebbPSF calc_psf function and uses a lot of the same keywords. # There are more advanced methods for generating your WebbPSF, but those are beyond the scope of this notebook. # The defaults used by get_jwst_psf in this notebook are: # oversample=4 # normalize='last' # Non-distorted PSF # Useful reference: https://webbpsf.readthedocs.io/en/latest/api/webbpsf.JWInstrument.html#webbpsf.JWInstrument.calc_psf orifox commented on 2023-11-23T16:51:51Z Ok, these are all great questions. A DEV note has been added. A DEV note is standard when more work needs to be done, but shouldn't hold up the publishing of the notebook. In this case there are a lot of things to do. First, space_phot needs updated documentation on what get_jwst_psf does. For now, I have added comments into the notebook referencing how it calls WebbPSF's calc_psf function using the defaults. That's sufficient for the purposes of this notebook. Generating more complex PSFs is beyond the scope of this notebook and should be built into the WebbPSF documentation or additional MIRI JDox documentation and/or notebooks. So that is Dev Note #2.
For moving forward, we have added brief comments to temporarily address these issues with details about how the PSF is being generated. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:10Z The lack of comments makes it hard to understand all outputs here:
Why is the table at the bottom showing the results in Equatorial coordinates? You are measuring coordinates of sources on a pixel grid, so the result should be in pixels
How do you get calibrated magnitudes from the PSF fit? Again, more explanations would help.
Why are there errors on the estimate of the position and flux? This is a single measurement of one source in one image, how can you get an error from that? I'm always skeptical about errors computed this way. orifox commented on 2023-11-23T17:31:58Z Ok, a lot of comments to respond to here. Let me try my best.
|
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:12Z By averaging directly the magnitudes from the 4 images, you make the implicit assumption that there is no zero-point between the 4 exposures. The zero-point between consecutive exposures should be very small, less than 0.01 mag, but depending on the precision you want to reach it might matter.
You estimate the mag error as the sum in quadrature of the errors of the individual measurements. This: (1) assumes that individual errors are legit, but they might not be (see my comment in the previous cell); (2) does not allow you to detect outliers. If you compute the error as the rms or the standard error of the mean, you can clearly see if the measurements are consistent with each other or not. orifox commented on 2023-11-23T18:04:14Z Good catch. The averaging was all a bit old and not supposed to be in there for the exact reasons you give. The whole point to simultaneous fitting is to account for all of this. As a result, the flux and corresponding error take into account a single overall fit. Therefore, there is no need to average the resulting magnitudes or errors. They should all be the same to within their individual zero-point differences (typically <1%).
This is now captured in the documentation, notes, and Dev Notes. I think that addresses all of these concerns. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:13Z In the followings sections, I have the same comments as in the previous part. I will add only new comments or additional explanations when necessary. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:14Z Why is the pipeline run here?
As I wrote for the corresponding cell in the previous section, details are missing? Is this the WebbPSF PSF corresponding for the LVL-3 image? If so, note that there is the problem of the missing cruciform.
One of the problems I have with doing PSF photometry on the LVL-3 images is that the flux distribution of the stars in the mosaic changes depending on how the mosaic was made. Kernels and weights can result in different flux distributions. So, PSFs might not be a good representation of point-like sources in all mosaics. orifox commented on 2023-11-23T17:42:13Z At this point, we are taking several Level2 PSFs and resampling with the pipeline to make a Level2 PSF. We don't know how to turn off the output when that step is run.
Regarding your questions, the Level2 PSFs are generated at the precise location of the source in each Level2 file to account for detector level effects. The resampling uses default resampling paramters. However, users should be aware that if they performed customized resampling for their Level2 data products, they should use similar resampling steps for their PSF below. We added a couple more comments and Dev Notes about these topics and believe that this comment is adequately addressed for the scope of this notebook. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:16Z Where is the source here? orifox commented on 2023-11-22T17:30:14Z The goal is to calculate an upper limit on the blank part of the sky. Additional clarification has been written. mlibralato commented on 2023-11-27T20:02:33Z I still do not understand this test and to me it does not make sense. orifox commented on 2023-11-27T20:34:08Z I'm not sure how else to say it. The goal is to calculate an upper limit. Let's say you want to try to determine how bright a source is, but you can't see it in a particular filter. Then you need to calculate an upper limit. How do you normally calculate upper limits?
mlibralato commented on 2023-11-27T20:48:33Z I think you mean lower limit. We add many sources of various brightness across the entire FoV and try to measure them. At a certain threshold, you recover less stars, let's say 50%. Then you go X mag fainter and you recover only 20% and so on. These values tell you what you can measure in an image.
orifox commented on 2023-11-27T20:51:57Z Right, that's how you would traditionally do it over a field. But in many cases, and supernovae in particular, you have a very complicated background in a very specific region of the detector. Dropping a source at that position is certainly one way to do it. But doing PSF photometry with the "Upper Limit" mode turned on essentially does this for you. And with less systematic error. Also, in general, we call this an "Upper Limit". Because the source could be fainter, but not brighter. If it were brighter, we would have detected it. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:17Z To me, it seems that you have fitted the background noise.
The result of the fit is a negative flux, which is unphysical. If the fit failed, the code should tell the user. orifox commented on 2023-11-23T18:06:21Z These parameters were supposed to be optimized before we submitted the notebook, but that never got done. They are now optimized. In any case, PSF fitting does fit the background noise. That is the whole point to being able to calculate upper limits from PSF fitting. The values returned are the same as in Photutils. Photutils doesn't report a failed fit either. Instead, users need to be able to understand the goodness of fit from their error bars. That is now captured in the text in the notebook. |
|
View / edit / reply to this conversation on ReviewNB mlibralato commented on 2023-11-06T15:26:18Z There is no source fit here. Whatever it was fitted, it was not a star. orifox commented on 2023-11-22T17:37:56Z Need to ask Justin about the "Fix Background" Parameter setting here. orifox commented on 2023-11-23T18:15:35Z Thanks. Again we never got to finalizing the optimal paramters. That has now been done. |
This notebook checklist has been made available to us by the the Notebooks For All team.
Its purpose is to serve as a guide for both the notebook author and the technical reviewer highlighting critical aspects to consider when striving to develop an accessible and effective notebook.
The First Cell
<h1>or# in markdown).1., 2.,etc. in Markdown).The Rest of the Cells
#in Markdown) used in the notebook.Text
Code
Images
All images (jpg, png, svgs) have an image description. This could be
altproperty)altattribute with no value)Any text present in images exists in a text form outside of the image (this can be alt text, captions, or surrounding text.)
Visualizations
All visualizations have an image description. Review the previous section, Images, for more information on how to add it.
Visualization descriptions include
All visualizations and their parts have enough color contrast (color contrast checker) to be legible. Remember that transparent colors have lower contrast than their opaque versions.
All visualizations convey information with more visual cues than color coding. Use text labels, patterns, or icons alongside color to achieve this.
All visualizations have an additional way for notebook readers to access the information. Linking to the original data, including a table of the data in the same notebook, or sonifying the plot are all options.