Skip to content

Commit

Permalink
More doc updates (#156)
Browse files Browse the repository at this point in the history
* Documentation updates following RTB review.
  • Loading branch information
schlafly authored Sep 18, 2024
1 parent ff0fb51 commit f3131ad
Show file tree
Hide file tree
Showing 24 changed files with 313 additions and 291 deletions.
7 changes: 5 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from datetime import datetime
from pathlib import Path

import stsci_rtd_theme
# import stsci_rtd_theme
if sys.version_info < (3, 11):
import tomli as tomllib
else:
Expand Down Expand Up @@ -46,6 +46,7 @@ def setup(app):
"sphinx_automodapi.automodapi",
"sphinx_automodapi.smart_resolver",
"sphinx.ext.intersphinx",
"sphinx_rtd_theme",
]

autosummary_generate = True
Expand All @@ -67,7 +68,9 @@ def setup(app):
html_theme_options = {
"collapse_navigation": True
}
html_theme_path = [stsci_rtd_theme.get_html_theme_path()]
# html_theme_path = [stsci_rtd_theme.get_html_theme_path()]
# the internet reports that this can cause problems with searching,
# which was hanging at readthedocs.
html_domain_indices = True
html_sidebars = {"**": ["globaltoc.html", "relations.html", "searchbox.html"]}
html_use_index = True
Expand Down
2 changes: 1 addition & 1 deletion docs/romanisim/bandpass.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Bandpasses

The simulator can render scenes in a number of different bandpasses. The choice of bandpass affects the point spread function used, the sky backgrounds, the fluxes of sources, and the reference files requested.

At present, romanisim simply passes the choice of bandpass to other packages---to webbpsf for PSF modeling, to galsim.roman for sky background estimation, to CRDS for reference file selection, or to the catalog for the selection of appropriate fluxes. However, because catalog fluxes are specified in "maggies" (i.e., in linear fluxes on the AB scale), the simulator needs to know how to convert between a maggie and the number of photons Roman receives from a source. Accordingly, the simulator knows about the AB zero points of the Roman filters, as derived from https://roman.gsfc.nasa.gov/science/WFI_technical.html . The conversion between photons received and fluxes in AB "maggies" is determined by integrating a flat 3631 Jy AB reference spectrum over the Roman filters to determine the number of photons per second corresponding to a zeroth magnitude AB source.
At present, romanisim simply passes the choice of bandpass to other packages---to webbpsf for PSF modeling, to galsim.roman for sky background estimation, to CRDS for reference file selection, or to the catalog for the selection of appropriate fluxes. However, because catalog fluxes are specified in "maggies" (i.e., such that :math:`\mathrm{flux} = 10^{-\mathrm{AB mag} / 2.5}`), the simulator needs to know how to convert between a maggie and the number of electrons Roman receives from a source. Accordingly, the simulator knows about the AB zero points of the Roman filters, as derived from https://roman.gsfc.nasa.gov/science/WFI_technical.html . The conversion between electrons received and fluxes in AB "maggies" is determined by integrating a flat 3631 Jy AB reference spectrum over the Roman filters to determine the number of electrons per second corresponding to a zeroth magnitude AB source.

One technical note: it is unclear what aperture is used for the bandpasses provided by Goddard. The Roman PSF formally extends to infinity and some light is received by the detector but is so far from the center of the PSF that it is not useful for flux, position, or shape measurements. Often for the purposes of computing effective area curves only light landing within a fixed aperture is counted. Presently the simulator assumes that an infinite aperture is used. This can result in an approximately 10% different flux scale than more reasonable aperture selections.

Expand Down
4 changes: 2 additions & 2 deletions docs/romanisim/image-input.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Image Input
===========

The simulator can alternatively take as input a catalog that references a separate list of images of sources. The format of the catalog is described below, and the underlying "list of images" is described by a ``RealGalaxyCatalog``; see :ref:`constructing-a-rgc`. The concept here is that users may have noiseless images of galaxies from, for example, hydro simulations, and may wish to inject those into simulated images at specific locations. Or users may have reference images from, for example, the COSMOS field, which could be used as truth images for simulations.
The simulator can alternatively take as input a catalog that references a separate list of images of sources. The format of the catalog is described below, and the underlying "list of images" is described by a ``RealGalaxyCatalog``; see :ref:`constructing-a-rgc`. The concept here is that users may have noiseless images of galaxies from, for example, hydrodynamical simulations, and may wish to inject those into simulated images at specific locations. Or users may have reference images from, for example, the COSMOS field, which could be used as truth images for simulations.

The simulator leverages GalSim's ``RealGalaxyCatalog`` framework to support this use case. There are a handful of existing catalogs of images in the appropriate format for use with the ``RealGalaxyCatalog``---in particular, `GalSim's COSMOS reference catalog <https://galsim-developers.github.io/GalSim/_build/html/real_gal.html#downloading-the-cosmos-catalog>`_ .

Expand All @@ -26,7 +26,7 @@ For image input, the simulator takes a catalog with the following structure::
269.9 66.1 1 6.91 109.7 0.5 0.4 2.5e-07
269.9 66.0 0 12.04 44.2 0.6 0.1 2.7e-07

Moreover, the table metadata must include the keyword ``real_galaxy_catalog_filename`` specifying the location of the ``RealGalaxyCatalog`` catalog file. The ``ident`` keyword then specifies the id of the galaxy image to use from the RealGalaxyCatalog, which is rendered at the specified location (ra, dec) and is optionally rotated (rotate), sheared (shear_pa, shear_ba), and dilated (dilate). Finally, total fluxes in maggies (see the catalog :doc:`docs </romanisim/catalog>`) in specified bandpasses are given.
Moreover, the table metadata must include the keyword ``real_galaxy_catalog_filename`` specifying the location of the ``RealGalaxyCatalog`` catalog file. The ``ident`` keyword then specifies the ID of the galaxy image to use from the RealGalaxyCatalog, which is rendered at the specified location (ra, dec) and is optionally rotated (rotate), sheared (shear_pa, shear_ba), and dilated (dilate). Finally, total fluxes in maggies (see the catalog :doc:`docs </romanisim/catalog>`) in specified bandpasses are given.

The following fields must be specified for each source:

Expand Down
4 changes: 2 additions & 2 deletions docs/romanisim/image.rst
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
Making images
=============

Ultimately, romanisim builds image by feeding source profiles, world coordinate system objects, and point spread functions to galsim. The ``image`` and ``l1`` modules currently implement this functionality.
Ultimately, romanisim builds images by feeding source profiles, world coordinate system objects, and point spread functions to galsim. The ``image`` and ``l1`` modules currently implement this functionality.

The ``image`` module is responsible for translating a metadata object that specifies everything about the conditions of the observation into objects that the simulation can understand. The metadata object follows the metadata that real WFI images will include; see `here <https://roman-pipeline.readthedocs.io/en/latest/roman/datamodels/metadata.html#metadata>`_ for more information.

The parsed metadata is used to make a ``counts`` image that is an idealized image containing the number of photons each WFI pixel would collect over an observation. It includes no systematic effects or noise beyond Poisson noise from the astronomical scene and backgrounds. Actual WFI observations are more complicated than just noisy versions of this idealized image, however, for several reasons:

* WFI pixels have a very uncertain pedestal.
* WFI pixels are sampled "up the ramp" during an observation, so a number of reads contribute to the final estimate for the rate of photons entering each pixel.
* WFI reads are averaged on the telescope into resultants; ground images see only resultants.
* WFI reads are averaged on the telescope into resultants; these resultants are downlinked to the ground, instead of the individual reads.

These idealized count images are then used to either make a level 2 image or a level 1 image, which are intended to include
the effects of these complications. The construction of L1 images is described :doc:`here </romanisim/l1>`, and the construction of L2 images is described :doc:`here </romanisim/l2>`.
Expand Down
29 changes: 14 additions & 15 deletions docs/romanisim/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,33 +7,36 @@ To install ::

and you should be largely set!

There are a few dependencies that may cause more difficulty. First,
`WebbPSF <https://webbpsf.readthedocs.io>`_ requires data files to
The most frequently encountered difficulty installing romanisim is
when GalSim is unable to find FFTW. If your system does not have the
FFTW library, these should be installed before romanisim and GalSim.

Another problematic dependency is `WebbPSF
<https://webbpsf.readthedocs.io>`_, which requires data files to
operate. See the `docs
<https://webbpsf.readthedocs.io/en/latest/installation.html#installing-the-required-data-files>`_
for instructions on obtaining the relevant data files and pointing the
``WEBBPSF_PATH`` environment variable to them. This issue can be
avoided by not setting the ``--webbpsf`` argument, in which case
``romanisim`` uses the GalSim modeling of the Roman PSF.

Second, some synthetic scene generation tools use images of galaxies
Additionally, some synthetic scene generation tools use images of galaxies
distributed separately from the main GalSim source. See `here
<https://galsim-developers.github.io/GalSim/_build/html/real_gal.html#downloading-the-cosmos-catalog>`_
for information on obtaining the COSMOS galaxies for use with GalSim.
The ``romanisim`` package also has a less sophisticated scene modeling
toolkit, which just renders Sersic galaxies. The command line
interface to ``romanisim`` presently uses supports Sersic galaxy
interface to ``romanisim`` presently uses Sersic galaxy
rendering, and so many users may not need to download the COSMOS galaxies.

Third, ``romanisim`` can work with the Roman `CRDS
<https://github.com/spacetelescope/crds>`_ system. This functionality
is not available to the general community at the time of writing.
Finally, ``romanisim`` can work with the Roman `CRDS
<https://github.com/spacetelescope/crds>`_ system.
Using CRDS requires specifying the ``CRDS_PATH`` and
``CRDS_SERVER_URL`` variables. CRDS is not used unless the
``--usecrds`` argument is specified; do not include this argument
unless you have access to the Roman CRDS.
``CRDS_SERVER_URL`` variables. Using CRDS brings in the latest
reference files and its use is encouraged; specify the
``--usecrds`` argument to enable it.

That said, the basic install process looks like this::
In summary, the basic install process looks like this::

pip install romanisim
# to get a specific version, use instead
Expand All @@ -58,7 +61,3 @@ That said, the basic install process looks like this::
You may wish to, for example, set up a new python virtual environment
before running the above, or choose a different directory for
WebbPSF's data files.

Some users report issues with the FFTW dependency of galsim on Mac Arm
systems. See galsim's installation page for hints there. In
particular it may be helpful to install FFTW before galsim and romanisim.
14 changes: 7 additions & 7 deletions docs/romanisim/l1.rst
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
Making L1 images
================

An L1 (level 1) image is a "raw" image received from the detectors. The actual measurements made on the spacecraft consist of a number of non-destructive reads of the pixels of the H4RG detectors. These reads have independent read noise but because the pixels count the total number of photons having entered each pixel, the Poisson noise in different reads of the same pixel is correlated.
An L1 (level 1) image is a "raw" image received from the detectors. The actual measurements made on the spacecraft consist of a number of non-destructive reads of the pixels of the H4RG detectors. These reads have independent read noise but because the pixels count the total number of electrons having entered each pixel, the Poisson noise in different reads of the same pixel is correlated.

Because the telescope has limited bandwidth, every read is not transferred to ground stations. Instead, reads are averaged into "resultants" according to a specification called a MultiAccum table, and these resultants are transferred, archived, and analyzed. These resultants make up an L1 image, which romanisim simulates.

L1 images are created using an idealized ``counts`` image described :doc:`here </romanisim/image>`, which contains the number of photons each pixel of the detector would receive absent any instrumental systematics. To transform this into an L1 image, these counts must be apportioned into reads and averaged into resultants, and instrumental effects must be added.
L1 images are created starting with an idealized ``counts`` image described :doc:`here </romanisim/image>`, which contains the number of electrons each pixel of the detector would receive in the absence of any instrumental systematics. To transform this into an L1 image, these electrons must be apportioned into reads and averaged into resultants, and instrumental effects must be added.

This process proceeds by simulating each read, drawing the appropriate number of photons from the total number of photons for each read following a binomial distribution. These photons are added to a running sum that is then averaged into a resultant according to the MultiAccum table specification. This process requires drawing random numbers from the binomial distribution for every read of every pixel, and so can take on the order of a minute, but it allows detailed simulation of the statistics of the noise in each resultant together with their correlations. It also makes it straightforward to add various instrumental effects into the simulation accurately, since these usually apply to individual reads rather than to resultants (e.g., cosmic rays affect individual reads, and their affect on a resultant depends on the read in the resultant to which they apply).
This process proceeds by simulating each read, drawing the appropriate number of electrons from the total number of electrons for each read following a binomial distribution. These electrons are added to a running sum that is then averaged into a resultant according to the MultiAccum table specification. This process requires drawing random numbers from the binomial distribution for every read of every pixel, and so can take on the order of a minute, but it allows detailed simulation of the statistics of the noise in each resultant together with their correlations. It also makes it straightforward to add various instrumental effects into the simulation accurately, since these usually apply to individual reads rather than to resultants (e.g., cosmic rays affect individual reads, and their affect on a resultant depends on the read in the resultant to which they apply).

After apportioning counts to resultants, systematic effects are added to the resultants. Presently only read noise is added. The read noise is averaged down like :math:`1/\sqrt{N}`, where :math:`N` is the number of reads contributing to the resultant.

Expand All @@ -24,7 +24,7 @@ Interpixel Capacitance

Interpixel capacitance (IPC) is added following non-linearity and before read-out. Read noise remains independent among different pixels but the Poisson noise is correlated between pixels by the IPC. We simply convolve the resultants by a 3x3 kernel after apportioning counts to resultants and applying non-linearity but before adding read noise.

This is slightly different than including IPC in the PSF kernel because including IPC in the PSF kernel leaves the Poisson noise uncorrelated.
This is slightly different than including IPC in the PSF kernel because including IPC in the PSF kernel leaves the Poisson noise uncorrelated between pixels.

Persistence
-----------
Expand All @@ -33,15 +33,15 @@ the Fermi description of persistence implemented in GalSim, where the flux in el

.. math:: P(t) = A \frac{1}{1+\exp\left(\frac{-(x-x_0)}{\delta x}\right)} \left(\frac{x}{x_0}\right)^\alpha \left(\frac{t}{1000 \mathrm{s}}\right)^\gamma \, .

Here :math:`P(x, t)` is the rate in electrons per second that the pixel records :math:`t` seconds following receiving a total number of electrons :math:`x`. The parameters :math:`A`, :math:`x_0`, :math:`\delta x`, :math:`\alpha`, :math:`\gamma` may vary from pixel to pixel, though are presently fixed to global constants. This equation for the rate only applies to pixels which were illuminated more than to fill more than their half-well. We follow GalSim and linearly increase the persistence from 0 to the half-well value for illuminations between 0 and half-well.
Here :math:`P(x, t)` is the rate in electrons per second that the pixel records :math:`t` seconds following receiving a total number of electrons :math:`x`. The parameters :math:`A`, :math:`x_0`, :math:`\delta x`, :math:`\alpha`, :math:`\gamma` may vary from pixel to pixel, though are presently fixed to global constants. This equation for the rate only applies to pixels that were illuminated more than to fill more than their half-well. We follow GalSim and linearly increase the persistence from 0 to the half-well value for illuminations between 0 and half-well.

This persistence rate is sampled with a Poisson distribution and added to each pixel read-by-read and incorporated into the resultants in the L1 images.

Persistence-affected pixels are expected to be rare, and are tracked sparsely via a list of the indices of affected pixels, the amount of the illumination, and the times of their illumination. Pixels are dropped from persistence tracking when their persistence rate is less than one electron per 100 seconds. If the same pixel is receives large fluxes multiple times, these are treated as two independent events and the resulting persistence flux is handled by summing the persistence rates given above over each event.
Persistence-affected pixels are expected to be rare, and are tracked sparsely via a list of the indices of affected pixels, the amount of the illumination, and the times of their illumination. Pixels are dropped from persistence tracking when their persistence rate is less than one electron per 100 seconds. If the same pixel receives large fluxes multiple times in past exposures, these are treated as independent events and the resulting persistence flux is handled by summing the persistence rates for each past event.

Cosmic rays
-----------
Cosmic rays are added to the simulation read-by-read. The cosmic ray parameters follow Wu et al. (2023). The locations of cosmic rays are chosen at random to sample the focal plane uniformly. Lengths are chosen according to a power law distribution :math:`p(l) \\sim l^{-4.33}`, with lengths between 10 and 10,000 microns. Charge deposition rates per micron are selected from a Moyal distribution located at 120 electrons per micron with a width of 50 electrons per micron. An idealized charge is computed for each pixel in a read according to the product of the deposition rate per micron and the length of the cosmic ray's path within that pixel. This idealized charge is Poisson sampled and added to the relevant pixels in a read.
Cosmic rays are added to the simulation read-by-read. The cosmic ray parameters follow Wu et al. (2023). The locations of cosmic rays are chosen at random to sample the focal plane uniformly. Lengths are chosen according to a power law distribution :math:`p(l) \propto l^{-4.33}`, with lengths between 10 and 10,000 microns. Charge deposition rates per micron are selected from a Moyal distribution located at 120 electrons per micron with a width of 50 electrons per micron. An idealized charge is computed for each pixel in a read according to the product of the deposition rate per micron and the length of the cosmic ray's path within that pixel. This idealized charge is Poisson sampled and added to the relevant pixels in a read.

Gain
----
Expand Down
Loading

0 comments on commit f3131ad

Please sign in to comment.