From f7bdd37772482770e027476eeac17e773177c102 Mon Sep 17 00:00:00 2001 From: Jenna Ryon Date: Fri, 7 Dec 2018 16:25:21 -0500 Subject: [PATCH] applied edits from Samantha --- .../acs_cte_forward_model_example.ipynb | 205 +++++++++++------- 1 file changed, 129 insertions(+), 76 deletions(-) diff --git a/notebooks/acs_cte_forward_model/acs_cte_forward_model_example.ipynb b/notebooks/acs_cte_forward_model/acs_cte_forward_model_example.ipynb index 7c1a272..75c4b3a 100644 --- a/notebooks/acs_cte_forward_model/acs_cte_forward_model_example.ipynb +++ b/notebooks/acs_cte_forward_model/acs_cte_forward_model_example.ipynb @@ -42,13 +42,13 @@ "\n", "The charge transfer efficiency (CTE) of the Advanced Camera for Surveys (ACS) Wide Field Channel (WFC) has been decreasing over the lifetime of the instrument. Radiation damage from cosmic rays and other sources leads to charge traps within the detector. These traps remove electrons from charge packets as they are transferred between rows of the detector, and release the electrons in subsequent pixels. This causes flux to be removed from bright features and released into pixels behind the features (relative to the row closest to the amplifier), creating bright trails.\n", "\n", - "A pixel-based CTE correction model for the ACS/WFC detector is fully described in [Anderson & Bedin (2010)](http://adsabs.harvard.edu/abs/2010PASP..122.1035A), and a recent update to the model is presented in ACS ISR 2018-04 (link to be added soon). The model is based on an empirical determination of the number and depth of charge traps distributed across the detector. It simulates detector readout of an input image, removes the result from the input, and iterates five times. In this way, a reverse model is successively approximated by the forward model. Electrons released in trails are removed and added back to the bright feature in which they originated.\n", + "A pixel-based CTE correction model for the ACS/WFC detector is fully described in [Anderson & Bedin (2010)](http://adsabs.harvard.edu/abs/2010PASP..122.1035A), and a recent update to the model is presented in [ACS ISR 2018-04](http://www.stsci.edu/hst/acs/documents/isrs/isr1804.pdf). The model is based on an empirical determination of the number and depth of charge traps distributed across the detector. It simulates detector readout of an input image, removes the result from the input, and iterates five times. In this way, a reverse model is successively approximated by the forward model. Electrons released in trails are removed and added back to the bright feature in which they originated.\n", "\n", "The pixel-based correction was implemented in the calibration pipeline code for ACS (`CALACS`) in 2012 and the algorithm was updated and improved in 2018. The CTE correction step within `CALACS` runs on bias-corrected images, `blv_tmp` files, producing `blc_tmp` files, which lack the bright trails due to poor CTE. Further calibration, including dark correction and flat-fielding, produces `flt` and `flc` files from the `blv_tmp` and `blc_tmps` files, respectively. For more information on calibration of ACS/WFC data, see the [ACS Data Handbook](http://www.stsci.edu/hst/acs/documents/handbooks/currentDHB/toc.html).\n", "\n", - "Users desiring to more fully understand the effects of pixel-based CTE correction on their science may wish to run the forward model (i.e., the detector readout simulation) on data containing artificial stars. Here we demonstrate two methods for running the CTE forward model. In [Option A](#option-a), we begin with an observed `flc` image, whereas in [Option B](#option_b), we begin with a raw image and generate synthetic data on which to run the forward model.\n", + "Users desiring to more fully understand the effects of pixel-based CTE correction on their science may wish to run the forward model (i.e., the detector readout simulation) on data containing artificial stars. Here we demonstrate two methods for running the CTE forward model. In [Option A](#option-a), we begin with an observed `flc` image, whereas in [Option B](#option_b), we begin with a `raw` image and generate synthetic data on which to run the forward model.\n", "\n", - "**Note: The forward model, like the CTE correction step in `CALACS`, adds 10% of the difference between the input and output SCI extensions to the ERR extensions to account for uncertainty in the CTE model. Below, we provide guidance for properly repopulating the ERR extensions of forward-modeled data.**" + "**Note: The forward model, like the CTE correction step in `CALACS`, adds 10% of the difference between the input and output `SCI` extensions to the `ERR` extensions to account for uncertainty in the CTE model. Below, we provide guidance for properly repopulating the `ERR` extensions of forward-modeled data.**" ] }, { @@ -58,7 +58,7 @@ "\n", "### 0. Update Astroconda\n", "\n", - "It is highly recommended to use [AstroConda](https://astroconda.readthedocs.io/en/latest/) to manage analysis packages for HST data. If you already use Astroconda, make sure the versions of `hstcal` and [`acstools`](https://acstools.readthedocs.io/en/latest/) are both 2.1.0. This version of `hstcal` should contain `CALACS` version 10.1.0. Check the versions of all three on the command line:\n", + "It is highly recommended to use [AstroConda](https://astroconda.readthedocs.io/en/latest/) to manage analysis packages for HST data. If you already use Astroconda, make sure the versions of `hstcal` and [`acstools`](https://acstools.readthedocs.io/en/latest/) are both at least 2.1.0. The version of `CALACS` should be at least 10.1.0. Check the versions of all three on the command line:\n", "\n", "`$ conda list hstcal`\n", "\n", @@ -155,7 +155,8 @@ "source": [ "obs_table = Observations.query_criteria(obs_id='jd0q14ctq')\n", "\n", - "Observations.download_products(obs_table['obsid'], mrp_only=False, productSubGroupDescription=['FLC','RAW'])\n", + "Observations.download_products(obs_table['obsid'], mrp_only=False, \n", + " productSubGroupDescription=['flc','raw'])\n", "\n", "# Move file to working directory and remove mast-created directory tree\n", "files = glob.glob('mastDownload/HST/jd0q14ctq/jd0q14ctq*.fits')\n", @@ -190,7 +191,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Next, we obtain the filenames of the flat (`PFLTFILE`), CTE-corrected dark (`DRKCFILE`), and superbias (`BIASFILE`) reference files from the image header. The flat will be used to add the effects of the flat field back into the image. If the data were post-flashed, then the flash file (`FLSHFILE`) is needed as well. This is shown in the commented line below. The CTE-corrected dark will be used to add dark current to the `flc` file and the synthetic data. The superbias will be used to repopulate the ERR extensions of the forward-modeled image." + "Next, we obtain the filenames of the flat (`PFLTFILE`), CTE-corrected dark (`DRKCFILE`), and superbias (`BIASFILE`) reference files from the image header. The flat will be used to add the effects of the flat field back into the image. If the data were post-flashed, then the flash file (`FLSHFILE`) is needed as well. This is shown in the commented line below. The CTE-corrected dark will be used to add dark current to the `flc` file and the synthetic data. The superbias will be used to repopulate the `ERR` extensions of the forward-modeled image." ] }, { @@ -224,26 +225,47 @@ "flat_hdu = fits.open('{}/{}'.format(os.environ['jref'],flat))\n", "\n", "flat_wfc1 = flat_hdu[4].data\n", - "flat_wfc2 = flat_hdu[1].data\n", - "\n", + "flat_wfc2 = flat_hdu[1].data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ "dkc_hdu = fits.open('{}/{}'.format(os.environ['jref'],dkc))\n", "\n", "dkc_wfc1 = dkc_hdu[4].data\n", "dkc_wfc2 = dkc_hdu[1].data\n", "\n", - "# Darks can sometimes have negative pixels because they have been flash-corrected and CTE-corrected\n", - "# Set all negative pixels to zero\n", + "# Darks can sometimes have negative pixels because they have been flash-corrected \n", + "# and CTE-corrected. Set all negative pixels to zero\n", "dkc_wfc1[dkc_wfc1 < 0.] = 0.\n", - "dkc_wfc2[dkc_wfc2 < 0.] = 0.\n", - "\n", + "dkc_wfc2[dkc_wfc2 < 0.] = 0." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ "bias_hdu = fits.open('{}/{}'.format(os.environ['jref'],bias))\n", "\n", "err_bias_wfc1 = bias_hdu[5].data\n", "err_bias_wfc2 = bias_hdu[2].data\n", "\n", "dq_bias_wfc1 = bias_hdu[6].data\n", - "dq_bias_wfc2 = bias_hdu[3].data\n", - "\n", + "dq_bias_wfc2 = bias_hdu[3].data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ "# flash_dhu = fits.open('{}/{}'.format(os.environ['jref'],flash))\n", "\n", "# flash_wfc1 = flash_hdu[4].data\n", @@ -264,7 +286,7 @@ "\n", "### 3. Create image of artificial stars\n", "\n", - "We will need an image containing artificial stars on zero background for both methods presented below. Artificial stars are typically generated using models that do not include the flat field, or are produced from data that have been flat-fielded. If this is the case, artificial stars should be added to the image at the `flc` stage.\n", + "We will need an image containing artificial stars on zero background for both options presented below. Artificial stars are typically generated using models that do not include the flat field, or are produced from data that have been flat-fielded. If this is the case, artificial stars should be added to the image at the `flc` stage.\n", "\n", "Users of this notebook may have a preferred method for generating artificial stars and adding them to data, so here we simply add several Gaussians to the image using utilities within `photutils.datasets` in `astropy`. These Gaussians are not representative of the true ACS/WFC PSF, and are added here for illustrative purposes only. Please note that artificial sources with peak values approaching or exceeding the WFC CCD full well value of about 80,000 electrons are not recommended for simulated data. Blooming of charge from saturated pixels is not implemented in this example.\n", "\n", @@ -294,7 +316,8 @@ "\n", "param_ranges = OrderedDict(param_ranges)\n", "\n", - "sources = datasets.make_random_gaussians_table(n_sources, param_ranges, random_state=12345)\n", + "sources = datasets.make_random_gaussians_table(n_sources, param_ranges, \n", + " random_state=12345)\n", "\n", "print(sources)" ] @@ -303,7 +326,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Next, we get the shape of one of the `FLC` image SCI extensions and make an image from the table of Gaussian sources. Finally, we run the synthetic image through a Poisson sampler in order to simulate the Poisson noise of the scene." + "Next, we get the shape of one of the `flc` image `SCI` extensions and make an image from the table of Gaussian sources. Note that this step may take a few minutes. Finally, we run the synthetic image through a Poisson sampler in order to simulate the Poisson noise of the scene." ] }, { @@ -329,9 +352,10 @@ "source": [ "fig, ax = plt.subplots(1,1, figsize=(9,4))\n", "\n", - "ax.imshow(synth_stars_image, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', origin='lower')\n", - "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=10, markerfacecolor='none', markeredgecolor='red',\n", - " linestyle='none')" + "ax.imshow(synth_stars_image, vmin=0, vmax=200, interpolation='nearest', \n", + " cmap='Greys_r', origin='lower')\n", + "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=10, \n", + " markerfacecolor='none', markeredgecolor='red', linestyle='none')" ] }, { @@ -341,7 +365,7 @@ "\n", "### Option A: Start with an observed FLC image\n", "\n", - "We begin with the `flc` image because it is the closest approximation to a pristine image of the sky. Below we plot a portion of the downloaded 47 Tuc image. Stars of various magnitudes are visible, as well as cosmic rays." + "In this Option, we add synthetic stars to the scene in an `flc` image and process it appropriately for use with the forward model. We use the `flc` image because it is the closest approximation to a pristine image of the sky. Below we plot a portion of the downloaded 47 Tuc image. Stars of various magnitudes are visible, as well as cosmic rays." ] }, { @@ -372,7 +396,7 @@ "metadata": {}, "source": [ "\n", - "### 4. Add artificial stars\n", + "### 4A. Add artificial stars\n", "\n", "Add the image of artificial stars generated above to both CCDs of the `flc` image, and save it as a new file." ] @@ -411,9 +435,10 @@ "\n", "fig, ax = plt.subplots(1,1, figsize=(9,4))\n", "\n", - "ax.imshow(flc_stars, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', origin='lower')\n", - "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, markerfacecolor='none', markeredgecolor='red',\n", - " linestyle='none')\n", + "ax.imshow(flc_stars, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', \n", + " origin='lower')\n", + "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, \n", + " markerfacecolor='none', markeredgecolor='red', linestyle='none')\n", "\n", "ax.set_xlim(2000, 2800)\n", "ax.set_ylim(1200, 1700)" @@ -431,7 +456,7 @@ "metadata": {}, "source": [ "\n", - "### 5. Reverse the flat and dark correction\n", + "### 5A. Reverse the flat and dark correction\n", "\n", "First, calculate the total exposure time of the 47 Tuc image by combining the exposure time, flash time (if any), and 3 seconds of extra dark time to approximate instrument commanding overheads." ] @@ -455,7 +480,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Next, we open the 47 Tuc image and obtain the `SCI` extensions of both CCDs. We then multiply by the flat and scale the CTE-corrected dark by the total exposure time. We also run the scaled dark image through a Poisson sampler to include Poisson noise in the dark scene. We then add the dark current to the image. We save the result, which is now effectively a `blc_tmp` file. If the data are post-flashed, we also need to reverse the flash correction. We do this by multiplying the flash file by the flash duration, running it through a Poisson sampler, and adding it to the 47 Tuc image. The lines for this are commented out below. **Note: It is not recommended to use a simulated exposure time that scales pixels in the dark or flash image to or above the full well depth of ~80,000 electrons.**" + "Next, we open the 47 Tuc image and obtain the `SCI` extensions of both CCDs. We multiply by the flat and scale the CTE-corrected dark by the total exposure time. We also run the scaled dark image through a Poisson sampler to include Poisson noise in the dark scene. We then add the dark current to the image. We save the result, which is now effectively a `blc_tmp` file. If the data are post-flashed, we also need to reverse the flash correction. We do this by multiplying the flash file by the flash duration, running it through a Poisson sampler, and adding it to the 47 Tuc image. The lines for this are commented out below. **Note: It is not recommended to use a simulated exposure time that scales pixels in the dark or flash image to or above the full well depth of ~80,000 electrons.**" ] }, { @@ -507,7 +532,7 @@ "metadata": {}, "source": [ "\n", - "### 6. Run CTE forward model\n", + "### 6A. Run CTE forward model\n", "\n", "We are now ready to run the CTE forward model, which simulates the effects of CTE losses while reading out the detector. In this example, we will use the `acstools` module `acscteforwardmodel`. Note that this step may take a few minutes. The resulting filename will be `*_ctefmod.fits`. We rename the file to have the suffix `*_blv_tmp.fits` so that it can be processed by `CALACS` in a later step." ] @@ -518,8 +543,15 @@ "metadata": {}, "outputs": [], "source": [ - "acscteforwardmodel.acscteforwardmodel('jd0q14ctq_stars_pfl_dkc.fits')\n", - "\n", + "acscteforwardmodel.acscteforwardmodel('jd0q14ctq_stars_pfl_dkc.fits')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ "os.rename('jd0q14ctq_stars_pfl_dkc_ctefmod.fits', 'jd0q14ctq_stars_ctefmod_blv_tmp.fits')" ] }, @@ -527,13 +559,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "After the forward model is run, the SCI extensions of the image are equivalent to a `blv_tmp` file, in principle. However, the ERR extensions are the original `flc` ERR extensions plus 10% of the forward model correction. To ensure the ERR extensions are accurate to a `blv_tmp` file, we will set every pixel to zero and calculate new values for each pixel according to\n", + "After the forward model is run, the `SCI` extensions of the image are equivalent to a `blv_tmp` file, in principle. However, the `ERR` extensions are the original `flc` `ERR` extensions plus 10% of the forward model correction. To ensure the `ERR` extensions are accurate for a `blv_tmp` file, we will set every pixel to zero and calculate new values for each pixel according to\n", "\n", "$\\mathrm{ERR} = \\sqrt{\\mathrm{SCI} + \\mathrm{RN}^2 + (\\mathrm{ERR}_{\\mathrm{superbias}}g)^2}$,\n", "\n", - "where $\\mathrm{SCI}$ is the pixel value in the SCI extension (all negative pixels are set to zero), $\\mathrm{RN}$ is the readnoise, $\\mathrm{ERR}_{\\mathrm{superbias}}$ is the pixel value in the ERR extension of the superbias, and $g$ is the gain.\n", + "where $\\mathrm{SCI}$ is the pixel value in the `SCI` extension (all negative pixels are set to zero), $\\mathrm{RN}$ is the readnoise, $\\mathrm{ERR}_{\\mathrm{superbias}}$ is the pixel value in the `ERR` extension of the superbias, and $g$ is the gain.\n", "\n", - "First, we access the header and SCI and ERR extensions of the forward-modeled data." + "First, we access the header and `SCI` and `ERR` extensions of the forward-modeled data." ] }, { @@ -581,7 +613,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Finally, we make copies of the SCI extensions in which to set all negative values to zero. We calculate the appropriate error for each quadrant and save them to the ERR extensions of the forward-modeled image." + "Finally, we make copies of the `SCI` extensions in which to set all negative values to zero. We calculate the appropriate error for each quadrant and save them to the `ERR` extensions of the forward-modeled image." ] }, { @@ -619,7 +651,7 @@ "metadata": {}, "source": [ "\n", - "### (Optional) 7. Run CTE correction\n", + "### (Optional) 7A. Run CTE correction\n", "\n", "If desired, we now CTE correct the forward-modeled image. To do this, we need to update the `PCTECORR` keyword to `PERFORM` again and update the `NEXTEND` keyword to 6, the number of extensions left after running the forward model. (This is because the forward model strips the distortion-related extensions from the input file, but does not update `NEXTEND`.) Finally, run `acscte` on the image. The resulting filename will be `*_blc_tmp.fits`." ] @@ -641,7 +673,7 @@ "metadata": {}, "source": [ "\n", - "### 8. Apply flat and dark correction\n", + "### 8A. Apply flat and dark correction\n", "\n", "Finally, we flat-field and dark-correct the forward-modeled image using `acs2d` to produce an `flt`-like image. First, we update the keywords `DARKCORR` and `FLATCORR` to `PERFORM`. All other relevant `CALACS` header keyword switches are set to `COMPLETE` because the original data was an `flc` image. The resulting filename will be `*_flt.fits`. **Note:** If the data were post-flashed, and the flash background was added back in during an earlier step, we must also set `FLSHCORR` equal to `PERFORM`. This option is shown below in a commented out line." ] @@ -663,7 +695,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "If the forward-modeled image was CTE-corrected in [Step 7](#A7), we run `acs2d` on the CTE-corrected image. The resulting filename will be `*_flc.fits`." + "If the forward-modeled image was CTE-corrected in [Step 7A](#A7), we run `acs2d` on the CTE-corrected image. The resulting filename will be `*_flc.fits`." ] }, { @@ -683,7 +715,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The 47 Tuc image(s) are now prepared for further analysis appropriate for the user's science. The cells below plot a portion of the final images." + "The 47 Tuc image(s) are now prepared for further analysis appropriate for the user's science. The cells below plot a portion of the final images, the `flt` and, if produced, the `flc`." ] }, { @@ -698,9 +730,10 @@ "\n", "fig, ax = plt.subplots(1,1, figsize=(9,4))\n", "\n", - "ax.imshow(flt_stars, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', origin='lower')\n", - "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, markerfacecolor='none', markeredgecolor='red',\n", - " linestyle='none')\n", + "ax.imshow(flt_stars, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', \n", + " origin='lower')\n", + "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, \n", + " markerfacecolor='none', markeredgecolor='red', linestyle='none')\n", "\n", "ax.set_xlim(2000, 2800)\n", "ax.set_ylim(1200, 1700)" @@ -718,9 +751,10 @@ "\n", "fig, ax = plt.subplots(1,1, figsize=(9,4))\n", "\n", - "ax.imshow(flc_stars, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', origin='lower')\n", - "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, markerfacecolor='none', markeredgecolor='red',\n", - " linestyle='none')\n", + "ax.imshow(flc_stars, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', \n", + " origin='lower')\n", + "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, \n", + " markerfacecolor='none', markeredgecolor='red', linestyle='none')\n", "\n", "ax.set_xlim(2000, 2800)\n", "ax.set_ylim(1200, 1700)" @@ -740,10 +774,10 @@ "\n", "### Option B: Start with a synthetic image\n", "\n", - "We start with the `raw` file from the `jd0q14ctq` dataset, and process the `SCI` extensions to make a completely synthetic dataset.\n", + "In this Option, we start with the `raw` file from the `jd0q14ctq` dataset and process the `SCI` extensions to make a completely synthetic dataset. We then process it appropriately for use in the forward model.\n", "\n", "\n", - "### 4. Create a synthetic image\n", + "### 4B. Create a synthetic image\n", "\n", "We will create an image that is equivalent to a `blc_tmp` file. This means that the image is not flat-fielded and includes sky background, Poisson noise from the sky, artificial stars or other sources, Poisson noise from the sources, dark current, and dark noise.\n", "\n", @@ -819,9 +853,10 @@ "\n", "fig, ax = plt.subplots(1,1, figsize=(9,4))\n", "\n", - "ax.imshow(synth, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', origin='lower')\n", - "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, markerfacecolor='none', markeredgecolor='red',\n", - " linestyle='none')\n", + "ax.imshow(synth, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', \n", + " origin='lower')\n", + "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, \n", + " markerfacecolor='none', markeredgecolor='red', linestyle='none')\n", "\n", "ax.set_xlim(2000, 2800)\n", "ax.set_ylim(1200, 1700)" @@ -857,9 +892,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Next, obtain the `SCI` extensions of both CCDs. We then multiply by the flat and scale the CTE-corrected dark by a chosen exposure time to simulate. We also run the scaled dark image through a Poisson sampler to include Poisson noise in the dark scene. We then add the dark current to the image. If post-flash is desired, multiply the flash reference file by the flash duration, run it through a Poisson sampler, and add to the synthetic data. This is shown in the commented out lines below. We save the result, which is now effectively a `blc_tmp` file. **Note: It is not recommended to use a simulated exposure time that scales pixels in the dark or flash image to or above the full well depth of ~80,000 electrons.**\n", + "Next, obtain the `SCI` extensions of both CCDs. We then multiply by the flat and scale the CTE-corrected dark by a chosen exposure time. We also run the scaled dark image through a Poisson sampler to include Poisson noise in the dark scene. We then add the dark current to the image. If post-flash is desired, multiply the flash reference file by the flash duration, run it through a Poisson sampler, and add to the synthetic data. This is shown in the commented out lines below. We save the result, which is now effectively a `blc_tmp` file. **Note: It is not recommended to use a simulated exposure time that scales pixels in the dark or flash image to or above the full well depth of ~80,000 electrons.**\n", "\n", - "Note that these reference files are specific to the [anneal cycle](http://www.stsci.edu/hst/acs/documents/handbooks/current/c04_detector4.html#415503) in which these data were taken. If an observation date other than that listed in the `DATE-OBS` header keyword is desired for the synthetic data, different reference files will be needed. These can be found by updating the `DATE-OBS` header keyword in the synthetic image to the desired observation date, and rerunning the cell in [Section 2]() which uses `CRDS bestrefs` to download the correct reference files." + "Note that these reference files are specific to the [anneal cycle](http://www.stsci.edu/hst/acs/documents/handbooks/current/c04_detector4.html#415503) in which these data were taken. If an observation date other than that listed in the `DATE-OBS` header keyword is desired for the synthetic data, different reference files will be needed. These can be found by updating the `DATE-OBS` header keyword in the synthetic image to the desired observation date, and rerunning the cell in [Step 2](#download) which uses `CRDS bestrefs` to download the correct reference files." ] }, { @@ -895,7 +930,7 @@ "metadata": {}, "source": [ "\n", - "### 5. Run CTE forward model\n", + "### 5B. Run CTE forward model\n", "\n", "We are now ready to run the CTE forward model, which simulates the effects of CTE losses while reading out the detector. In this example, we will use the `acstools` module `acscteforwardmodel`. Note that this step may take a few minutes. The resulting filename will be `*_ctefmod.fits`. We rename this to `*_blv_tmp.fits` in order to ensure the correct behavior from `acs2d` in a later step." ] @@ -915,7 +950,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "At this point, we also add readnoise to the forward-modeled SCI extensions to complete the readout simulation. We find the readnoise values for each quadrant of the image from the header keywords `READNSEA`, `READNSEB`, etc. We make a noise image for each quadrant, concatenate quadrants A and B and quadrants C and D, and add them to the synthetic image." + "At this point, we also add readnoise to the forward-modeled `SCI` extensions to complete the readout simulation. We find the readnoise values for each quadrant of the image from the header keywords `READNSEA`, `READNSEB`, etc. We make a noise image for each quadrant, concatenate quadrants A and B and quadrants C and D, and add them to the synthetic image." ] }, { @@ -929,10 +964,14 @@ "rn_C = hdr['READNSEC']\n", "rn_D = hdr['READNSED']\n", "\n", - "img_rn_A = datasets.make_noise_image((shape[0], int(shape[1]/2)), type='gaussian', mean=0., stddev=rn_A)\n", - "img_rn_B = datasets.make_noise_image((shape[0], int(shape[1]/2)), type='gaussian', mean=0., stddev=rn_B)\n", - "img_rn_C = datasets.make_noise_image((shape[0], int(shape[1]/2)), type='gaussian', mean=0., stddev=rn_C)\n", - "img_rn_D = datasets.make_noise_image((shape[0], int(shape[1]/2)), type='gaussian', mean=0., stddev=rn_D)\n", + "img_rn_A = datasets.make_noise_image((shape[0], int(shape[1]/2)), type='gaussian', \n", + " mean=0., stddev=rn_A)\n", + "img_rn_B = datasets.make_noise_image((shape[0], int(shape[1]/2)), type='gaussian', \n", + " mean=0., stddev=rn_B)\n", + "img_rn_C = datasets.make_noise_image((shape[0], int(shape[1]/2)), type='gaussian', \n", + " mean=0., stddev=rn_C)\n", + "img_rn_D = datasets.make_noise_image((shape[0], int(shape[1]/2)), type='gaussian', \n", + " mean=0., stddev=rn_D)\n", "\n", "wfc1_rn = np.hstack((img_rn_A, img_rn_B))\n", "wfc2_rn = np.hstack((img_rn_C, img_rn_D))" @@ -959,13 +998,13 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The SCI extensions of this image are now equivalent to a `blv_tmp` file, in principle. However, the ERR extensions are the original `blv_tmp` ERR extensions plus 10% of the forward model correction. To ensure the ERR extensions are accurate, we will calculate new values for each pixel according to\n", + "The `SCI` extensions of this image are now equivalent to a `blv_tmp` file, in principle. However, the `ERR` extensions are the original `blv_tmp` `ERR` extensions plus 10% of the forward model correction. To ensure the `ERR` extensions are accurate, we will calculate new values for each pixel according to\n", "\n", "$\\mathrm{ERR} = \\sqrt{\\mathrm{SCI} + \\mathrm{RN}^2 + (\\mathrm{ERR}_{\\mathrm{superbias}}g)^2}$,\n", "\n", - "where $\\mathrm{SCI}$ is the pixel value in the SCI extension (all negative pixels are set to zero), $\\mathrm{RN}$ is the readnoise, $\\mathrm{ERR}_{\\mathrm{superbias}}$ is the pixel value in the ERR extension of the superbias, and $g$ is the gain.\n", + "where $\\mathrm{SCI}$ is the pixel value in the `SCI` extension (all negative pixels are set to zero), $\\mathrm{RN}$ is the readnoise, $\\mathrm{ERR}_{\\mathrm{superbias}}$ is the pixel value in the `ERR` extension of the superbias, and $g$ is the gain.\n", "\n", - "First, we access the header and SCI and ERR extensions of the forward-modeled data." + "First, we access the header and `SCI` and `ERR` extensions of the forward-modeled data." ] }, { @@ -1013,7 +1052,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Finally, we make copies of the SCI extensions in which to set all negative values to zero. We calculate the appropriate error for each quadrant and save them to the ERR extensions of the forward-modeled image." + "Finally, we make copies of the `SCI` extensions in which to set all negative values to zero. We calculate the appropriate error for each quadrant and save them to the `ERR` extensions of the forward-modeled image." ] }, { @@ -1028,8 +1067,9 @@ "sci_wfc1_pos[sci_wfc1_pos < 0] = 0\n", "sci_wfc2_pos[sci_wfc2_pos < 0] = 0\n", "\n", - "# The superbias ERR arrays contain 20 rows of virtual overscan at the edge of each CCD furthest from the amplifier\n", - "# and 24 columns of physical prescan on the left and right edges.\n", + "# The superbias ERR arrays contain 20 rows of virtual overscan at the edge of \n", + "# each CCD furthest from the amplifier and 24 columns of physical prescan on the\n", + "# left and right edges.\n", "err_A = np.sqrt(sci_wfc1_pos[:,:2048] + rn_A**2 + (err_bias_wfc1[20:,24:2072]*gain_A)**2)\n", "err_B = np.sqrt(sci_wfc1_pos[:,2048:] + rn_B**2 + (err_bias_wfc1[20:,2072:-24]*gain_B)**2)\n", "err_C = np.sqrt(sci_wfc2_pos[:,:2048] + rn_C**2 + (err_bias_wfc2[:-20,24:2072]*gain_C)**2)\n", @@ -1045,7 +1085,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We also repopulate the DQ extensions because they reflect the processing of the original `blv_tmp` SCI extensions. To do this, we reprocess the data with `DQICORR` and `SINKCORR`, both within `acsccd`, and add in the DQ extensions of the appropriate superbias file. First, we rename the synthetic image to have a filename `*_raw.fits`, or `acsccd` will fail. Then, we update the `DQICORR` and `SINKCORR` header keywords to `PERFORM` and run `acsccd`. The output will be `*_blv_tmp.fits` again." + "We also repopulate the DQ extensions because they reflect the processing of the original `blv_tmp` `SCI` extensions. To do this, we reprocess the data with `DQICORR` and `SINKCORR`, both within `acsccd`, and add in the DQ extensions of the appropriate superbias file. First, we rename the synthetic image to have a filename `*_raw.fits`, or `acsccd` will fail. Then, we update the `DQICORR` and `SINKCORR` header keywords to `PERFORM` and run `acsccd`. The output will be `*_blv_tmp.fits` again." ] }, { @@ -1066,7 +1106,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Next, obtain the DQ extensions of the superbias file, and add them to the new DQ extensions of the synthetic data with a bitwise OR operator." + "Next, obtain the DQ extensions of the superbias file, and add them to the new DQ extensions of the synthetic data with a `bitwise_or` operator." ] }, { @@ -1098,7 +1138,7 @@ "metadata": {}, "source": [ "\n", - "### 6. (Optional) Run CTE correction\n", + "### 6B. (Optional) Run CTE correction\n", "\n", "If desired, we now CTE correct the forward-modeled image. To do this, we need to update the `PCTECORR` keyword to `PERFORM` again, and run `acscte` on the image. The resulting filename will be `*_blc_tmp.fits`." ] @@ -1119,7 +1159,7 @@ "metadata": {}, "source": [ "\n", - "### 7. Apply flat and dark correction\n", + "### 7B. Apply flat and dark correction\n", "\n", "Finally, we flat-field and dark-correct the forward-modeled image using `acs2d` to produce an `flt`-like image. We first ensure that the `DARKCORR`, `FLATCORR`, and if necessary, `FLSHCORR` header keywords are set to `PERFORM`. The resulting filename will be `*_flt.fits`." ] @@ -1161,7 +1201,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "The 47 Tuc image(s) are now prepared for further analysis appropriate for the user's science. The cells below plot a portion of the final images." + "The image(s) are now prepared for further analysis appropriate for the user's science. The cells below plot a portion of the final images, the `flt` and, if produced, the `flc`." ] }, { @@ -1174,9 +1214,10 @@ "\n", "fig, ax = plt.subplots(1,1, figsize=(9,4))\n", "\n", - "ax.imshow(synth_ctefmod, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', origin='lower')\n", - "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, markerfacecolor='none', markeredgecolor='red',\n", - " linestyle='none')\n", + "ax.imshow(synth_ctefmod, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', \n", + " origin='lower')\n", + "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, \n", + " markerfacecolor='none', markeredgecolor='red', linestyle='none')\n", "\n", "ax.set_xlim(2000, 2800)\n", "ax.set_ylim(1200, 1700)" @@ -1192,14 +1233,26 @@ "\n", "fig, ax = plt.subplots(1,1, figsize=(9,4))\n", "\n", - "ax.imshow(synth_ctefmod_flc, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', origin='lower')\n", - "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, markerfacecolor='none', markeredgecolor='red',\n", - " linestyle='none')\n", + "ax.imshow(synth_ctefmod_flc, vmin=0, vmax=200, interpolation='nearest', cmap='Greys_r', \n", + " origin='lower')\n", + "ax.plot(sources['x_mean'], sources['y_mean'], marker='o', markersize=20, \n", + " markerfacecolor='none', markeredgecolor='red', linestyle='none')\n", "\n", "ax.set_xlim(2000, 2800)\n", "ax.set_ylim(1200, 1700)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### For more help:\n", + "\n", + "More details may be found on the [ACS website](http://www.stsci.edu/hst/acs) and in the [ACS Instrument](http://www.stsci.edu/hst/acs/documents/handbooks/current/cover.html) and [Data Handbooks](http://www.stsci.edu/hst/acs/documents/handbooks/currentDHB/acs_cover.html).\n", + "\n", + "Please visit the [HST Help Desk](hsthelp.stsci.edu). Through the help desk portal, you can explore the HST Knowledge Base and request additional help from experts." + ] + }, { "cell_type": "markdown", "metadata": {}, @@ -1214,12 +1267,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "[Top of Page](#title_ID)\n", - "\"stsci_pri_combo_mark_horizonal_white_bkgd\" " + "[Top of Page](#title_ID)" ] } ], "metadata": { + "anaconda-cloud": {}, "kernelspec": { "display_name": "Python [conda env:astroconda]", "language": "python",