diff --git a/notebooks/niriss_imaging/niriss-imaging-tutorial.ipynb b/notebooks/niriss_imaging/niriss-imaging-tutorial.ipynb
new file mode 100755
index 000000000..7e75ffe19
--- /dev/null
+++ b/notebooks/niriss_imaging/niriss-imaging-tutorial.ipynb
@@ -0,0 +1,825 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "0393e357-9d9d-4516-b28d-4d335fad33a0",
+ "metadata": {},
+ "source": [
+ "# Advanced: NIRISS Imaging Tutorial Notebook\n",
+ "\n",
+ "## Table of Contents\n",
+ "1. [Introduction](#intro)\n",
+ "2. [Examining uncalibrated data products](#examine)\n",
+ "3. [Stage 1 Processing](#det1)\n",
+ "4. [Stage 2 Processing](#image2)\n",
+ "5. [Stage 3 Processing](#image3)\n",
+ "6. [Visualize Detected Sources](#detections)\n",
+ "\n",
+ "Date published: January 24, 2024"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dc6b2aba-072e-4179-a856-29d547f277d6",
+ "metadata": {},
+ "source": [
+ "## 1. Introduction\n",
+ "In this notebook, we will process a [NIRISS imaging dataset](https://jwst-docs.stsci.edu/jwst-near-infrared-imager-and-slitless-spectrograph/niriss-observing-modes/niriss-imaging) through the [JWST calibration pipeline](https://jwst-docs.stsci.edu/jwst-science-calibration-pipeline-overview). The example dataset is from [Program ID](https://www.stsci.edu/jwst/science-execution/program-information) 1475 (PI: Boyer, CoI: Volk) which is a sky flat calibration program. NIRCam is used as the primary instrument with NIRISS as a [coordinated parallel instrument](https://jwst-docs.stsci.edu/methods-and-roadmaps/jwst-parallel-observations/jwst-coordinated-parallels-roadmap). The NIRISS imaging dataset uses a 17-step dither pattern.\n",
+ "\n",
+ "For illustrative purposes, we focus on data taken through the NIRISS [F150W filter](https://jwst-docs.stsci.edu/jwst-near-infrared-imager-and-slitless-spectrograph/niriss-instrumentation/niriss-filters) and start with uncalibrated data products. The files are named\n",
+ "`jw01475006001_02201_000nn_nis_uncal.fits`, where *nn* refers to the dither step number which ranges from 01 - 17. See the [\"File Naming Schemes\" documentation](https://jwst-pipeline.readthedocs.io/en/stable/jwst/data_products/file_naming.html) to learn more about the file naming convention.\n",
+ "\n",
+ "In this notebook, we assume all uncalibrated fits files are saved in a directory named `1475_f150w`."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "026a649d-a0d2-4e3f-ab6d-cf8c26e6ce1d",
+ "metadata": {},
+ "source": [
+ "### Install pipeline and dependencies"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "9c5b415a-f0f4-4c48-a038-d319e825ee4f",
+ "metadata": {},
+ "source": [
+ "To make sure that the pipeline version is compatabile with the steps discussed below and the required dependencies and packages are installed, you can create a fresh conda environment and install the provided `requirements.txt` file:\n",
+ "```\n",
+ "conda create -n niriss_imaging_pipeline python=3.11\n",
+ "conda activate niriss_imaging_pipeline\n",
+ "pip install -r requirements.txt\n",
+ "```\n",
+ "\n",
+ "### Imports"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "fb9f7f54-ff98-428b-9fa9-b39c50210c3a",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import os\n",
+ "import glob\n",
+ "import zipfile\n",
+ "import urllib.request\n",
+ "import numpy as np\n",
+ "\n",
+ "# For visualizing images\n",
+ "from jdaviz import Imviz\n",
+ "\n",
+ "# Astropy routines for visualizing detected sources:\n",
+ "from astropy.table import Table\n",
+ "from astropy.coordinates import SkyCoord\n",
+ "\n",
+ "# Configure CRDS\n",
+ "os.environ[\"CRDS_PATH\"] = 'crds_cache'\n",
+ "os.environ[\"CRDS_SERVER_URL\"] = \"https://jwst-crds.stsci.edu\"\n",
+ "\n",
+ "# for JWST calibration pipeline\n",
+ "import jwst\n",
+ "from jwst import datamodels\n",
+ "from jwst.pipeline import Detector1Pipeline, Image2Pipeline, Image3Pipeline\n",
+ "from jwst.associations import asn_from_list\n",
+ "from jwst.associations.lib.rules_level3_base import DMS_Level3_Base\n",
+ "\n",
+ "# To confirm which version of the pipeline you're running:\n",
+ "print(f\"jwst pipeline version: {jwst.__version__}\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "459d06b6-4f6a-402c-85a8-d06715a3d9e5",
+ "metadata": {},
+ "source": [
+ "### Download uncalibrated data products"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "8d595f1e-2590-4f01-bb92-13bb43a22b3c",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# APT program ID number:\n",
+ "pid = '01475'\n",
+ "\n",
+ "# Set up directory to download uncalibrated data files:\n",
+ "data_dir = '1475_f150w/'\n",
+ "# Create directory if it does not exist\n",
+ "if not os.path.isdir(data_dir):\n",
+ " os.mkdir(data_dir)\n",
+ "\n",
+ "# Download uncalibrated data from Box into current directory:\n",
+ "boxlink = 'https://data.science.stsci.edu/redirect/JWST/jwst-data_analysis_tools/niriss_imaging/1475_f150w.zip'\n",
+ "boxfile = os.path.join(data_dir, '1475_f150w.zip')\n",
+ "urllib.request.urlretrieve(boxlink, boxfile)\n",
+ "\n",
+ "zf = zipfile.ZipFile(boxfile, 'r')\n",
+ "zf.extractall()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "881f784e-8df2-4012-9927-7aa19cdd8f67",
+ "metadata": {},
+ "source": [
+ "***\n",
+ "## 2. Examining uncalibrated data products"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1ee9487b-a642-482f-a645-c8f8c2f8fb1b",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "uncal_files = sorted(glob.glob(os.path.join(data_dir, '*_uncal.fits')))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "dca6413e-6a91-40c1-9bfa-0ebf78e6431b",
+ "metadata": {},
+ "source": [
+ "### Look at the first file to determine exposure parameters and practice using JWST datamodels"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "f767a7fa-50cb-4411-8aef-a32b7fde8917",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# print file name\n",
+ "print(uncal_files[0])\n",
+ "\n",
+ "# Open file as JWST datamodel\n",
+ "examine = datamodels.open(uncal_files[0])\n",
+ "\n",
+ "# Print out exposure info\n",
+ "print(\"Instrument: \" + examine.meta.instrument.name)\n",
+ "print(\"Filter: \" + examine.meta.instrument.filter)\n",
+ "print(\"Pupil: \" + examine.meta.instrument.pupil)\n",
+ "print(\"Number of integrations: {}\".format(examine.meta.exposure.nints))\n",
+ "print(\"Number of groups: {}\".format(examine.meta.exposure.ngroups))\n",
+ "print(\"Readout pattern: \" + examine.meta.exposure.readpatt)\n",
+ "print(\"Dither position number: {}\".format(examine.meta.dither.position_number))\n",
+ "print(\"\\n\")"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0469a648-9bb1-40f9-9f8b-fb3a29bc5c7f",
+ "metadata": {},
+ "source": [
+ "From the above, we confirm that the data file is for the NIRISS instrument using the `F150W` filter in the [Pupil Wheel](https://jwst-docs.stsci.edu/jwst-near-infrared-imager-and-slitless-spectrograph/niriss-instrumentation/niriss-pupil-and-filter-wheels) crossed with the `CLEAR` filter in the Filter Wheel. This observation uses the [`NIS` readout pattern](https://jwst-docs.stsci.edu/jwst-near-infrared-imager-and-slitless-spectrograph/niriss-instrumentation/niriss-detector-overview/niriss-detector-readout-patterns), 16 groups per integration, and 1 integration per exposure. This data file is the 1st dither position in this exposure sequence. For more information about how JWST exposures are defined by up-the-ramp sampling, see the [Understanding Exposure Times JDox article](https://jwst-docs.stsci.edu/understanding-exposure-times).\n",
+ "\n",
+ "This metadata will be the same for all exposures in this observation other than the dither position number."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "46be5421-7fc8-4d96-b2cd-2484f294e082",
+ "metadata": {},
+ "source": [
+ "### Display uncalibrated image\n",
+ "\n",
+ "We can visualize an uncalibrated dataset that will show detector artifacts that will be removed when calibrating the data through the `DETECTOR1` stage of the pipeline. Uncalibrated data files thus are 4D: nintegrations x ngroups x nrows x ncols. Here, we are visualizing the full detector (i.e., all columns and rows) and the 1st group.\n",
+ "\n",
+ "We are using the [Imviz tool](https://jdaviz.readthedocs.io/en/latest/imviz/index.html) within the `jdaviz` package to visualize the NIRISS image."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1db43de4-896e-4eb4-8e92-aecf67eed5be",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create an Imviz instance and set up default viewer\n",
+ "imviz_uncal = Imviz()\n",
+ "viewer_uncal = imviz_uncal.default_viewer\n",
+ "\n",
+ "# Read in the science array for our visualization dataset:\n",
+ "uncal_science = examine.data\n",
+ "\n",
+ "# Load the dataset into Imviz\n",
+ "imviz_uncal.load_data(uncal_science[0, 0, :, :])\n",
+ "\n",
+ "# Visualize the dataset:\n",
+ "imviz_uncal.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b2d33e40-64b1-4d24-9a72-af586ccddf0c",
+ "metadata": {},
+ "source": [
+ "Adjust settings for the viewer:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "00479015-d39a-493e-9c0f-300823901bc1",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "viewer_uncal.stretch = 'sqrt'\n",
+ "viewer_uncal.set_colormap('Viridis')\n",
+ "viewer_uncal.cuts = '99.5%'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "768c0cdb-72a3-4b2f-90f8-161f52b6d954",
+ "metadata": {},
+ "source": [
+ "***\n",
+ "## 3. Stage 1 Processing \n",
+ "Run the datasets through the [Detector1 stage of the pipeline](https://jwst-docs.stsci.edu/jwst-science-calibration-pipeline-overview/stages-of-jwst-data-processing/calwebb_detector1) to apply detector level calibrations and create a countrate data product where slopes are fitted to the integration ramps. These `*_rate.fits` products are 2D (nrows x ncols), averaged over all integrations. 3D countrate data products (`*_rateints.fits`) are also created (nintegrations x nrows x ncols) which have the fitted ramp slopes for each integration.\n",
+ "\n",
+ "The [pipeline documentation](https://jwst-pipeline.readthedocs.io/en/latest/jwst/user_documentation/running_pipeline_python.html) discusses how to run the pipeline in the Python Interface, including how to configure pipeline steps and override reference files. By returning the results of the pipeline to a variable, the pipeline returns a datamodel. Note that the [`pipeline.call()` method is preferred over the `pipeline.run()` method](https://jwst-pipeline.readthedocs.io/en/latest/jwst/user_documentation/running_pipeline_python.html#advanced-use-pipeline-run-vs-pipeline-call).\n",
+ "\n",
+ "By default, [all steps in the Detector1 stage of the pipeline](https://jwst-pipeline.readthedocs.io/en/latest/jwst/pipeline/calwebb_detector1.html) are run for NIRISS except: the `ipc` correction step and the `gain_scale` step. Note that while the [`persistence` step](https://jwst-pipeline.readthedocs.io/en/latest/jwst/persistence/description.html) is set to run by default, this step does not automatically correct the science data for persistence. The `persistence` step creates a `*_trapsfilled.fits` file which is a model that records the number of traps filled at each pixel at the end of an exposure. This file would be used as an input to the `persistence` step, via the `input_trapsfilled` argument, to correct a science exposure for persistence. Since persistence is not well calibrated for NIRISS, we do not perform a persistence correction and thus turn off this step to speed up calibration and to not create files that will not be used in the subsequent analysis. This step can be turned off when running the pipeline in Python by doing:\n",
+ "```\n",
+ "rate_result = Detector1Pipeline.call(uncal,\n",
+ " steps={'persistence': {'skip': True}})\n",
+ "```\n",
+ "\n",
+ "The [charge_migration step](https://jwst-pipeline.readthedocs.io/en/latest/jwst/charge_migration/index.html#charge-migration-step) is particularly important for NIRISS images to mitigate apparent flux loss in resampled images due to the spilling of charge from a central pixel into its neighboring pixels (see [Goudfrooij et al. 2023](https://ui.adsabs.harvard.edu/abs/2023arXiv231116301G/abstract) for details). Charge migration occurs when the accumulated charge in a central pixel exceeds a certain signal limit, which is ~25,000 ADU. This step is turned on by default for NIRISS imaging, Wide Field Slitless Spectroscopy [(WFSS)](https://jwst-docs.stsci.edu/jwst-near-infrared-imager-and-slitless-spectrograph/niriss-observing-modes/niriss-wide-field-slitless-spectroscopy), and Aperture Masking Interferometry [(AMI)](https://jwst-docs.stsci.edu/jwst-near-infrared-imager-and-slitless-spectrograph/niriss-observing-modes/niriss-aperture-masking-interferometry) modes when using CRDS contexts of `jwst_1159.pmap` or later. Different signal limits for each filter are provided by the [pars-chargemigrationstep parameter files](https://jwst-crds.stsci.edu). Users can specify a different signal limit by running this step with the `signal_threshold` flag and entering another signal limit in units of ADU.\n",
+ "\n",
+ "As of CRDS context `jwst_1155.pmap` and later, the [`jump` step](https://jwst-pipeline.readthedocs.io/en/latest/api/jwst.jump.JumpStep.html) of the `DETECTOR1` stage of the pipeline will remove residuals associated with [snowballs](https://jwst-docs.stsci.edu/data-artifacts-and-features/snowballs-and-shower-artifacts) for NIRISS imaging, WFSS, and AMI modes. The default parameters for this correction, where `expand_large_events` set to `True` turns on the snowball halo removal algorithm, are specified in the `pars-jumpstep` parameter reference files. Users may wish to alter parameters to optimize removal of snowball residuals. Available parameters are discussed in the [Detection and Flagging of Showers and Snowballs in JWST Technical Report (Regan 2023)](https://www.stsci.edu/files/live/sites/www/files/home/jwst/documentation/technical-documents/_documents/JWST-STScI-008545.pdf)."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "0f8f107b-5f80-4674-bc34-2d88791e4409",
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "# Define directory to save output from detector1\n",
+ "det1_dir = 'detector1/'\n",
+ "# Create directory if it does not exist\n",
+ "if not os.path.isdir(det1_dir):\n",
+ " os.mkdir(det1_dir)\n",
+ "\n",
+ "# Run Detector1 stage of pipeline, specifying:\n",
+ "# output directory to save *_rate.fits files\n",
+ "# save_results flag set to True so the rate files are saved\n",
+ "# skipping the persistence step\n",
+ " \n",
+ "for uncal in uncal_files:\n",
+ " rate_result = Detector1Pipeline.call(uncal,\n",
+ " output_dir=det1_dir,\n",
+ " save_results=True,\n",
+ " steps={'persistence': {'skip': True}})"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7fc5520a-6097-482b-b3e0-a6bf94cf7af3",
+ "metadata": {},
+ "source": [
+ "### Identify `*_rate.fits` files"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "26c5dca4-2390-4919-a7e6-9fae755aeda6",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "rate_files = sorted(glob.glob(os.path.join(det1_dir, '*_rate.fits')))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "f5845ca6-5280-489b-a1da-c2267857d034",
+ "metadata": {},
+ "source": [
+ "### Verify which pipeline steps were run and which calibration reference files were applied\n",
+ "\n",
+ "The header contains information about which calibration steps were completed and skipped and which reference files were used to process the data."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1c122766-a545-4042-93e8-f68c18f62fdf",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Read in file as datamodel\n",
+ "rate_f = datamodels.open(rate_files[0])\n",
+ "\n",
+ "rate_f.meta.cal_step.instance"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "da6698a5-d1ce-4043-80a1-8b9ff35108b3",
+ "metadata": {},
+ "source": [
+ "Check which reference files were used to calibrate the dataset:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "5f1430b7-9106-4e13-b668-394cdcc53420",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "rate_f.meta.ref_file.instance"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "3ff813f4-f9d6-4765-b882-48f954b8969d",
+ "metadata": {},
+ "source": [
+ "### Display rate image\n",
+ "\n",
+ "Visualize a countrate image, using the dataset from the first dither position as an example."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "176e27f5-f59f-4a6c-960b-4d688288eae3",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create an Imviz instance and set up default viewer\n",
+ "imviz_rate = Imviz()\n",
+ "viewer_rate = imviz_rate.default_viewer\n",
+ "\n",
+ "# Read in the science array for our visualization dataset:\n",
+ "rate_science = rate_f.data\n",
+ "\n",
+ "# Load the dataset into Imviz\n",
+ "imviz_rate.load_data(rate_science)\n",
+ "\n",
+ "# Visualize the dataset:\n",
+ "imviz_rate.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "fb6f488c-eaef-4b96-9284-07f0e1d13f6b",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "viewer_rate.stretch = 'sqrt'\n",
+ "viewer_rate.set_colormap('Viridis')\n",
+ "viewer_rate.cuts = '95%'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "350808ee-9579-4cb5-8f02-a74904c91449",
+ "metadata": {},
+ "source": [
+ "***\n",
+ "## 3. Stage 2 Processing \n",
+ "\n",
+ "In the [Image2 stage of the pipeline](https://jwst-pipeline.readthedocs.io/en/latest/jwst/pipeline/calwebb_image2.html), calibrated unrectified data products are created (`*_cal.fits` or `*_calints.fits` files, depending on whether the input files are `*_rate.fits` or `*_rateints.fits`). \n",
+ "\n",
+ "In this pipeline processing stage, the [world coordinate system (WCS)](https://jwst-pipeline.readthedocs.io/en/latest/jwst/assign_wcs/index.html#assign-wcs-step) is assigned, the data are [flat fielded](https://jwst-pipeline.readthedocs.io/en/latest/jwst/flatfield/index.html#flatfield-step), and a [photometric calibration](https://jwst-pipeline.readthedocs.io/en/latest/jwst/photom/index.html#photom-step) is applied to convert from units of countrate (ADU/s) to surface brightness (MJy/sr). \n",
+ "\n",
+ "By default, the [background subtraction step](https://jwst-pipeline.readthedocs.io/en/latest/jwst/background_step/index.html#background-step) and the [resampling step](https://jwst-pipeline.readthedocs.io/en/latest/jwst/resample/index.html#resample-step) are turned off for NIRISS at this stage of the pipeline. The background subtraction is turned off since there is no background template for the imaging mode and the local background is removed during the background correction for photometric measurements around individual sources. The resampling step occurs during the `Image3` stage by default. While the resampling step can be turned on during the `Image2` stage to, e.g., generate a source catalog for each image, the data quality from the `Image3` stage will be better since the bad pixels, which adversely affect both the centroids and photometry in individual images, will be mostly removed."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "81d916e5-dd5d-472d-9aff-b2b4c86decf8",
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "# Define directory to save output from Image2\n",
+ "image2_dir = 'image2/'\n",
+ "# Create directory if it does not exist\n",
+ "if not os.path.isdir(image2_dir):\n",
+ " os.mkdir(image2_dir)\n",
+ "\n",
+ "# Run Image2 stage of pipeline, specifying:\n",
+ "# output directory to save *_cal.fits files\n",
+ "# save_results flag set to True so the rate files are saved\n",
+ "\n",
+ "for rate in rate_files:\n",
+ " cal_result = Image2Pipeline.call(rate,\n",
+ " output_dir=image2_dir,\n",
+ " save_results=True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "75b26730-40b9-4ea7-a6ad-47727f00e401",
+ "metadata": {},
+ "source": [
+ "### Identify `*_cal.fits` files"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e80bb4d0-858f-4779-b927-c32332f7a670",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "cal_files = sorted(glob.glob(os.path.join(image2_dir, '*_cal.fits')))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "da57b7d0-95ef-4227-b730-6f54a3eaaa16",
+ "metadata": {},
+ "source": [
+ "### Verify which pipeline steps were run"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "27390bbd-ae2d-49e7-977a-59d3765c0946",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "cal_f = datamodels.open(cal_files[0])\n",
+ "\n",
+ "cal_f.meta.cal_step.instance"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "0bd56084-72bb-4491-acd3-c8faeabca331",
+ "metadata": {},
+ "source": [
+ "Check which reference files were used to calibrate the dataset:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3a8aced8-773b-4f8b-86f4-2c62d3b3bcc2",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "cal_f.meta.ref_file.instance"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "19b9d7e4-536c-4661-ac9a-d21e7daf1389",
+ "metadata": {},
+ "source": [
+ "### Display cal image\n",
+ "\n",
+ "Visualize a calibrated image, using the dataset from the first dither position as an example."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "15455eda-74b5-453b-b5c9-e21b48b73446",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create an Imviz instance and set up default viewer\n",
+ "imviz_cal = Imviz()\n",
+ "viewer_cal = imviz_cal.default_viewer\n",
+ "\n",
+ "# Read in the science array for our visualization dataset:\n",
+ "cal_science = cal_f.data\n",
+ "\n",
+ "# Load the dataset into Imviz\n",
+ "imviz_cal.load_data(cal_science)\n",
+ "\n",
+ "# Visualize the dataset:\n",
+ "imviz_cal.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "2b89947f-c9c6-4eee-a4e1-0f880d28bf31",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "viewer_cal.stretch = 'sqrt'\n",
+ "viewer_cal.set_colormap('Viridis')\n",
+ "viewer_cal.cuts = '95%'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "7cfbb244-7af9-4cbf-90d9-08598378c16a",
+ "metadata": {},
+ "source": [
+ "***\n",
+ "## 3. Stage 3 Processing \n",
+ "\n",
+ "In the [Image3 stage of the pipeline](https://jwst-pipeline.readthedocs.io/en/latest/jwst/pipeline/calwebb_image3.html), the individual `*_cal.fits` files for each of the dither positions are combined to one single distortion corrected image. First, an [Association](https://jwst-pipeline.readthedocs.io/en/latest/jwst/associations/overview.html) needs to be created to inform the pipeline that these individual exposures are linked together. \n",
+ "\n",
+ "By default, the `Image3` stage of the pipeline performs the following steps on NIRISS data: \n",
+ "- [tweakreg](https://jwst-pipeline.readthedocs.io/en/latest/jwst/tweakreg/index.html#tweakreg-step) - creates source catalogs of pointlike sources for each input image. The source catalog for each input image is compared to each other to derive coordinate transforms to align the images relative to each other.\n",
+ " - As of CRDS context `jwst_1156.pmap` and later, the `pars-tweakreg` parameter reference file for NIRISS performs an absolute astrometric correction to GAIA data release 3 by default (i.e., the `abs_refcat` parameter is set to `GAIADR3`). Though this default correction generally improves results compared with not doing this alignment, it can sometimes result in poor performance in crowded or sparse fields, so users are encouraged to check astrometric accuracy and revisit this step if necessary.\n",
+ " - As of pipeline version 1.12.5, the default source finding algorithm is `DAOStarFinder` which can result in up to 0.5 pix uncertainties in the centroids for undersampled PSFs, like the NIRISS PSFs at short wavelengths [(Goudfrooij 2022)](https://www.stsci.edu/files/live/sites/www/files/home/jwst/documentation/technical-documents/_documents/JWST-STScI-008116.pdf). There are plans to update the default algorithm to `IRAFStarFinder` in future pipeline versions.\n",
+ "- [skymatch](https://jwst-pipeline.readthedocs.io/en/latest/jwst/skymatch/index.html#skymatch-step) - measures the background level from the sky to use as input into the subsequent `outlier detection` and `resample` steps.\n",
+ "- [outlier detection](https://jwst-pipeline.readthedocs.io/en/latest/jwst/outlier_detection/index.html#outlier-detection-step) - flags any remaining cosmic rays, bad pixels, or other artifacts not already flagged during the `DETECTOR1` stage of the pipeline, using all input images to create a median image so that outliers in individual images can be identified.\n",
+ "- [resample](https://jwst-pipeline.readthedocs.io/en/latest/jwst/resample/index.html#resample-step) - resamples each input image based on its WCS and distortion information and creates a single undistorted image.\n",
+ "- [source catalog](https://jwst-pipeline.readthedocs.io/en/latest/jwst/source_catalog/index.html#source-catalog-step) - creates a catalog of detected sources along with measured photometries and morphologies (i.e., point-like vs extended). Useful for quicklooks, but optimization is likely needed for specific science cases, which is an on-going investigation for the NIRISS team. Users may wish to experiment with changing the `snr_threshold` and `deblend` options. Modifications to the following parameters will not significantly improve data quality and it is advised to keep them at their default values: `aperture_ee1`, `aperture_ee2`, `aperture_ee3`, `ci1_star_threshold`, `ci2_star_threshold`.\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "db5cb6f5-44a1-4f28-95dc-b23a97451465",
+ "metadata": {},
+ "source": [
+ "### Create Association File\n",
+ "\n",
+ "An association file lists the exposures to calibrated together in `Stage 3` of the pipeline. Note that an association file is available for download from MAST, with a filename of `*_asn.json`. Here we show how to create an association file to point to the data products created when processing data through the pipeline. Note that the output products will have a rootname that is specified by the `product_name` in the association file. For this tutorial, the rootname of the output products will be `image3_association`."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1b31f80f-8d7b-45ae-8537-15d0208637df",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create a Level 3 Association\n",
+ "associations = asn_from_list.asn_from_list(cal_files, \n",
+ " rule=DMS_Level3_Base, \n",
+ " product_name='image3_association')\n",
+ "\n",
+ "associations.data['asn_type'] = 'image3'\n",
+ "associations.data['program'] = pid\n",
+ "\n",
+ "# Format association as .json file\n",
+ "asn_filename, serialized = associations.dump(format=\"json\")\n",
+ "\n",
+ "# Write out association file\n",
+ "with open(asn_filename, \"w\") as fd:\n",
+ " fd.write(serialized)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "629cad5b-e5a7-473e-8583-ad12cda55fa4",
+ "metadata": {},
+ "source": [
+ "### Run Image3 stage of the pipeline\n",
+ "\n",
+ "Given the grouped exposures in the association file, the `Image3` stage of the pipeline will produce:\n",
+ "- a `*_cr.fits` file produced by the `outlier_detection` step, where the `DQ` array marks the pixels flagged as outliers.\n",
+ "- a final combined, rectified image with name `*_i2d.fits`,\n",
+ "- a source catalog with name `*_cat.ecsv`,\n",
+ "- a segmentation map file (`*_segm.fits`) which has integer values at the pixel locations where a source is detected where the pixel values match the source ID number in the catalog."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e0cebfd8-a650-41a3-8ff0-a8fd43e079d6",
+ "metadata": {
+ "scrolled": true
+ },
+ "outputs": [],
+ "source": [
+ "# Define directory to save output from Image2\n",
+ "image3_dir = 'image3/'\n",
+ "# Create directory if it does not exist\n",
+ "if not os.path.isdir(image3_dir):\n",
+ " os.mkdir(image3_dir)\n",
+ "\n",
+ "# Run Stage 3\n",
+ "i2d_result = Image3Pipeline.call(asn_filename, \n",
+ " output_dir=image3_dir,\n",
+ " save_results=True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "4bcb6c40-fbb8-4226-b620-64b0c02c8ceb",
+ "metadata": {},
+ "source": [
+ "### Verify which pipeline steps were run"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "c754530a-d4e2-493b-bf8b-bf8d1ad08770",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Identify *_i2d file and open as datamodel\n",
+ "i2d = glob.glob(os.path.join(image3_dir, \"*_i2d.fits\"))[0]\n",
+ "i2d_f = datamodels.open(i2d)\n",
+ "\n",
+ "i2d_f.meta.cal_step.instance"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "c0a53657-ad44-4430-a7b0-070c6be8fa2c",
+ "metadata": {},
+ "source": [
+ "Check which reference files were used to calibrate the dataset:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "e2ee94ea-dd2b-43d4-ad7d-ea427810df0d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "i2d_f.meta.ref_file.instance"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "8d44a0de-774a-4d63-8de9-80a9c6aa24b2",
+ "metadata": {},
+ "source": [
+ "### Display combined image\n",
+ "\n",
+ "Visualize the drizzle-combined image."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "c6f1ed63-0bed-412b-ab04-54caa7d61bbd",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Create an Imviz instance and set up default viewer\n",
+ "imviz_i2d = Imviz()\n",
+ "viewer_i2d = imviz_i2d.default_viewer\n",
+ "\n",
+ "# Read in the science array for our visualization dataset:\n",
+ "i2d_science = i2d_f.data\n",
+ "\n",
+ "# Load the dataset into Imviz\n",
+ "imviz_i2d.load_data(i2d_science)\n",
+ "\n",
+ "# Visualize the dataset:\n",
+ "imviz_i2d.show()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "8d22a117-e4cf-4d2f-9fec-f52545c8aa5a",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "viewer_i2d.stretch = 'sqrt'\n",
+ "viewer_i2d.set_colormap('Viridis')\n",
+ "viewer_i2d.cuts = '95%'"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "018733eb-07ec-44a4-a5a7-d6d2f27d193a",
+ "metadata": {},
+ "source": [
+ "## Visualize Detected Sources\n",
+ "Using the source catalog created by the `IMAGE3` stage of the pipeline, mark the detected sources, using different markers for point sources and extended sources. The source catalog is saved in `image3/image3_association_cat.ecsv` file. We will need to read in the `i2d` file again to make sure the world coordinate system (WCS) info is read in."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "2ad6ee55-f940-4be3-96c9-114bc83eb784",
+ "metadata": {},
+ "source": [
+ "### Read in catalog file and identify point/extended sources"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "32235fdc-4ec2-42e6-89e7-b9f62824e179",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "catalog_file = glob.glob(os.path.join(image3_dir, \"*_cat.ecsv\"))[0]\n",
+ "catalog = Table.read(catalog_file)\n",
+ "\n",
+ "# To identify point/extended sources, use the 'is_extended' column in the source catalog\n",
+ "pt_src, = np.where(~catalog['is_extended'])\n",
+ "ext_src, = np.where(catalog['is_extended'])\n",
+ "\n",
+ "# Define coordinates of point and extended sources\n",
+ "pt_coord = Table({'coord': [SkyCoord(ra=catalog['sky_centroid'][pt_src].ra,\n",
+ " dec=catalog['sky_centroid'][pt_src].dec)]})\n",
+ "ext_coord = Table({'coord': [SkyCoord(ra=catalog['sky_centroid'][ext_src].ra,\n",
+ " dec=catalog['sky_centroid'][ext_src].dec)]})"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "be9e9069-375d-431b-bbfb-e84e6cefc2d4",
+ "metadata": {},
+ "source": [
+ "### Mark the extended and point sources on the image\n",
+ "\n",
+ "Display combined image:"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "302bc3ce-358d-4ea8-a533-37b9a9a2152c",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Read in i2d file to Imviz\n",
+ "imviz_cat = Imviz()\n",
+ "viewer_cat = imviz_cat.default_viewer\n",
+ "imviz_cat.load_data(i2d)\n",
+ "\n",
+ "# Adjust settings for viewer\n",
+ "viewer_cat.stretch = 'sqrt'\n",
+ "viewer_cat.set_colormap('Viridis')\n",
+ "viewer_cat.cuts = '95%'\n",
+ "\n",
+ "imviz_cat.show()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "bb9dc081-7a24-442c-bf86-ed2faa6cf342",
+ "metadata": {},
+ "source": [
+ "Point sources will be marked by small pink circles and extended sources will be marked by larger white circles."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "fcb8f619-3807-44d1-a8d9-fbf41abb5522",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Add marker for point sources:\n",
+ "viewer_cat.marker = {'color': 'pink', 'markersize': 50, 'fill': False}\n",
+ "\n",
+ "viewer_cat.add_markers(pt_coord, use_skycoord=True, marker_name='point_sources')\n",
+ "\n",
+ "# Add marker for extended sources:\n",
+ "viewer_cat.marker = {'color': 'white', 'markersize': 100, 'fill': False}\n",
+ "\n",
+ "viewer_cat.add_markers(ext_coord, use_skycoord=True, marker_name='extended_sources')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "1c1d9abd-b2d4-4c5d-a5ed-a1ef98e136fd",
+ "metadata": {},
+ "source": [
+ "From zoooming in on different portions of the image, we can see that some saturated point sources are erroneously labeled as point sources, some extended and saturated objects are not detected, parts of diffraction spikes are sometimes detected as \"extended sources\", and in some cases, the detected source centroid is offset from the center of a source. It is recommended for users to optimize source detection for their science goals using either their own tools or by updating parameters to the `source_catalog` step of the pipeline."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/notebooks/niriss_imaging/requirements.txt b/notebooks/niriss_imaging/requirements.txt
new file mode 100644
index 000000000..c6fb7dc62
--- /dev/null
+++ b/notebooks/niriss_imaging/requirements.txt
@@ -0,0 +1,5 @@
+numpy
+crds
+jwst>=1.2.3
+jdaviz
+astropy>=4.3.1
\ No newline at end of file