Skip to content

Commit

Permalink
Cleaned up reduction guide
Browse files Browse the repository at this point in the history
  • Loading branch information
@abhimat
abhimat committed Mar 11, 2022
1 parent 7ea7043 commit 15b1fb1
Show file tree
Hide file tree
Showing 2 changed files with 62 additions and 43 deletions.
105 changes: 62 additions & 43 deletions kai/TheReductionGuide.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@
"\n",
"**This notebook (@author Shrihan Agarwal) is written like it is a replacement for the reduce.py, but it is recommended that you run it using the reduce.py file and terminal.** Follow the setup directories and data as shown below.\n",
" \n",
"The observing log for this example/template data collection (22 June, 2018) can be found at: \n",
"https://docs.google.com/spreadsheets/d/1kyva2I7UGRoNp8cnrpHP5caPNNh-HlAsXTkEUq9DMHg/\n",
"The observing log for this example/template data collection (22 June, 2018) can be found at in this repository [here](./TheReductionGuide_example_observing_log.pdf)\n",
"\n",
"---"
]
Expand All @@ -22,13 +21,15 @@
"source": [
"### Set Up Directories and Data\n",
"\n",
"1. In the main directory, E.g. 20jul14os, set up the necessary directories, i.e.\n",
" * `mkdir reduce`\n",
" * `mkdir clean`\n",
" * `mkdir combo`\n",
" * `ln -s <downloaded raw/ folder>`\n",
" * Eg. `ln -s /g/lu/data/KECK/20200525_OSIRIS/IMAG/raw/` (Symbolic link to location of downloaded /raw file)\n",
" * If you haven't downloaded the data from KECK yet, follow the instructions in the next header first.\n",
"In the main directory, E.g. 20jul14os, set up the necessary directories, i.e.:\n",
"\n",
"* `mkdir reduce`\n",
"* `mkdir clean`\n",
"* `mkdir combo`\n",
"* `ln -s <downloaded raw/ folder>`\n",
" * Eg. `ln -s /g/lu/data/KECK/20200525_OSIRIS/IMAG/raw/` (Symbolic link to location of downloaded /raw file)\n",
" * If you haven't downloaded the data from KECK yet, follow the instructions in the next header first.\n",
" * If working with NIRC2 data, make sure file names are in the `n####.fits` format (i.e. the \"Original File Name\" from the observation). Data downloaded from the KOA may have file names with a different scheme and will need to be renamed to reflect the Original File Name. The Original File Name can be found under the FITS keyword `FILENAME` for each file.\n",
" \n",
"---"
]
Expand All @@ -39,9 +40,7 @@
"source": [
"### Downloading the Data (@raw directory)\n",
"\n",
"In general, if we haven't, we'll need to download the data itself from Keck. The summary is given below. For more details, refer to: \n",
"https://docs.google.com/document/d/1kYLTgnkhu4feJ-YQx4mecdsFx8UJo4jjliZ4d3qXsE8/edit# | Casey's OSIRIS Data Reduction Notes \n",
"https://jluastro.atlassian.net/wiki/spaces/LUGROUP/pages/796819521/Data+Reduction | Data Reduction Page, Confluence\n",
"In general, if we haven't, we'll need to download the data itself from Keck. The summary is given below. Example imaging datasets, from both the NIRC2 and OSIRIS instruments, are also available at this [Google Drive link](https://drive.google.com/drive/folders/1FpTN3wiG4U826H328JIJcPLbScNCTRQW?usp=sharing).\n",
"\n",
"#### Downloading Using SCP/FTP\n",
"\n",
Expand Down Expand Up @@ -109,15 +108,20 @@
" * .fits is the combined/stacked final image after reduction\n",
" * **`starfinder/`**\n",
"\n",
" \n",
"File name components:\n",
"\n",
"* `<filter>` is the name of the filter used. In the example of /20jul14os used, `<filter>` is `kp_tdOpen`.\n",
"* `<framename>`:\n",
" * For OSIRIS, e.g.: `i200714-a011008`, consists of `i`: image, `200714`: date, `a`: random letter, `011`: dataset number, and `008`: frame number.\n",
" * For NIRC2, e.g.: `n0035.fits`, consists of prefix `n` and image number `0035`\n",
"* `<target>` is the name of the target. In the example of /20jul14os, it is kb200101.\n",
"* `<submap number>` is simply a number for the submap indicating which submap it is, e.g. 1 \n",
"\n",
"File types:\n",
"\n",
"`<filter>` is the name of the filter used. In the example of /20jul14os used, `<filter>` is kp_tdOpen. \n",
"`<framename>` Eg. `i200714-a011008`, consists of `i`: image, `200714`: date, `a`: random letter, `011`: dataset number, and `008`: frame number (for OSIRIS). \n",
"`<target>` is the name of the target. In the example of /20jul14os, it is kb200101. \n",
"`<submap number>` is simply a number for the submap indicating which submap it is, Eg. 1 \n",
"`.coo` is the coordinates of the target in x, y of the frame, tells us if the alignment was correct\n",
"`.max` is the saturation limit, after correcting for the cleaning of the data \n",
"`.fits` is the image file.\n",
"* `.coo` are the coordinates of the reference source in x, y of the frame\n",
"* `.max` is the saturation limit, after correcting for the cleaning of the data \n",
"* `.fits` is an image file.\n",
"\n",
"**Note: The naming conventions and files themselves may differ from one date/folder to another, and from OSIRIS to NIRC2 - this is just a general structure.**\n",
"\n",
Expand All @@ -128,47 +132,62 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Necessary Packages"
"### Running reduce.py to reduce data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### kai\n",
"\n",
"If you don't have it already, you would need to ask for access to the kai package. If you have access, ensure that it is installed on your user, and your bash profile has a PYTHONPATH to it. (If you've copied the bash profile from the general startup guide, just install kai at the user/python/code directory as referenced in .bash profile). \n",
"\n",
"You can add kai from the git repository by going to the repository you want to add kai in and performing: \n",
"\n",
"`git init` \n",
"`git clone https://github.com/jluastro/kai`\n",
"\n",
"It's very important you checkout the osiris_img branch. This branch should work for KAI as well.\n",
"\n",
"`cd kai` \n",
"`git checkout osiris_img`"
"Follow KAI installation as detailed [here](https://keck-datareductionpipelines.github.io/KAI/#installation)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Reduction is carried out in the `iraf27` conda environment, activated by the following command\n",
"```bash\n",
"conda activate iraf27\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### pyraf (if not using this notebook to replace reduce.py)\n",
"All reduction steps are run from the `reduce/` directory:\n",
"```bash\n",
"cd reduce/\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Make a reduce.py script to carry out the reduction. You can follow the format of [the template reduce script for NIRC2](./reduce/reduce_template.py) or [the template reduce script for OSIRIS](./reduce/reduce_template_osiris.py)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the `reduce/` directory, the functions in the `reduce.py` script can be run in a python environment, e.g. `ipython`:\n",
"```python\n",
"import reduce\n",
"\n",
"Besides that, the process must be done in an iraf27 environment such that pyraf is present. Dependencies on pyraf are slowly being removed. If you're not using this notebook, you can switch from the normal astroconda environment by typing: \n",
"`conda activate iraf27` \n",
" \n",
"Then, you can open an iPython environment in the reduce directory (`ipython`) and import reduce.py (`import reduce`). Then you can use the functions defined in reduce.py. Keep in mind that you would have to modify this below reduce.py code as well, as explained in the next section. \n",
"reduce.makelog()\n",
"\n",
"---"
"reduce.go()\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### reduce.py"
"### The components of reduce.py"
]
},
{
Expand Down Expand Up @@ -736,7 +755,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
Expand All @@ -750,7 +769,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
"version": "3.7.10"
}
},
"nbformat": 4,
Expand Down
Binary file added kai/TheReductionGuide_example_observing_log.pdf
View file
Binary file not shown.

0 comments on commit 15b1fb1

Please sign in to comment.