From d930a3cc0c4937ec6eb3741b5bbdeffe3d48df94 Mon Sep 17 00:00:00 2001 From: Tuan Do Date: Tue, 2 May 2023 20:59:25 -0700 Subject: [PATCH 1/4] made modifications of cshell scripts to work better with modern linux --- scripts/osirisDropDRF | 18 +++++++++++++++++- scripts/run_odrp | 2 +- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/scripts/osirisDropDRF b/scripts/osirisDropDRF index c079ea7..0b08d0d 100755 --- a/scripts/osirisDropDRF +++ b/scripts/osirisDropDRF @@ -1,3 +1,19 @@ #!/bin/csh -f -cp $1 $DRF_QUEUE_DIR/$2.$1:r.waiting +if ( ! -d "$DRF_QUEUE_DIR" ) then + echo "Error: $DRF_QUEUE_DIR does not exist or is not a directory" + exit 1 +endif + +if ( ! -e "$1" ) then + echo "Error: $1 does not exist or is not a file" + exit 1 +endif + +if ( ! -w "$DRF_QUEUE_DIR" ) then + echo "Error: insufficient permissions to write to $DRF_QUEUE_DIR" + exit 1 +endif + +cp "$1" "$DRF_QUEUE_DIR/$2.$1:r.waiting" + diff --git a/scripts/run_odrp b/scripts/run_odrp index 019f6c8..cecfcd4 100755 --- a/scripts/run_odrp +++ b/scripts/run_odrp @@ -13,7 +13,7 @@ #setenv OSIRIS_BACKBONE_DIR $KROOT/rel/default/idl/odrs/backbone -if ( $?1 & $1 == "-n" ) then +if ( $1 == "-n" ) then onintr - idl -IDL_STARTUP ${OSIRIS_BACKBONE_DIR}/drpStartup.pro ${OSIRIS_BACKBONE_DIR}/osiris_drp_backbone_startup.pro From 9dacf0d12264ec5a0455cf75fd1812602db3045a Mon Sep 17 00:00:00 2001 From: Tuan Do Date: Tue, 2 May 2023 21:17:28 -0700 Subject: [PATCH 2/4] updated README --- README.md | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index e600422..0f30bd5 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,13 @@ * [Testing the Pipeline](#testing-the-pipeline) * [Citing the Pipeline](#citing-the-pipeline) +## Release Notes for v6.0.1 +**2023-05-02** +- Small changes to the ``run_odrp`` and ``osirisDropDRF`` scripts to run better on more modern Linux environments. +- Added more instructions on how to install on Linux +- Added instructions on downloading test data for testing the pipeline + + ## Release Notes for v6.0 **2022-04-12** - Update to the manual including: discussion of new imager, new tables of sensitivities for imager and spectrograph, updates on how to observe with the TRICK NIR TT sensor, discussion of the new exposure time calculator, new discussion of the OSIRIS imager reduction pipeline KAI. @@ -67,13 +74,14 @@ Please cite [Lyke et al. (2017)](https://ui.adsabs.harvard.edu/abs/2017ascl.soft To install and run the OSIRIS DRP, you will need the following: - A working C compiler (e.g. ``gcc``) -- A copy of the compiled library cfitsio +- A copy of the compiled library cfitsio (for Linux, it's best to use the version that can be insalled with your package manager like ``apt``) +- For Linux users: note that while you can use the bash shell and bash environment, some of the scripts require cshell to launch so you will need to install it. Newer versions of linux often lack cshell. To install on Ubuntu use ``sudo apt get csh``. - A working installation of IDL 7 or IDL 8 (the IDL binary directory should be in your ``PATH`` environment variable) -- Python dependencies (optional, for testing): pytest, astropy - If using a computer with Apple M1 ARM chips, see [INSTALLPROBLEMS.md](https://github.com/Keck-DataReductionPipelines/OsirisDRP/blob/master/INSTALLPROBLEMS.md) for a workaround. - ODRFGUI: Java version 17 (newer versions of Java are likely to run into issues when running the GUI) + ### Installing from source Either clone or download the source from github, choose either the master branch or the develop branch. @@ -82,8 +90,8 @@ Either clone or download the source from github, choose either the master branch Set up the following environment variables to compile the code (you can remove these variables after compiling). The defaults should work for installations of IDL on Mac OS X and CFITSIO installed using [MacPorts][]: -- ``IDL_INCLUDE``: The IDL include directory. If you don't set ``IDL_INCLUDE``, it defaults to ``IDL_INCLUDE=/Applications/exelis/idl/external/include`` -- ``CFITSIOLIBDIR``: The directory containing your installation of CFITSIO. If you don't set ``CFITSIOLIBDIR``, it will default to ``CFITSIOLIBDIR=/opt/local/lib``, which is correct for [MacPorts][]. +- ``IDL_INCLUDE``: The IDL include directory. If you don't set ``IDL_INCLUDE``, it defaults to ``IDL_INCLUDE=/Applications/exelis/idl/external/include``. On linux, this might be ``IDL_INCLUDE=/usr/local/harris/idl/external/include`` +- ``CFITSIOLIBDIR``: The directory containing your installation of CFITSIO. If you don't set ``CFITSIOLIBDIR``, it will default to ``CFITSIOLIBDIR=/opt/local/lib``, which is correct for [MacPorts][]. On Ubuntu Linux, you can install ``libcfitsio-dev`` from ``apt``. The library is then located at ``CFITSIOLIBDR=/usr/lib/x86_64-linux-gnu`` Run the makefile from the top level of the OSIRIS DRP source code: @@ -91,7 +99,7 @@ Run the makefile from the top level of the OSIRIS DRP source code: make all ``` -You should see that the pipeline has been built correctly. Be sure you are using ``gmake`` (which on OS X is the only ``make``, so using ``make`` works.) +You should see that the pipeline has been built correctly by seeing that the file ``modules/source/libosiris_drp_ext_null.so.0.0`` exists. [MacPorts]: https://www.macports.org @@ -147,6 +155,13 @@ to open a new xterm window, call ``run_odrp -n``. Please check out the OSIRIS pipeline manual: OSIRIS_Manual_v5.pdf in this directory. +### Testing the Pipeline + +You can run the xml file in ``tests/test_emission_line/emission_obj.xml`` by downloading the following files: +- [s150531_a025001.fits](https://drive.google.com/file/d/0B_YkzZoUSrX-bUtKOXFmalNxQ3M/view?usp=share_link&resourcekey=0-NSjQcVOu5To5S0Jx5QlXHA) - place in ``tests/test_emission_line/`` +- [s150531_a025002.fits](https://drive.google.com/file/d/0B_YkzZoUSrX-bkJGbnc3Zzh3amM/view?usp=share_link&resourcekey=0-6j9yF66ERZxd2xj7LkRQRA) - place in ``tests/test_emission_line/`` +- [s150901_c008___infl_Kn5_035.fits](https://drive.google.com/file/d/0B_YkzZoUSrX-azZsWEV0clVUbnM/view?usp=share_link&resourcekey=0-rcn-lVT8bc4LhUCmYmJWvQ) - rectification matrix to be placed in ``tests/calib/`` + ### OOPGUI & ODRFGUI settings To set the default directories for the guis, you can edit the following two files: From 446964fefb4a966f79a1dd6106a155d4dc081e2b Mon Sep 17 00:00:00 2001 From: Tuan Do Date: Tue, 2 May 2023 21:22:04 -0700 Subject: [PATCH 3/4] incremented verson --- backbone/version.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/backbone/version.txt b/backbone/version.txt index e0ea36f..5fe6072 100644 --- a/backbone/version.txt +++ b/backbone/version.txt @@ -1 +1 @@ -6.0 +6.0.1 From 288cf0bd4d59a4b48a07e7fc2319fc1a8f54a273 Mon Sep 17 00:00:00 2001 From: Tuan Do Date: Fri, 21 Jul 2023 20:57:20 -1000 Subject: [PATCH 4/4] Modified QLook2 so that it can load FITS files without correct image headers without launching a dialog box that must be closed to continue. This helps to support saving all reads in up the ramp mode and visualizing it. --- README.md | 6 ++++++ backbone/version.txt | 2 +- ql2/cimwin__define.pro | 25 +++++++++++++++++-------- 3 files changed, 24 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 0f30bd5..0ac1ae3 100644 --- a/README.md +++ b/README.md @@ -7,6 +7,11 @@ * [Testing the Pipeline](#testing-the-pipeline) * [Citing the Pipeline](#citing-the-pipeline) + +## Release Notes for v6.0.2 +**2023-07-21** +- Modified QLook2 so that it can load FITS files without correct image headers without launching a dialog box that must be closed to continue. This helps to support saving all reads in up the ramp mode and visualizing it. + ## Release Notes for v6.0.1 **2023-05-02** - Small changes to the ``run_odrp`` and ``osirisDropDRF`` scripts to run better on more modern Linux environments. @@ -14,6 +19,7 @@ - Added instructions on downloading test data for testing the pipeline + ## Release Notes for v6.0 **2022-04-12** - Update to the manual including: discussion of new imager, new tables of sensitivities for imager and spectrograph, updates on how to observe with the TRICK NIR TT sensor, discussion of the new exposure time calculator, new discussion of the OSIRIS imager reduction pipeline KAI. diff --git a/backbone/version.txt b/backbone/version.txt index 5fe6072..9b9a244 100644 --- a/backbone/version.txt +++ b/backbone/version.txt @@ -1 +1 @@ -6.0.1 +6.0.2 diff --git a/ql2/cimwin__define.pro b/ql2/cimwin__define.pro index 2c1b37e..c5aaf9c 100644 --- a/ql2/cimwin__define.pro +++ b/ql2/cimwin__define.pro @@ -1723,11 +1723,11 @@ if (itime ne 0) then begin endif else begin - print, ['Could not find the coadds keyword.', $ + print, ['ERROR: Could not find the coadds keyword.', $ 'No manipulation of the data'] endelse endif else begin - print, ['Could not find the integration time keyword.', $ + print, ['ERROR: Could not find the integration time keyword.', $ 'No manipulation of the data'] endelse @@ -2918,14 +2918,23 @@ pro CImWin::DisplayAsDN, no_rescale=no_rescale itime=sxpar(hd, itime_kw, count=itime_count) coadds=sxpar(hd, coadds_kw, count=coadd_count) if itime_count eq 0 then begin - message=['Could not find the coadds keyword.', $ - 'No manipulation of the data'] - answer=dialog_message(message, dialog_parent=self.BaseId, /error) + message=['ERROR: Could not find the coadds keyword.', $ + 'No manipulation of the data'] + ;; commented this out so that it can display images of + ;; individual reads which do not have headers. + ;;answer=dialog_message(message, dialog_parent=self.BaseId, /error) + + ;; instead, print out to the terminal + print, message endif if coadd_count eq 0 then begin - message=['Could not find the integration time keyword.', $ - 'No manipulation of the data'] - answer=dialog_message(message, dialog_parent=self.BaseId, /error) + message=['ERROR Could not find the integration time keyword.', $ + 'No manipulation of the data'] + ;; commented this out so that it can display images of + ;; individual reads which do not have headers. + + ;; answer = dialog_message(message, dialog_parent = self.BaseId, /error) + print, message endif print, "Integration time was ", itime