From f27687c2cbe58dd19e4e4bbe3fff4ea3dcb9fb15 Mon Sep 17 00:00:00 2001 From: Bonnarel <52417996+Bonnarel@users.noreply.github.com> Date: Wed, 24 Feb 2021 17:50:18 +0100 Subject: [PATCH 01/10] Update SODA.tex RElated to issue #4 this Commit introduces a couple of subsections for new rebinning / reprojecting parameters in SODA --- SODA.tex | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/SODA.tex b/SODA.tex index 164920e..f34c7d6 100755 --- a/SODA.tex +++ b/SODA.tex @@ -654,6 +654,47 @@ \subsubsection{POL} The UCD describing the POL parameter is \ucd{meta.code;phys.polarization}. +\subsubsection{SPATRES} + +Implementation of this parameter is optional. Using a SPATRES value different from the original spatial resolution makes the service to rebin the data in order to match the spatres value. The parameter unit is arcsec. + +Example : SPATRES = 3 + +This statement resquests a response dataset with spatial resolution equal to 3 arcsec. + +\subsubsection{SPECRP} + +Implementation of this parameter is optional. Using a SPECREC value different from the original spectral resolving power makes the service to rebin the data in order to match the specrp resolution power value. The parameter is uniless. + +Example : SPECRP = 1000 + +This statement resquests a response dataset with spectral resolving power equal to 1000. + +\subsubsection{TIMERES} + +Implementation of this parameter is optional. Using a TIMERES value different from the original time resolution makes the service to rebin the data in order to match the timeres value. The parameter unit is seconds. + +Example : TIMERES = 0.001 + +This statement resquests a response dataset with time resolution equal to 1 ms. + +\subsubsection{PROJECTION} + +Implementation of this parameter is optional. Using a PROJECTION value different from the original sky projection of the data makes the service repoject the data onto the requested sky projection. The possible values of this parameter is similar to the list of WCS projections and expressed with the 3 lettre WCS projection code. + +Example PROJECTION = SIN + +This statement requests a response dataset reprojected using the sinus sky projection. + +\subsubsection{ROTATION} + +Implementation of this parameter is optional. Using a ROTATION value different from the original rotation of the vertical axis of the data with respect to the North makes the service rotate the data in order to match the requested position angle. The parameter unit is in degree and is in the range -180 to +180. + +Example ROTATION = 50.0 + +This statement requests a response dataset rotated 50 degrees towards East. + + \subsection{Filtering parameters and ObsCore data model} From f019f3721f418356265c7dc36f15aa283fe8cea7 Mon Sep 17 00:00:00 2001 From: BONNAREL FRANCOIS Date: Thu, 9 Nov 2023 18:08:23 +0100 Subject: [PATCH 02/10] reintroduce WCS parameters --- SODA.tex | 3 +++ 1 file changed, 3 insertions(+) diff --git a/SODA.tex b/SODA.tex index 7916766..4f4e25a 100644 --- a/SODA.tex +++ b/SODA.tex @@ -669,6 +669,7 @@ \subsubsection{POL} \subsubsection{SPATRES} + Implementation of this parameter is optional. Using a SPATRES value different from the original spatial resolution makes the service to rebin the data in order to match the spatres value. The parameter unit is arcsec. Example : SPATRES = 3 @@ -709,6 +710,8 @@ \subsubsection{ROTATION} + + \subsection{Filtering parameters and ObsCore data model} From ecf104a830f0bc5fc48ba30824221d96bbfae984 Mon Sep 17 00:00:00 2001 From: BONNAREL FRANCOIS Date: Fri, 10 Nov 2023 05:49:42 +0100 Subject: [PATCH 03/10] Add FORMAT DPTYPE PIXELS METADATA --- Makefile | 4 ++-- SODA.tex | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 68 insertions(+), 5 deletions(-) diff --git a/Makefile b/Makefile index dae4024..282bed6 100644 --- a/Makefile +++ b/Makefile @@ -4,10 +4,10 @@ DOCNAME = SODA # count up; you probably do not want to bother with versions <1.0 -DOCVERSION = 1.0 +DOCVERSION = 1.1 # Publication date, ISO format; update manually for "releases" -DOCDATE = 2017-05-17 +DOCDATE = 2023-11-09 # What is it you're writing: NOTE, WD, PR, or REC DOCTYPE = REC diff --git a/SODA.tex b/SODA.tex index 4f4e25a..cc6d77b 100644 --- a/SODA.tex +++ b/SODA.tex @@ -412,6 +412,52 @@ \subsubsection{ID} The UCD describing the ID parameter is \ucd{meta.ref.url;meta.curation}.SODA +\subsubsection{RESPONSEFORMAT} +\label{sec:FORMAT} + +This optional parameter allows format conversion. By default the native format is kept. Values taken by this parameter will be the media types of the target format. \\ +Examples: +\begin{itemize} +\item Conversion of a FITS image to png: +\begin{lstlisting} +RESPONSEFORMAT="image/png" +\end{lstlisting} +\item Conversion of one or several FITS images or cubes to HiPS format. +\begin{lstlisting} +RESPONSEFORMAT="application/HiPS" +\end{lstlisting} +\item And the reverse conversion: +\begin{lstlisting} +RESPONSEFORMAT="application/fits" +\end{lstlisting} +\end{itemize} + +The format conversion can be done in combination with the filtering or the transformation described below. When a conversion is not made available by the service or is impossible the service MUST answer by an error message stating "Format transformation not available". By default there will be no format conversion. + +\subsubsection{METADATA} +\label{sec:METADATA} +This optional parameter allows to provide dataset metadata instead of an excerpt of the dataset itself. It allows to retrieve the metadata consistent with a specific datamodel given by its standardID or a FITS header for the "headder" value. All SODA service MUST be able to provide at least the Obscore metadata. Other datamodels are possible. + +This parameter can be used in combination with filtering or transformation parameters in such a way that the metadata describe the filtered or transformed dataset. + + The "none" value for this parameter will drive the excerpt of data as in version 1.0 of the SODA specification. The default value fo this parameter is "None". + + Example - for a FITS header : +\begin{lstlisting} +METADATA="header" +\end{lstlisting} + + +\subsubsection{DPTYPE} +\label{sec:DPTYPE} +This optional parameter allows to modify the product type of the output dataset. Typical case is to reduce a spectral subcube to a 2D image or a spectrum. DPTYPE allowed values are taken from the dataproduct\_type vocabulary. This parameter is binded to the filtering parameters which define the ranges and area in which to sum up the dataset measurements to achive the dimensionality reduction. + +Example - for a spectrum: +\begin{lstlisting} +DPTYPE="spectrum"&POS=POLYGON 50.5 27.2 50.7 27.2 50.7 27.6 50.5 27.6 +\end{lstlisting} + + \subsection{Filtering Parameters} Filtering parameters are used to extract subsets of large @@ -542,15 +588,29 @@ \subsubsection{POLYGON} Example: a polygon from (12,34) to (14,34) to (14,36) to (12,36) and (implicitly) back to (12,34): -\begin{lstlisting} -POLYGON=12 34 14 34 14 36 12 36 -\end{lstlisting} The UCD describing the POLYGON parameter is \ucd{pos.outline;obs}. POLYGON is equivalent in functionality to \texttt{POS=POLYGON ...}. Data type and unit metadata are unambiguously defined. +\subsubsection{PIXELS} +\label{sec:PIXELS} +The PIXELS parameter defines a subpart of the dataset to be extracted following the fitsio syntax. \\ +Examples : + +\begin{itemize} +\item pixels 50 to 70 in x and 200 to 300 in y from a 2D image +\begin{lstlisting} +PIXELS=[50:70,200:300] +\end{lstlisting} + +\item pixels 50 to 70 in x, 200 to 300 in y and all pixels in z from a 3D cube +\begin{lstlisting} +PIXELS=[50:70,200:300,*] +\end{lstlisting} + +\end{itemize} \subsubsection{BAND} \label{sec:BAND} @@ -667,6 +727,9 @@ \subsubsection{POL} The UCD describing the POL parameter is \ucd{meta.code;phys.polarization}. +\subsection{Transform parameters} + +This subsection is an addition of the current specification. The parameters described here were not present in SODA version 1.0. they define together a sODA service capability which is optional for a SODA service. \subsubsection{SPATRES} From 6d1be525e66849752a3b84902e160137e2edc7bd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fran=C3=A7ois=20Bonnarel?= <52417996+Bonnarel@users.noreply.github.com> Date: Fri, 15 Mar 2024 17:58:58 +0100 Subject: [PATCH 04/10] Update SODA.tex adding sub-sampling by picking up "1 upon n" pixels in the area of interest --- SODA.tex | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/SODA.tex b/SODA.tex index cc6d77b..e0b01d5 100644 --- a/SODA.tex +++ b/SODA.tex @@ -596,7 +596,8 @@ \subsubsection{POLYGON} \subsubsection{PIXELS} \label{sec:PIXELS} -The PIXELS parameter defines a subpart of the dataset to be extracted following the fitsio syntax. \\ +The PIXELS parameter defines a subpart of the dataset to be extracted following the fitsio syntax. +It is also possible to extract only every pixel upon n.\\ Examples : \begin{itemize} @@ -610,8 +611,14 @@ \subsubsection{PIXELS} PIXELS=[50:70,200:300,*] \end{lstlisting} +\item every even pixel from 50 to 70 in x, 200 to 300 in y and every pixel upon 10 between 100 and 1500 in z from a 3D cube +\begin{lstlisting} +PIXELS=[50:70:2,200:300:2,100:1500:10] +\end{lstlisting} + \end{itemize} + \subsubsection{BAND} \label{sec:BAND} From 755777b6e3a6e69cee788af7bfc879e3d2a1f952 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fran=C3=A7ois=20Bonnarel?= <52417996+Bonnarel@users.noreply.github.com> Date: Fri, 15 Mar 2024 18:24:25 +0100 Subject: [PATCH 05/10] Update Makefile --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 282bed6..9cc66f7 100644 --- a/Makefile +++ b/Makefile @@ -7,7 +7,7 @@ DOCNAME = SODA DOCVERSION = 1.1 # Publication date, ISO format; update manually for "releases" -DOCDATE = 2023-11-09 +DOCDATE = 2024-03-15 # What is it you're writing: NOTE, WD, PR, or REC DOCTYPE = REC From 024e27bcf2d608721f497211fc4f19d3d0de3a70 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fran=C3=A7ois=20Bonnarel?= <52417996+Bonnarel@users.noreply.github.com> Date: Fri, 15 Mar 2024 18:24:47 +0100 Subject: [PATCH 06/10] Update Makefile --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index 9cc66f7..ae0b7b7 100644 --- a/Makefile +++ b/Makefile @@ -10,7 +10,7 @@ DOCVERSION = 1.1 DOCDATE = 2024-03-15 # What is it you're writing: NOTE, WD, PR, or REC -DOCTYPE = REC +DOCTYPE = WD AUTHOR_EMAIL=francois.bonnarel@astro.unistra.fr From c640fe500aacdf86251b9416549f0aa9245e2826 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fran=C3=A7ois=20Bonnarel?= <52417996+Bonnarel@users.noreply.github.com> Date: Fri, 15 Mar 2024 18:56:53 +0100 Subject: [PATCH 07/10] Update SODA.tex moving DPTYPE subsection to Transform and reorder the filtering section according to Pat's comments --- SODA.tex | 90 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 48 insertions(+), 42 deletions(-) diff --git a/SODA.tex b/SODA.tex index e0b01d5..4eb175d 100644 --- a/SODA.tex +++ b/SODA.tex @@ -448,14 +448,7 @@ \subsubsection{METADATA} \end{lstlisting} -\subsubsection{DPTYPE} -\label{sec:DPTYPE} -This optional parameter allows to modify the product type of the output dataset. Typical case is to reduce a spectral subcube to a 2D image or a spectrum. DPTYPE allowed values are taken from the dataproduct\_type vocabulary. This parameter is binded to the filtering parameters which define the ranges and area in which to sum up the dataset measurements to achive the dimensionality reduction. -Example - for a spectrum: -\begin{lstlisting} -DPTYPE="spectrum"&POS=POLYGON 50.5 27.2 50.7 27.2 50.7 27.6 50.5 27.6 -\end{lstlisting} \subsection{Filtering Parameters} @@ -463,19 +456,7 @@ \subsection{Filtering Parameters} Filtering parameters are used to extract subsets of large datasets or data files. The extraction uses a best possible match to the requested subset. In case the parameter values excede the size of the archived dataset the service operates a reduction of these values to the archived dataset size. -\subsubsection{MOC} -The MOC parameter defines a subset of space using the \xtype{moc} defined in DALI. The parameter syntax is defined as in the MOC specification \citep{MOC2} -Examples : -\begin{itemize} - -\item Extracting cells 1 and 2 at order 1 will read this way MOC = 1/1 2 -\item Extracting cells 1 at order 1 and cells 1 to 6 at order 2 will read MOC = 1/1 2/1-6 - -\end{itemize} - - -At the time of writing there is no possibility to extend this ascii syntax to TMOC or STMOC \subsubsection{POS} \label{sec:POS} @@ -562,6 +543,20 @@ \subsubsection{POS} service descriptor; either POS is included or not and if included then all keywords must be supported. +\subsubsection{MOC} +The MOC parameter defines a subset of space using the \xtype{moc} defined in DALI. The parameter syntax is defined as in the MOC specification \citep{MOC2} + +Examples : +\begin{itemize} + +\item Extracting cells 1 and 2 at order 1 will read this way MOC = 1/1 2 +\item Extracting cells 1 at order 1 and cells 1 to 6 at order 2 will read MOC = 1/1 2/1-6 + +\end{itemize} + + +At the time of writing there is no possibility to extend this ascii syntax to TMOC or STMOC + \subsubsection{CIRCLE} \label{sec:CIRCLE} @@ -594,29 +589,6 @@ \subsubsection{POLYGON} POLYGON is equivalent in functionality to \texttt{POS=POLYGON ...}. Data type and unit metadata are unambiguously defined. -\subsubsection{PIXELS} -\label{sec:PIXELS} -The PIXELS parameter defines a subpart of the dataset to be extracted following the fitsio syntax. -It is also possible to extract only every pixel upon n.\\ -Examples : - -\begin{itemize} -\item pixels 50 to 70 in x and 200 to 300 in y from a 2D image -\begin{lstlisting} -PIXELS=[50:70,200:300] -\end{lstlisting} - -\item pixels 50 to 70 in x, 200 to 300 in y and all pixels in z from a 3D cube -\begin{lstlisting} -PIXELS=[50:70,200:300,*] -\end{lstlisting} - -\item every even pixel from 50 to 70 in x, 200 to 300 in y and every pixel upon 10 between 100 and 1500 in z from a 3D cube -\begin{lstlisting} -PIXELS=[50:70:2,200:300:2,100:1500:10] -\end{lstlisting} - -\end{itemize} \subsubsection{BAND} @@ -734,9 +706,43 @@ \subsubsection{POL} The UCD describing the POL parameter is \ucd{meta.code;phys.polarization}. +\subsubsection{PIXELS} +\label{sec:PIXELS} +The PIXELS parameter defines a subpart of the dataset to be extracted following the fitsio syntax. +It is also possible to extract only every pixel upon n.\\ +Examples : + +\begin{itemize} +\item pixels 50 to 70 in x and 200 to 300 in y from a 2D image +\begin{lstlisting} +PIXELS=[50:70,200:300] +\end{lstlisting} + +\item pixels 50 to 70 in x, 200 to 300 in y and all pixels in z from a 3D cube +\begin{lstlisting} +PIXELS=[50:70,200:300,*] +\end{lstlisting} + +\item every even pixel from 50 to 70 in x, 200 to 300 in y and every pixel upon 10 between 100 and 1500 in z from a 3D cube +\begin{lstlisting} +PIXELS=[50:70:2,200:300:2,100:1500:10] +\end{lstlisting} + +\end{itemize} + \subsection{Transform parameters} This subsection is an addition of the current specification. The parameters described here were not present in SODA version 1.0. they define together a sODA service capability which is optional for a SODA service. + +\subsubsection{DPTYPE} +\label{sec:DPTYPE} +This optional parameter allows to modify the product type of the output dataset. Typical case is to reduce a spectral subcube to a 2D image or a spectrum. DPTYPE allowed values are taken from the dataproduct\_type vocabulary. This parameter is binded to the filtering parameters which define the ranges and area in which to sum up the dataset measurements to achive the dimensionality reduction. + +Example - for a spectrum: +\begin{lstlisting} +DPTYPE="spectrum"&POS=POLYGON 50.5 27.2 50.7 27.2 50.7 27.6 50.5 27.6 +\end{lstlisting} + \subsubsection{SPATRES} From 1b071751517799ff0ce91d7f9984ea5ad799e779 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fran=C3=A7ois=20Bonnarel?= <52417996+Bonnarel@users.noreply.github.com> Date: Wed, 22 May 2024 23:04:37 +0200 Subject: [PATCH 08/10] Update Makefile Working draft 1.1 first version --- Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Makefile b/Makefile index ae0b7b7..24b9a11 100644 --- a/Makefile +++ b/Makefile @@ -7,7 +7,7 @@ DOCNAME = SODA DOCVERSION = 1.1 # Publication date, ISO format; update manually for "releases" -DOCDATE = 2024-03-15 +DOCDATE = 2024-05-23 # What is it you're writing: NOTE, WD, PR, or REC DOCTYPE = WD From d53939d751a99dc0bc2e615c5f3fbe4ebee313de Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fran=C3=A7ois=20Bonnarel?= <52417996+Bonnarel@users.noreply.github.com> Date: Wed, 22 May 2024 23:20:27 +0200 Subject: [PATCH 09/10] Update SODA.tex clarifying behavior for DPTYPE parameter --- SODA.tex | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/SODA.tex b/SODA.tex index 4eb175d..aaa540b 100644 --- a/SODA.tex +++ b/SODA.tex @@ -736,13 +736,20 @@ \subsection{Transform parameters} \subsubsection{DPTYPE} \label{sec:DPTYPE} -This optional parameter allows to modify the product type of the output dataset. Typical case is to reduce a spectral subcube to a 2D image or a spectrum. DPTYPE allowed values are taken from the dataproduct\_type vocabulary. This parameter is binded to the filtering parameters which define the ranges and area in which to sum up the dataset measurements to achive the dimensionality reduction. - -Example - for a spectrum: +This optional parameter allows to modify the product type of the output dataset. Typical case is to reduce a spectral subcube to a 2D image or a spectrum. DPTYPE allowed values are taken from the dataproduct\_type vocabulary. This parameter is binded to the filtering parameters which define the ranges and area in which to sum up the dataset measurements to achieve the dimensionality reduction. \begin{lstlisting} DPTYPE="spectrum"&POS=POLYGON 50.5 27.2 50.7 27.2 50.7 27.6 50.5 27.6 \end{lstlisting} +For a cube, SODA operation above will sum up all spectra inside the spatial coverage, while the same query parameters with DPTYPE=cube (which is the default) will produce a cube spatial cutout. In the case of dimensionality reduction the corresponding resolution transform parameter MUST be ignored. + +\begin{lstlisting} +DPTYPE="image"&BAND=0.1 0.4 +\end{lstlisting} +For a cube, SODA operation above will sum up all channels inside the spectral bounds, while the same query parameters with DPTYPE=cube will produce a cube spectral cutout. + +In the case of dimensionality reduction the corresponding appropriate resolution transform parameter (SPATRES,SPECRP) MUST be ignored. + \subsubsection{SPATRES} From ae01f3d5e7ffa959ba87ab3a9b63d65326473c1f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Fran=C3=A7ois=20Bonnarel?= <52417996+Bonnarel@users.noreply.github.com> Date: Wed, 22 May 2024 23:40:38 +0200 Subject: [PATCH 10/10] Update SODA.tex - PIXELS and parameter multiplicity - RESPONSEFORMAT = application/hips specificities --- SODA.tex | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/SODA.tex b/SODA.tex index aaa540b..f317703 100644 --- a/SODA.tex +++ b/SODA.tex @@ -424,7 +424,7 @@ \subsubsection{RESPONSEFORMAT} \end{lstlisting} \item Conversion of one or several FITS images or cubes to HiPS format. \begin{lstlisting} -RESPONSEFORMAT="application/HiPS" +RESPONSEFORMAT="application/hips" \end{lstlisting} \item And the reverse conversion: \begin{lstlisting} @@ -434,8 +434,9 @@ \subsubsection{RESPONSEFORMAT} The format conversion can be done in combination with the filtering or the transformation described below. When a conversion is not made available by the service or is impossible the service MUST answer by an error message stating "Format transformation not available". By default there will be no format conversion. -\subsubsection{METADATA} -\label{sec:METADATA} +Conversion to the hips format requires the transformation from any sky projection to HPX. Any other provided PROJECTION value MUST be ignored. In the same conversion to hips case, the SODA response provides the URL to the top of the HiPS directory. + +\subsubsection{METADATA}\label{sec:METADATA} This optional parameter allows to provide dataset metadata instead of an excerpt of the dataset itself. It allows to retrieve the metadata consistent with a specific datamodel given by its standardID or a FITS header for the "headder" value. All SODA service MUST be able to provide at least the Obscore metadata. Other datamodels are possible. This parameter can be used in combination with filtering or transformation parameters in such a way that the metadata describe the filtered or transformed dataset. @@ -709,7 +710,8 @@ \subsubsection{POL} \subsubsection{PIXELS} \label{sec:PIXELS} The PIXELS parameter defines a subpart of the dataset to be extracted following the fitsio syntax. -It is also possible to extract only every pixel upon n.\\ +It is also possible to extract only every pixel upon n. Supporting PIXELS in the same query with spatial, spectral or time filtering +parameters is optional and SHOULD be interpreted within the scope of PARAMETER multiplicity\\ Examples : \begin{itemize}