Merge pull request #154 from tomdonaldson/change-headings

Modify notebook headers to match file names; update index.md

content/reference_notebooks/basic_reference.md

@@ -1,53 +1,31 @@
 ---
 jupytext:
-  notebook_metadata_filter: all
+  notebook_metadata_filter: a
   text_representation:
     extension: .md
     format_name: myst
     format_version: 0.13
-    jupytext_version: 1.14.4
+    jupytext_version: 1.16.0
 kernelspec:
   display_name: Python 3 (ipykernel)
   language: python
   name: python3
-language_info:
-  codemirror_mode:
-    name: ipython
-    version: 3
-  file_extension: .py
-  mimetype: text/x-python
-  name: python
-  nbconvert_exporter: python
-  pygments_lexer: ipython3
-  version: 3.9.9
-toc:
-  base_numbering: 1
-  nav_menu: {}
-  number_sections: false
-  sideBar: true
-  skip_h1_title: false
-  title_cell: Table of Contents
-  title_sidebar: Contents
-  toc_cell: false
-  toc_position: {}
-  toc_section_display: true
-  toc_window_display: true
 ---
 
-# Quick Reference
+# Basic Reference
 
 +++
 
 ## 0. Setup
-Please make sure your environment is set up according to the [instructions here](https://nasa-navo.github.io/navo-workshop/00_SETUP.html).
 
-Ensure you have the latest version of the workshop material by updating your environment:
-TBD
+Please make sure your environment is set up according to the [instructions here](https://nasa-navo.github.io/navo-workshop/00_SETUP.html).
 
 +++
 
 ## 1. Overview
+
 NASA services can be queried from Python in multiple ways.
+
 * Generic Virtual Observatory (VO) queries.
   * Call sequence is consistent, including for non-NASA resources.
   * Use the [`pyvo` package](https://pyvo.readthedocs.io/en/latest/).
@@ -57,7 +35,9 @@ NASA services can be queried from Python in multiple ways.
 * Ad hoc archive-specific interfaces
 
 ## 2. VO Services
+
 This workshop will introduce 4 types of VO queries:
+
 * **VO Registry** - Discover what services are available worldwide
 * **Simple Cone Search** - Search for catalog object within a specified cone region
 * **Simple Image Access** - Search for image products within a spatial region
@@ -84,6 +64,7 @@ warnings.filterwarnings('ignore', '.*Unknown element mirrorURL.*', vo.utils.xml.
 ```
 
 ### 2.1 Look Up Services in VO Registry
+
 Simple example:  Find Simple Cone Search (conesearch) services related to SWIFT.
 
 ```{code-cell} ipython3
@@ -92,15 +73,19 @@ services
 ```
 
 #### 2.1.1 Use different arguments/values to modify the simple example
+
 | Argument | Description | Examples |
 | :-----: | :----------- | :-------- |
 | **servicetype** | Type of service | `conesearch` or `scs` for **Simple Cone Search**<br> `image` or `sia` for **Simple Image Access**<br> `spectrum` or `ssa` for **Simple Spectral Access**<br> `table` or `tap` for **Table Access Protocol**|
 | **keyword** | List of one or more keyword(s) to match service's metadata. Both ORs and ANDs may be specified.<br><ul><li>(OR) A list of keywords match a service if **any** of the keywords match the service.</li><li>(AND) If a  keyword contains multiple space-delimited words, **all** the words must match the metadata.</li></ul>| `['galex', 'swift']` matches 'galex' or 'swift'<br>`['hst survey']` matches services mentioning both 'hst' and 'survey' |
 | **waveband** | Resulting services have data in the specified waveband(s) | ‘radio’, ‘millimeter’, ‘infrared’, ‘optical’, ‘uv’, ‘euv’, ‘x-ray’ ‘gamma-ray’ |
 
-#### 2.1.2 Inspect the results.
+#### 2.1.2 Inspect the results
+
 ##### Using pyvo
+
 Although not lists, `pyvo` results can be iterated over to see each individual result.  The results are specialized based on the type of query, providing access to the important properties of the results.  Some useful accessors with registry results are:
+
 * `short_name` - A short name
 * `res_title` - A more descriptive title
 * `res_description` - A more verbose description
@@ -115,6 +100,7 @@ for s in list(services)[:4]:  # (Treat services as list to get the subset of row
 ```
 
 ##### Filtering results
+
 Of the services we found, which one(s) have 'stsci.edu' in their unique identifier?
 
 ```{code-cell} ipython3
@@ -124,34 +110,38 @@ for s in stsci_services:
 ```
 
 ##### Using astropy
+
 With the `to_table()` method, `pyvo` results can also be converted to Astropy `Table` objects which offer a variety of addional features. See [astropy.table](http://docs.astropy.org/en/stable/table/) for more on working with Astropy Tables.
 
 ```{code-cell} ipython3
 # Convert to an Astropy Table
 services_table = services.to_table()
 
 # Print the column names and display 1st 3 rows with a subset of columns
-print(f'\nColumn Names:\n{services_table.colnames}\n')  
-services_table['short_name', 'res_title', 'res_description'][:3]  
+print(f'\nColumn Names:\n{services_table.colnames}\n')
+services_table['short_name', 'res_title', 'res_description'][:3]
 ```
 
 ### 2.2 Cone search
-Example:  Find a cone search service for the USNO-B catalog and search it around M51 with a .1 degree radius.  (More inspection could be done on the service list instead of blindly choosing the first service.)  
 
-The position (`pos`) is best specified with [SkyCoord](http://docs.astropy.org/en/stable/api/astropy.coordinates.SkyCoord.html) objects. 
+Example:  Find a cone search service for the USNO-B catalog and search it around M51 with a .1 degree radius.  (More inspection could be done on the service list instead of blindly choosing the first service.)
 
-The size of the region is specified with the `radius` keyword and may be decimal degrees or an Astropy [Angle](http://docs.astropy.org/en/stable/api/astropy.coordinates.Angle.html#astropy.coordinates.Angle). 
+The position (`pos`) is best specified with [SkyCoord](http://docs.astropy.org/en/stable/api/astropy.coordinates.SkyCoord.html) objects.
+
+The size of the region is specified with the `radius` keyword and may be decimal degrees or an Astropy [Angle](http://docs.astropy.org/en/stable/api/astropy.coordinates.Angle.html#astropy.coordinates.Angle).
 
 ```{code-cell} ipython3
 m51_pos = SkyCoord.from_name("m51")
 services = vo.regsearch(servicetype='conesearch', keywords='usno-b')
 results = services[0].search(pos=m51_pos, radius=0.1)
 # Astropy Table is useful for displaying cone search results.
-results.to_table()  
+results.to_table()
 ```
 
 ### 2.3 Image search
+
 Example:  Find an image search service for GALEX, and search it around coordinates 13:37:00.950,-29:51:55.51 (M83) with a radius of .2 degrees.  Download the first file in the results.
+
 #### Find an image service
 
 ```{code-cell} ipython3
@@ -160,6 +150,7 @@ services.to_table()['ivoid', 'short_name', 'res_title']
 ```
 
 #### Search one of the services
+
 The first service looks good.  Search it!
 
 For more details on using `SkyCoord`, see [its documentation](http://docs.astropy.org/en/stable/api/astropy.coordinates.SkyCoord.html#astropy.coordinates.SkyCoord)
@@ -175,20 +166,23 @@ results.to_table()
 ```
 
 #### Download an image
+
 For the first result, print the file format and download the file. If repeatedly executing this code, add `cache=True` to `download_file()` to prevent repeated downloads.
 
 See [`download_file()` documentation here.](https://docs.astropy.org/en/stable/api/astropy.utils.data.download_file.html#astropy.utils.data.download_file)
 
 ```{code-cell} ipython3
 print(results[0].format)
-file_name = download_file(results[0].getdataurl())  
+file_name = download_file(results[0].getdataurl())
 file_name
 ```
 
 ### 2.4 Spectral search
+
 Example:  Find a spectral service for x-ray data.  Query it around Delta Ori with a search **diameter** of 10 arc minutes, and download the first data product.  Note that the results table can be inspected for potentially useful columns.
 
 Spectral search is very similar to image search. In this example, note:
+
 * **`diameter`** defines the size of the search region
 * `waveband` used in `regsearch()`
 * Astropy `Angle` used to specify radius units other than degrees.
@@ -198,16 +192,17 @@ Spectral search is very similar to image search. In this example, note:
 services = vo.regsearch(servicetype='spectrum', waveband='x-ray')
 
 # Assuming there are services and the first one is OK...
-results = services[0].search(pos=SkyCoord.from_name("Delta Ori"), 
+results = services[0].search(pos=SkyCoord.from_name("Delta Ori"),
                              diameter=Angle(10 * u.arcmin))
 
 # Assuming there are results, download the first file.
 print(f'Title: {results[0].title}, Format: {results[0].format}')
-file_name = download_file(results[0].getdataurl())  
+file_name = download_file(results[0].getdataurl())
 file_name
 ```
 
 ### 2.5 Table search
+
 Example:  Find the HEASARC Table Access Protocol (TAP) service, get some information about the available tables.
 
 ```{code-cell} ipython3
@@ -224,6 +219,7 @@ for t in tables:
 ```
 
 #### Column Information
+
 For any table, we can list the column names and descriptions.
 
 ```{code-cell} ipython3
@@ -232,20 +228,22 @@ for c in tables['zcat'].columns:
 ```
 
 #### Perform a Query
+
 Example:  Perform a cone search on the ZCAT catalog at M83 with a 1.0 degree radius.
 
 ```{code-cell} ipython3
 coord = SkyCoord.from_name("m83")
 query = f'''
-SELECT ra, dec, Radial_Velocity, radial_velocity_error, bmag, morph_type FROM public.zcat as cat where 
+SELECT ra, dec, Radial_Velocity, radial_velocity_error, bmag, morph_type FROM public.zcat as cat where
 contains(point('ICRS',cat.ra,cat.dec),circle('ICRS',{coord.ra.deg},{coord.dec.deg},1.0))=1
 '''
 results = services[0].service.run_async(query)
 
 results.to_table()
 ```
 
-## 3. Astroquery 
+## 3. Astroquery
+
 Many archives have Astroquery or Astroquery-compliant modules for data access, including:
 
 * Astroquery
@@ -261,9 +259,10 @@ Many archives have Astroquery or Astroquery-compliant modules for data access, i
 * Astroquery-compliant
   * [KOA Queries (pykoa.koa)](https://koa.ipac.caltech.edu/UserGuide/PyKOA/PyKOA.html)
 
-For more, see https://astroquery.readthedocs.io/en/latest/
+For more, see <https://astroquery.readthedocs.io/en/latest/>
 
 ### 3.1 NED
+
 Example:  Get an Astropy Table containing the objects from paper 2018ApJ...858...62K.  For more on the API, see [astroquery](https://astroquery.readthedocs.io/en/latest/ipac/ned/ned.html)
 
 ```{code-cell} ipython3

content/reference_notebooks/catalog_queries.md

@@ -6,7 +6,7 @@ jupytext:
     extension: .md
     format_name: myst
     format_version: 0.13
-    jupytext_version: 1.14.4
+    jupytext_version: 1.16.0
 kernelspec:
   display_name: Python 3 (ipykernel)
   language: python
@@ -20,7 +20,7 @@ language_info:
   name: python
   nbconvert_exporter: python
   pygments_lexer: ipython3
-  version: 3.10.11
+  version: 3.11.7
 toc:
   base_numbering: 1
   nav_menu: {}
@@ -42,25 +42,25 @@ widgets:
   version: 1.1.1
 ---
 
-# Accessing astronomical catalogs
+# Catalog Queries
 
 There are two ways to access astronomical data catalogs that are provided as table data with a VO API.
 
 First, there is a __[Simple Cone Search (SCS) protocol](http://www.ivoa.net/documents/latest/ConeSearch.html)__  used to search a given table with a given position and radius, getting back a table of results.  The interface requires only a position and search radius.
 
 For more complicated searches, the __[Table Access Protocol](http://www.ivoa.net/documents/TAP/)__ (TAP) protocol is  a powerful tool to search any VO table.  Here, we expand on its usage and that of the __[Astronomical Data Query Language](http://www.ivoa.net/documents/latest/ADQL.html)__ (ADQL) that it uses.
 
-- [Accessing astronomical catalogs](#accessing-astronomical-catalogs)
-    - [1. Simple cone search](#1-simple-cone-search)
-    - [2. Table Access Protocol queries](#2-table-access-protocol-queries)
-        - [2.1 TAP services](#21-tap-services)
-        - [2.2 Expressing queries in ADQL](#22-expressing-queries-in-adql)
-        - [2.3 A use case](#23-a-use-case)
-        - [2.4 TAP examples for a given service](#24-tap-examples-for-a-given-service)
-    - [3. Using the TAP to cross-correlate and combine](#3-using-the-tap-to-cross-correlate-and-combine)
-        - [3.1 Cross-correlating to combine catalogs](#31-cross-correlating-to-combine-catalogs)
-        - [3.2 Cross-correlating with user-defined columns](#32-cross-correlating-with-user-defined-columns)
-    - [4. Synchronous versus asynchronous queries](#4-synchronous-versus-asynchronous-queries)
+- [Catalog Queries](#catalog-queries)
+  - [1. Simple cone search](#1-simple-cone-search)
+  - [2. Table Access Protocol queries](#2-table-access-protocol-queries)
+    - [2.1 TAP services](#21-tap-services)
+    - [2.2 Expressing queries in ADQL](#22-expressing-queries-in-adql)
+    - [2.3 A use case](#23-a-use-case)
+    - [2.4 TAP examples for a given service](#24-tap-examples-for-a-given-service)
+  - [3. Using the TAP to cross-correlate and combine](#3-using-the-tap-to-cross-correlate-and-combine)
+    - [3.1 Cross-correlating to combine catalogs](#31-cross-correlating-to-combine-catalogs)
+    - [3.2 Cross-correlating with user-defined columns](#32-cross-correlating-with-user-defined-columns)
+  - [4. Synchronous versus asynchronous queries](#4-synchronous-versus-asynchronous-queries)
 
 ```{code-cell} ipython3
 # suppress some specific warnings that are not important
@@ -97,10 +97,11 @@ print(coord)
 Below, we go through the exercise of how we can figure out the most relevant table. But for now, let's assume that we know that we want the CFA redshift catalog refered to as 'zcat'. VO services are listed in a central Registry that can be searched through a [web interface](http://vao.stsci.edu/keyword-search/) or using PyVO's `regsearch`.  We use the registry to find the corresponding cone service and then submit our cone search.
 
 Registry services are of the following type:
-* simple cone search:  "scs"
-* table access protocol:  "tap" or "table"
-* simple image search:  "sia" or "image"
-* simple spectral access: "ssa"
+
+- simple cone search:  "scs"
+- table access protocol:  "tap" or "table"
+- simple image search:  "sia" or "image"
+- simple spectral access: "ssa"
 
 There are a number of things in the registry related to 'zcat' so we find the specific one that we want, which is the CFA version:
 
@@ -193,35 +194,35 @@ Now that we know all the possible column information in the zcat catalog, we can
 
 The basics of ADQL:
 
-* *SELECT * FROM my.interesting.catalog as cat...*
+- *SELECT * FROM my.interesting.catalog as cat...*
 
 says you want all ("*") columns from the catalog called "my.interesting.catalog", which you will refer to in the rest of the query by the more compact name of "cat".
 
 Instead of returning all columns, you can
 
-* *SELECT cat.RA, cat.DEC, cat.bmag from catalog as cat...*
+- *SELECT cat.RA, cat.DEC, cat.bmag from catalog as cat...*
 
 to only return the columns you're interested in. To use multiple catalogs, your query could start, e.g.,
 
-* *SELECT c1.RA,c1.DEC,c2.BMAG FROM catalog1 as c1 natural join catalog2 as c2...*
+- *SELECT c1.RA,c1.DEC,c2.BMAG FROM catalog1 as c1 natural join catalog2 as c2...*
 
 says that you want to query two catalogs zipped together the "natural" way, i.e., by looking for a common column.
 
 To select only some rows of the catalog based on the value in a column, you can add:
 
-* *WHERE cat.bmag < 14*
+- *WHERE cat.bmag < 14*
 
 says that you want to retrieve only those entries in the catalog whose bmag column has a value less than 14.
 
 You can also append
 
-* *ORDER by cat.bmag*
+- *ORDER by cat.bmag*
 
 to return the result sorted ascending by one of the columns, adding *DESC* to the end for descending.
 
 A few special functions in the ADQL allow you to query regions:
 
-* *WHERE contains( point('ICRS', cat.ra, cat.dec), circle('ICRS', 210.5, -6.5, 0.5))=1*
+- *WHERE contains( point('ICRS', cat.ra, cat.dec), circle('ICRS', 210.5, -6.5, 0.5))=1*
 
 is how you would ask for any catalog entries whose RA,DEC lie within a circular region defined by RA,DEC 210.5,-6.5 and a radius of 0.5 (all in degrees).  The 'ICRS' specifies the coordinate system.