From 069055f7c696b3508f8edd83f271a2848b95c591 Mon Sep 17 00:00:00 2001 From: Tom Donaldson Date: Tue, 2 Jan 2024 15:57:10 -0500 Subject: [PATCH] Modify notebook headers to match file names; update index.md --- .../reference_notebooks/basic_reference.md | 81 +++++---- .../reference_notebooks/catalog_queries.md | 49 +++--- content/reference_notebooks/image_access.md | 40 ++--- .../reference_notebooks/spectral_access.md | 24 +-- .../ucds_unified_content_descriptors.md | 38 ++-- content/reference_notebooks/votables.md | 11 +- .../candidate_list_exercise.md | 47 ++--- .../candidate_list_solution.md | 92 +++++----- .../use_case_notebooks/hr_diagram_exercise.md | 135 +++++++------- .../use_case_notebooks/hr_diagram_solution.md | 166 +++++++++--------- .../proposal_prep_exercise.md | 52 +++--- .../proposal_prep_solution.md | 46 ++--- index.md | 32 +++- 13 files changed, 423 insertions(+), 390 deletions(-) diff --git a/content/reference_notebooks/basic_reference.md b/content/reference_notebooks/basic_reference.md index 87fb41e..90f3541 100644 --- a/content/reference_notebooks/basic_reference.md +++ b/content/reference_notebooks/basic_reference.md @@ -1,53 +1,31 @@ --- jupytext: - notebook_metadata_filter: all + notebook_metadata_filter: a text_representation: extension: .md format_name: myst format_version: 0.13 - jupytext_version: 1.14.4 + jupytext_version: 1.16.0 kernelspec: display_name: Python 3 (ipykernel) language: python name: python3 -language_info: - codemirror_mode: - name: ipython - version: 3 - file_extension: .py - mimetype: text/x-python - name: python - nbconvert_exporter: python - pygments_lexer: ipython3 - version: 3.9.9 -toc: - base_numbering: 1 - nav_menu: {} - number_sections: false - sideBar: true - skip_h1_title: false - title_cell: Table of Contents - title_sidebar: Contents - toc_cell: false - toc_position: {} - toc_section_display: true - toc_window_display: true --- -# Quick Reference +# Basic Reference +++ ## 0. Setup -Please make sure your environment is set up according to the [instructions here](https://nasa-navo.github.io/navo-workshop/00_SETUP.html). -Ensure you have the latest version of the workshop material by updating your environment: -TBD +Please make sure your environment is set up according to the [instructions here](https://nasa-navo.github.io/navo-workshop/00_SETUP.html). +++ ## 1. Overview + NASA services can be queried from Python in multiple ways. + * Generic Virtual Observatory (VO) queries. * Call sequence is consistent, including for non-NASA resources. * Use the [`pyvo` package](https://pyvo.readthedocs.io/en/latest/). @@ -57,7 +35,9 @@ NASA services can be queried from Python in multiple ways. * Ad hoc archive-specific interfaces ## 2. VO Services + This workshop will introduce 4 types of VO queries: + * **VO Registry** - Discover what services are available worldwide * **Simple Cone Search** - Search for catalog object within a specified cone region * **Simple Image Access** - Search for image products within a spatial region @@ -84,6 +64,7 @@ warnings.filterwarnings('ignore', '.*Unknown element mirrorURL.*', vo.utils.xml. ``` ### 2.1 Look Up Services in VO Registry + Simple example: Find Simple Cone Search (conesearch) services related to SWIFT. ```{code-cell} ipython3 @@ -92,15 +73,19 @@ services ``` #### 2.1.1 Use different arguments/values to modify the simple example + | Argument | Description | Examples | | :-----: | :----------- | :-------- | | **servicetype** | Type of service | `conesearch` or `scs` for **Simple Cone Search**
`image` or `sia` for **Simple Image Access**
`spectrum` or `ssa` for **Simple Spectral Access**
`table` or `tap` for **Table Access Protocol**| | **keyword** | List of one or more keyword(s) to match service's metadata. Both ORs and ANDs may be specified.
| `['galex', 'swift']` matches 'galex' or 'swift'
`['hst survey']` matches services mentioning both 'hst' and 'survey' | | **waveband** | Resulting services have data in the specified waveband(s) | ‘radio’, ‘millimeter’, ‘infrared’, ‘optical’, ‘uv’, ‘euv’, ‘x-ray’ ‘gamma-ray’ | -#### 2.1.2 Inspect the results. +#### 2.1.2 Inspect the results + ##### Using pyvo + Although not lists, `pyvo` results can be iterated over to see each individual result. The results are specialized based on the type of query, providing access to the important properties of the results. Some useful accessors with registry results are: + * `short_name` - A short name * `res_title` - A more descriptive title * `res_description` - A more verbose description @@ -115,6 +100,7 @@ for s in list(services)[:4]: # (Treat services as list to get the subset of row ``` ##### Filtering results + Of the services we found, which one(s) have 'stsci.edu' in their unique identifier? ```{code-cell} ipython3 @@ -124,6 +110,7 @@ for s in stsci_services: ``` ##### Using astropy + With the `to_table()` method, `pyvo` results can also be converted to Astropy `Table` objects which offer a variety of addional features. See [astropy.table](http://docs.astropy.org/en/stable/table/) for more on working with Astropy Tables. ```{code-cell} ipython3 @@ -131,27 +118,30 @@ With the `to_table()` method, `pyvo` results can also be converted to Astropy `T services_table = services.to_table() # Print the column names and display 1st 3 rows with a subset of columns -print(f'\nColumn Names:\n{services_table.colnames}\n') -services_table['short_name', 'res_title', 'res_description'][:3] +print(f'\nColumn Names:\n{services_table.colnames}\n') +services_table['short_name', 'res_title', 'res_description'][:3] ``` ### 2.2 Cone search -Example: Find a cone search service for the USNO-B catalog and search it around M51 with a .1 degree radius. (More inspection could be done on the service list instead of blindly choosing the first service.) -The position (`pos`) is best specified with [SkyCoord](http://docs.astropy.org/en/stable/api/astropy.coordinates.SkyCoord.html) objects. +Example: Find a cone search service for the USNO-B catalog and search it around M51 with a .1 degree radius. (More inspection could be done on the service list instead of blindly choosing the first service.) -The size of the region is specified with the `radius` keyword and may be decimal degrees or an Astropy [Angle](http://docs.astropy.org/en/stable/api/astropy.coordinates.Angle.html#astropy.coordinates.Angle). +The position (`pos`) is best specified with [SkyCoord](http://docs.astropy.org/en/stable/api/astropy.coordinates.SkyCoord.html) objects. + +The size of the region is specified with the `radius` keyword and may be decimal degrees or an Astropy [Angle](http://docs.astropy.org/en/stable/api/astropy.coordinates.Angle.html#astropy.coordinates.Angle). ```{code-cell} ipython3 m51_pos = SkyCoord.from_name("m51") services = vo.regsearch(servicetype='conesearch', keywords='usno-b') results = services[0].search(pos=m51_pos, radius=0.1) # Astropy Table is useful for displaying cone search results. -results.to_table() +results.to_table() ``` ### 2.3 Image search + Example: Find an image search service for GALEX, and search it around coordinates 13:37:00.950,-29:51:55.51 (M83) with a radius of .2 degrees. Download the first file in the results. + #### Find an image service ```{code-cell} ipython3 @@ -160,6 +150,7 @@ services.to_table()['ivoid', 'short_name', 'res_title'] ``` #### Search one of the services + The first service looks good. Search it! For more details on using `SkyCoord`, see [its documentation](http://docs.astropy.org/en/stable/api/astropy.coordinates.SkyCoord.html#astropy.coordinates.SkyCoord) @@ -175,20 +166,23 @@ results.to_table() ``` #### Download an image + For the first result, print the file format and download the file. If repeatedly executing this code, add `cache=True` to `download_file()` to prevent repeated downloads. See [`download_file()` documentation here.](https://docs.astropy.org/en/stable/api/astropy.utils.data.download_file.html#astropy.utils.data.download_file) ```{code-cell} ipython3 print(results[0].format) -file_name = download_file(results[0].getdataurl()) +file_name = download_file(results[0].getdataurl()) file_name ``` ### 2.4 Spectral search + Example: Find a spectral service for x-ray data. Query it around Delta Ori with a search **diameter** of 10 arc minutes, and download the first data product. Note that the results table can be inspected for potentially useful columns. Spectral search is very similar to image search. In this example, note: + * **`diameter`** defines the size of the search region * `waveband` used in `regsearch()` * Astropy `Angle` used to specify radius units other than degrees. @@ -198,16 +192,17 @@ Spectral search is very similar to image search. In this example, note: services = vo.regsearch(servicetype='spectrum', waveband='x-ray') # Assuming there are services and the first one is OK... -results = services[0].search(pos=SkyCoord.from_name("Delta Ori"), +results = services[0].search(pos=SkyCoord.from_name("Delta Ori"), diameter=Angle(10 * u.arcmin)) # Assuming there are results, download the first file. print(f'Title: {results[0].title}, Format: {results[0].format}') -file_name = download_file(results[0].getdataurl()) +file_name = download_file(results[0].getdataurl()) file_name ``` ### 2.5 Table search + Example: Find the HEASARC Table Access Protocol (TAP) service, get some information about the available tables. ```{code-cell} ipython3 @@ -224,6 +219,7 @@ for t in tables: ``` #### Column Information + For any table, we can list the column names and descriptions. ```{code-cell} ipython3 @@ -232,12 +228,13 @@ for c in tables['zcat'].columns: ``` #### Perform a Query + Example: Perform a cone search on the ZCAT catalog at M83 with a 1.0 degree radius. ```{code-cell} ipython3 coord = SkyCoord.from_name("m83") query = f''' -SELECT ra, dec, Radial_Velocity, radial_velocity_error, bmag, morph_type FROM public.zcat as cat where +SELECT ra, dec, Radial_Velocity, radial_velocity_error, bmag, morph_type FROM public.zcat as cat where contains(point('ICRS',cat.ra,cat.dec),circle('ICRS',{coord.ra.deg},{coord.dec.deg},1.0))=1 ''' results = services[0].service.run_async(query) @@ -245,7 +242,8 @@ results = services[0].service.run_async(query) results.to_table() ``` -## 3. Astroquery +## 3. Astroquery + Many archives have Astroquery or Astroquery-compliant modules for data access, including: * Astroquery @@ -261,9 +259,10 @@ Many archives have Astroquery or Astroquery-compliant modules for data access, i * Astroquery-compliant * [KOA Queries (pykoa.koa)](https://koa.ipac.caltech.edu/UserGuide/PyKOA/PyKOA.html) -For more, see https://astroquery.readthedocs.io/en/latest/ +For more, see ### 3.1 NED + Example: Get an Astropy Table containing the objects from paper 2018ApJ...858...62K. For more on the API, see [astroquery](https://astroquery.readthedocs.io/en/latest/ipac/ned/ned.html) ```{code-cell} ipython3 diff --git a/content/reference_notebooks/catalog_queries.md b/content/reference_notebooks/catalog_queries.md index c1ab71a..776a0ac 100644 --- a/content/reference_notebooks/catalog_queries.md +++ b/content/reference_notebooks/catalog_queries.md @@ -6,7 +6,7 @@ jupytext: extension: .md format_name: myst format_version: 0.13 - jupytext_version: 1.14.4 + jupytext_version: 1.16.0 kernelspec: display_name: Python 3 (ipykernel) language: python @@ -20,7 +20,7 @@ language_info: name: python nbconvert_exporter: python pygments_lexer: ipython3 - version: 3.10.11 + version: 3.11.7 toc: base_numbering: 1 nav_menu: {} @@ -42,7 +42,7 @@ widgets: version: 1.1.1 --- -# Accessing astronomical catalogs +# Catalog Queries There are two ways to access astronomical data catalogs that are provided as table data with a VO API. @@ -50,17 +50,17 @@ First, there is a __[Simple Cone Search (SCS) protocol](http://www.ivoa.net/docu For more complicated searches, the __[Table Access Protocol](http://www.ivoa.net/documents/TAP/)__ (TAP) protocol is a powerful tool to search any VO table. Here, we expand on its usage and that of the __[Astronomical Data Query Language](http://www.ivoa.net/documents/latest/ADQL.html)__ (ADQL) that it uses. -- [Accessing astronomical catalogs](#accessing-astronomical-catalogs) - - [1. Simple cone search](#1-simple-cone-search) - - [2. Table Access Protocol queries](#2-table-access-protocol-queries) - - [2.1 TAP services](#21-tap-services) - - [2.2 Expressing queries in ADQL](#22-expressing-queries-in-adql) - - [2.3 A use case](#23-a-use-case) - - [2.4 TAP examples for a given service](#24-tap-examples-for-a-given-service) - - [3. Using the TAP to cross-correlate and combine](#3-using-the-tap-to-cross-correlate-and-combine) - - [3.1 Cross-correlating to combine catalogs](#31-cross-correlating-to-combine-catalogs) - - [3.2 Cross-correlating with user-defined columns](#32-cross-correlating-with-user-defined-columns) - - [4. Synchronous versus asynchronous queries](#4-synchronous-versus-asynchronous-queries) +- [Catalog Queries](#catalog-queries) + - [1. Simple cone search](#1-simple-cone-search) + - [2. Table Access Protocol queries](#2-table-access-protocol-queries) + - [2.1 TAP services](#21-tap-services) + - [2.2 Expressing queries in ADQL](#22-expressing-queries-in-adql) + - [2.3 A use case](#23-a-use-case) + - [2.4 TAP examples for a given service](#24-tap-examples-for-a-given-service) + - [3. Using the TAP to cross-correlate and combine](#3-using-the-tap-to-cross-correlate-and-combine) + - [3.1 Cross-correlating to combine catalogs](#31-cross-correlating-to-combine-catalogs) + - [3.2 Cross-correlating with user-defined columns](#32-cross-correlating-with-user-defined-columns) + - [4. Synchronous versus asynchronous queries](#4-synchronous-versus-asynchronous-queries) ```{code-cell} ipython3 # suppress some specific warnings that are not important @@ -97,10 +97,11 @@ print(coord) Below, we go through the exercise of how we can figure out the most relevant table. But for now, let's assume that we know that we want the CFA redshift catalog refered to as 'zcat'. VO services are listed in a central Registry that can be searched through a [web interface](http://vao.stsci.edu/keyword-search/) or using PyVO's `regsearch`. We use the registry to find the corresponding cone service and then submit our cone search. Registry services are of the following type: -* simple cone search: "scs" -* table access protocol: "tap" or "table" -* simple image search: "sia" or "image" -* simple spectral access: "ssa" + +- simple cone search: "scs" +- table access protocol: "tap" or "table" +- simple image search: "sia" or "image" +- simple spectral access: "ssa" There are a number of things in the registry related to 'zcat' so we find the specific one that we want, which is the CFA version: @@ -193,35 +194,35 @@ Now that we know all the possible column information in the zcat catalog, we can The basics of ADQL: -* *SELECT * FROM my.interesting.catalog as cat...* +- *SELECT * FROM my.interesting.catalog as cat...* says you want all ("*") columns from the catalog called "my.interesting.catalog", which you will refer to in the rest of the query by the more compact name of "cat". Instead of returning all columns, you can -* *SELECT cat.RA, cat.DEC, cat.bmag from catalog as cat...* +- *SELECT cat.RA, cat.DEC, cat.bmag from catalog as cat...* to only return the columns you're interested in. To use multiple catalogs, your query could start, e.g., -* *SELECT c1.RA,c1.DEC,c2.BMAG FROM catalog1 as c1 natural join catalog2 as c2...* +- *SELECT c1.RA,c1.DEC,c2.BMAG FROM catalog1 as c1 natural join catalog2 as c2...* says that you want to query two catalogs zipped together the "natural" way, i.e., by looking for a common column. To select only some rows of the catalog based on the value in a column, you can add: -* *WHERE cat.bmag < 14* +- *WHERE cat.bmag < 14* says that you want to retrieve only those entries in the catalog whose bmag column has a value less than 14. You can also append -* *ORDER by cat.bmag* +- *ORDER by cat.bmag* to return the result sorted ascending by one of the columns, adding *DESC* to the end for descending. A few special functions in the ADQL allow you to query regions: -* *WHERE contains( point('ICRS', cat.ra, cat.dec), circle('ICRS', 210.5, -6.5, 0.5))=1* +- *WHERE contains( point('ICRS', cat.ra, cat.dec), circle('ICRS', 210.5, -6.5, 0.5))=1* is how you would ask for any catalog entries whose RA,DEC lie within a circular region defined by RA,DEC 210.5,-6.5 and a radius of 0.5 (all in degrees). The 'ICRS' specifies the coordinate system. diff --git a/content/reference_notebooks/image_access.md b/content/reference_notebooks/image_access.md index 7cc6742..a821880 100644 --- a/content/reference_notebooks/image_access.md +++ b/content/reference_notebooks/image_access.md @@ -38,19 +38,19 @@ widgets: version: 1.1.1 --- -# Searching for and retrieving images +# Image Access In this notebook, we show how to search for and retrieve images from VO services using the Registry and the __[Simple Image Access](http://www.ivoa.net/documents/SIA/)__ (SIA) protocol. -- [Searching for and retrieving images](#searching-for-and-retrieving-images) - - [1. Finding SIA resources from the Registry](#1-finding-sia-resources-from-the-registry) - - [2. Using SIA to retrieve an image](#2-using-sia-to-retrieve-an-image) - - [3. Viewing the resulting image](#3-viewing-the-resulting-image) - - [JPG images](#jpg-images) - - [Fits files](#fits-files) - - [4. Example of data available through multiple services](#4-example-of-data-available-through-multiple-services) - - [Using HEASARC](#using-heasarc) - - [Using SDSS SkyServer](#using-sdss-skyserver) +- [Image Access](#image-access) + - [1. Finding SIA resources from the Registry](#1-finding-sia-resources-from-the-registry) + - [2. Using SIA to retrieve an image](#2-using-sia-to-retrieve-an-image) + - [3. Viewing the resulting image](#3-viewing-the-resulting-image) + - [JPG images](#jpg-images) + - [Fits files](#fits-files) + - [4. Example of data available through multiple services](#4-example-of-data-available-through-multiple-services) + - [Using HEASARC](#using-heasarc) + - [Using SDSS SkyServer](#using-sdss-skyserver) +++ @@ -63,7 +63,7 @@ import numpy as np import matplotlib import matplotlib.pyplot as plt -%matplotlib inline +%matplotlib inline import pyvo as vo @@ -81,7 +81,7 @@ warnings.filterwarnings("ignore", module="pyvo.utils.xml.*") ## 1. Finding SIA resources from the Registry -First, how do we find out what services are available? These are listed in a registry at STScI (__[see here](http://www.ivoa.net/documents/RegTAP/)__). Our Registry function gives a simple interface for how to search for services. +First, how do we find out what services are available? These are listed in a registry at STScI (__[see here](http://www.ivoa.net/documents/RegTAP/)__). Our Registry function gives a simple interface for how to search for services. Let's search for services providing images in the ultraviolet bands: @@ -103,9 +103,9 @@ This shows us that the data we are interested in comes from the HEASARC's SkyVie ## 2. Using SIA to retrieve an image -Now we look for images of our favorite source. See __[the SIA definition](http://www.ivoa.net/documents/WD/SIA/sia-20040524.html)__ for usage. In short, you can specify the central position and the size (degrees as one or two floats for the RA, DEC directions). It is up to the service to determine how to provide this. Optionally, you can limit it to the format you want, e.g., "image/fits" or "image/png" etc. +Now we look for images of our favorite source. See __[the SIA definition](http://www.ivoa.net/documents/WD/SIA/sia-20040524.html)__ for usage. In short, you can specify the central position and the size (degrees as one or two floats for the RA, DEC directions). It is up to the service to determine how to provide this. Optionally, you can limit it to the format you want, e.g., "image/fits" or "image/png" etc. -What is returned to you is not the image itself but a list of images available and how to access them. This is easiest shown by example: +What is returned to you is not the image itself but a list of images available and how to access them. This is easiest shown by example: ```{code-cell} ipython3 coords = coord.SkyCoord.from_name("m51") @@ -114,7 +114,7 @@ im_table = uvot_services[0].search(pos=coords,size=0.2,format='image/jpeg') im_table.to_table() ``` -Extract the fields you're interested in, e.g., the URLs of the images made by skyview. Note that specifying as we did SwiftUVOT, we get a number of different images, e.g., UVOT U, V, B, W1, W2, etc. For each survey, there are two URLs, first the FITS IMAGE and second the JPEG. +Extract the fields you're interested in, e.g., the URLs of the images made by skyview. Note that specifying as we did SwiftUVOT, we get a number of different images, e.g., UVOT U, V, B, W1, W2, etc. For each survey, there are two URLs, first the FITS IMAGE and second the JPEG. Note that different services will return different column names, but all will have a column giving the URL to access the image. Though it has different column names in different services, it can always be accessed through the `getdataurl` function. @@ -166,7 +166,7 @@ services = vo.regsearch(servicetype='image', keywords=['sloan'], waveband='optic services.to_table()[np.where(np.isin(services.to_table()['short_name'], 'SDSSDR7'))]['ivoid', 'short_name'] ``` -So one of these is served by SDSS's SkyServer and the other by HEASARC's SkyView. +So one of these is served by SDSS's SkyServer and the other by HEASARC's SkyView. +++ @@ -183,8 +183,8 @@ sdss_table_heasarc.to_table() ## If you only run this once, you can do it in memory in one line: ## This fetches the FITS as an astropy.io.fits object in memory # hdu_list = sdss_table_heasarc[0].getdataobj() -## But if you might run this notebook repeatedly with limited bandwidth, -## download it once and cache it. +## But if you might run this notebook repeatedly with limited bandwidth, +## download it once and cache it. # Get the filter g version file_name=download_file(sdss_table_heasarc[0].getdataurl(), cache=True, timeout=600) @@ -198,8 +198,8 @@ plt.imshow(hdu_list[0].data, cmap='gray', origin='lower', vmax=1200, vmin=1010) ```{code-cell} ipython3 jhu_dr7_service = [s for s in services if 'SDSSDR7' in s.short_name and 'jhu' in s.ivoid][0] -# Note: jhu_dr7_service access url has hard-wired "format=image/fits". -# If you specify anythign else, it errors. If you specify nothing, +# Note: jhu_dr7_service access url has hard-wired "format=image/fits". +# If you specify anythign else, it errors. If you specify nothing, # then the search() method puts "format=all", which errors. So specify "format=None" for now. sdss_table_jhu=jhu_dr7_service.search(pos=coords,size=0.2, format=None) sdss_table_jhu.to_table().show_in_notebook(display_length = 5) diff --git a/content/reference_notebooks/spectral_access.md b/content/reference_notebooks/spectral_access.md index 8d323d1..4864a65 100644 --- a/content/reference_notebooks/spectral_access.md +++ b/content/reference_notebooks/spectral_access.md @@ -32,9 +32,9 @@ toc: toc_window_display: true --- -# Retrieve spectra using Simple Spectral Access protocol +# Spectral Access -This notebook is one of a set produced by NAVO to demonstrate data access with python tools. +This notebook is one of a set produced by NAVO to demonstrate data access with python tools. In this notebook, we show how to search for and retrieve spectra from VO services using the Registry and the __[Simple Spectral Access](http://www.ivoa.net/documents/SSA/)__ (SSA) protocol. @@ -43,7 +43,7 @@ import numpy as np import matplotlib import matplotlib.pyplot as plt -%matplotlib inline +%matplotlib inline import requests, io @@ -73,7 +73,7 @@ services.to_table()['ivoid','short_name'] We can look at only the Chandra entry: ```{code-cell} ipython3 -chandra_service = [s for s in services if 'Chandra' in s.short_name][0] +chandra_service = [s for s in services if 'Chandra' in s.short_name][0] chandra_service.access_url ``` @@ -94,8 +94,8 @@ Accessing one of the spectra. ## If you only run this once, you can do it in memory in one line: ## This fetches the FITS as an astropy.io.fits object in memory # hdu_list = spec_tables[0].getdataobj() -## But if you might run this notebook repeatedly with limited bandwidth, -## download it once and cache it. +## But if you might run this notebook repeatedly with limited bandwidth, +## download it once and cache it. file_name = download_file(spec_tables[0].getdataurl(),cache=True) hdu_list = fits.open(file_name) ``` @@ -110,25 +110,25 @@ spec_table ```{code-cell} ipython3 matplotlib.rcParams['figure.figsize'] = (12, 10) -for i in range(len(spec_table)): - +for i in range(len(spec_table)): + ax = plt.subplot(6,2,i+1) pha = plt.plot( spec_table['CHANNEL'][i],spec_table['COUNTS'][i]) ax.set_yscale('log') - + if spec_table['TG_PART'][i] == 1: instr='HEG' if spec_table['TG_PART'][i] == 2: instr='MEG' if spec_table['TG_PART'][i] == 3: instr='LEG' - + ax.set_title("{grating}{order:+d}".format(grating=instr, order=spec_table['TG_M'][i])) - + plt.tight_layout() ``` -This can then be analyzed in your favorite spectral analysis tool, e.g., [pyXspec](https://heasarc.gsfc.nasa.gov/xanadu/xspec/python/html/index.html). (For the winter 2018 AAS workshop, we demonstrated this in a [notebook](https://github.com/NASA-NAVO/aas_workshop_2018/blob/master/heasarc/heasarc_Spectral_Access.md) that you can consult for how to use pyXspec, but the pyXspec documentation will have more information.) +This can then be analyzed in your favorite spectral analysis tool, e.g., [pyXspec](https://heasarc.gsfc.nasa.gov/xanadu/xspec/python/html/index.html). (For the winter 2018 AAS workshop, we demonstrated this in a [notebook](https://github.com/NASA-NAVO/aas_workshop_2018/blob/master/heasarc/heasarc_Spectral_Access.md) that you can consult for how to use pyXspec, but the pyXspec documentation will have more information.) ```{code-cell} ipython3 diff --git a/content/reference_notebooks/ucds_unified_content_descriptors.md b/content/reference_notebooks/ucds_unified_content_descriptors.md index f1fc249..115504a 100644 --- a/content/reference_notebooks/ucds_unified_content_descriptors.md +++ b/content/reference_notebooks/ucds_unified_content_descriptors.md @@ -22,15 +22,15 @@ language_info: version: 3.7.10 --- -# UCDs: working with heterogeneous tables +# UCDs (Unified Content Descriptors) -Suppose you want to do something using a column that you expect to find in a bunch of different tables, like coordinates and time. It's a good bet that many if not most of the tables have coordinate columns, but there's no rule about what they have to be named. +Suppose you want to do something using a column that you expect to find in a bunch of different tables, like coordinates and time. It's a good bet that many if not most of the tables have coordinate columns, but there's no rule about what they have to be named. -When doing detailed catalog queries with the TAP, you can obviously examine the columns of every table you're interested in to find the columns you want. Then you can hard-code the correct ones into each query for each table and service. +When doing detailed catalog queries with the TAP, you can obviously examine the columns of every table you're interested in to find the columns you want. Then you can hard-code the correct ones into each query for each table and service. -Or, you can also search for keywords like "ra" or "ascension" in the columns and their descriptions to get the columns you want automatically that way. +Or, you can also search for keywords like "ra" or "ascension" in the columns and their descriptions to get the columns you want automatically that way. -But is there are more generic way? [Unified Content Descriptors (UCDs)](http://www.ivoa.net/documents/latest/UCD.html) are a VO standard that allows table publishers to name their columns whatever they (or their contributors) want but to identify those that contain standard sorts of data. For example, the RA column could be called "RA", "ra", "Right_Ascension", etc. But in all cases, a VO service can label the column with its UCD, which is "pos.eq.ra". This information is not part of the table but part of the meta-data that the service may provide with that data. Though not required of all VO services, UCDs are commonly provided precisely to make such tasks as identifying the columns of interest easier to automate. +But is there are more generic way? [Unified Content Descriptors (UCDs)](http://www.ivoa.net/documents/latest/UCD.html) are a VO standard that allows table publishers to name their columns whatever they (or their contributors) want but to identify those that contain standard sorts of data. For example, the RA column could be called "RA", "ra", "Right_Ascension", etc. But in all cases, a VO service can label the column with its UCD, which is "pos.eq.ra". This information is not part of the table but part of the meta-data that the service may provide with that data. Though not required of all VO services, UCDs are commonly provided precisely to make such tasks as identifying the columns of interest easier to automate. This is easiest to show by example. @@ -70,7 +70,7 @@ for c in columns: print(f'{f"{c.name} [{c.ucd}]":30s} - {c.description}') ``` -The PyVO method to get the columns will automatically fetch all the meta-data about those columns. It's up to the service provider to set them correctly, of course, but in this case, we see that the column named "MatchRA" is identified with the UCD "pos.eq.ra". +The PyVO method to get the columns will automatically fetch all the meta-data about those columns. It's up to the service provider to set them correctly, of course, but in this case, we see that the column named "MatchRA" is identified with the UCD "pos.eq.ra". So if we did not know the exact name used in HSCv3 for the RA, we could do something like this looking for the string "RA": @@ -79,7 +79,7 @@ ra_name=[c.name for c in columns if 'RA' in c.name or "ascension" in c.name.lowe print(ra_name) ``` -But a more general approach is to check for the correct UCD. It also has the further advantage that it can be used to label columns that should be used for certain purposes when there are multiple possibilities. For instance, this table has MatchRA and SourceRA. Let's check the UCD: +But a more general approach is to check for the correct UCD. It also has the further advantage that it can be used to label columns that should be used for certain purposes when there are multiple possibilities. For instance, this table has MatchRA and SourceRA. Let's check the UCD: (Note that the UCD is not required. If it isn't there, you get a None type, so code the check carefully) @@ -89,11 +89,11 @@ dec_name=[c.name for c in columns if c.ucd and 'pos.eq.dec' in c.ucd][0] ra_name,dec_name ``` -What that shows you is that though there are two columns in this table that give RA information, only one has the 'pos.eq.ra' UCD. The documentation for this ought to explain the usage of these columns, and the UCD should not be used as a substitute for understanding the table. But it can be a useful tool. +What that shows you is that though there are two columns in this table that give RA information, only one has the 'pos.eq.ra' UCD. The documentation for this ought to explain the usage of these columns, and the UCD should not be used as a substitute for understanding the table. But it can be a useful tool. +++ -In particular, you can use the UCDs to look for catalogs that might have the information you're interested in. Then you can code the same query to work for different tables (with different column names) in a loop. This sends a bunch of queries but doesn't take too long, a minute maybe. (One is particularly slow.) +In particular, you can use the UCDs to look for catalogs that might have the information you're interested in. Then you can code the same query to work for different tables (with different column names) in a loop. This sends a bunch of queries but doesn't take too long, a minute maybe. (One is particularly slow.) ```{code-cell} ipython3 # Look for all TAP services with x-ray and optical data @@ -111,30 +111,30 @@ for s in vo.regsearch(servicetype='tap',keywords=['x-ray','optical']): names={} for ucd in ['pos.eq.ra','pos.eq.dec','time.start','time.end']: cols=[c.name for c in t.columns if c.ucd and ucd in c.ucd] - if len(cols) > 0: + if len(cols) > 0: names[ucd]=cols[0] # use the first that matches - if len(names.keys()) == 4: + if len(names.keys()) == 4: print(f" Table {t.name} has the right columns. Counting rows matching my time.") - # For a first look, a very simple query counting rows in a + # For a first look, a very simple query counting rows in a # time range of interest: query=f"select count({names['time.start']}) from {t.name}" \ - f" where {names['time.start']} > 52000 " + f" where {names['time.start']} > 52000 " try: results=s.search(query) except: print("Problem executing query. Continuing to next.") continue - # For this simple query, the result is a single number, the count. - # But different services might name the result differently, so - # don't assume you know the column name. + # For this simple query, the result is a single number, the count. + # But different services might name the result differently, so + # don't assume you know the column name. print(" Found {} results from {}\n".format(results.to_table()[0][0],t.name)) # If the query above asked for the matching data rather than the - # count, you might want to collect the results. + # count, you might want to collect the results. # Careful: here we're assuming the table names are unique collection[t.name]=results ``` -You can also use UCDs to look at the results. Above, we collected just the first 10 rows of the four columns we're interested in from every catalog that had them. But these tables still have their original column names. So the UCDs will still be useful, and PyVO provides a simple routine to convert from UCD to column (field) name. +You can also use UCDs to look at the results. Above, we collected just the first 10 rows of the four columns we're interested in from every catalog that had them. But these tables still have their original column names. So the UCDs will still be useful, and PyVO provides a simple routine to convert from UCD to column (field) name. Note, however, that returning the UCDs as part of the result is not mandatory, and some services do not do it. So you'll have to check. @@ -163,7 +163,7 @@ results=hsc.service.search("select top 10 * from dbo.DetailedCatalog") [r.getbyucd('phot.mag') for r in results] ``` -Note that we can see earlier in this notebook, when we looked at this table's contents, that there are two phot.mag fields in this table, MagAper2 and MagAuto. The getbyucd() and fieldname_with_ucd() routines do not currently allow you to handle multiple columns with the same UCD. The code can help you find what you want, but it depends on the meta data the service defines, and you still must look at the detailed information for each catalog you use to understand what it contains. +Note that we can see earlier in this notebook, when we looked at this table's contents, that there are two phot.mag fields in this table, MagAper2 and MagAuto. The getbyucd() and fieldname_with_ucd() routines do not currently allow you to handle multiple columns with the same UCD. The code can help you find what you want, but it depends on the meta data the service defines, and you still must look at the detailed information for each catalog you use to understand what it contains. ```{code-cell} ipython3 diff --git a/content/reference_notebooks/votables.md b/content/reference_notebooks/votables.md index 4c858c5..94e0532 100644 --- a/content/reference_notebooks/votables.md +++ b/content/reference_notebooks/votables.md @@ -35,18 +35,21 @@ toc: toc_window_display: true --- -# Creating a VO Table from a CSV file +# VO Tables + +## Create a VO Table from an Astropy Table +++ There are several ways of doing this, and there are a few object layers here, which can be confusing: + - Standard [astropy Table](https://docs.astropy.org/en/stable/table/) objects - [Votable Table](https://docs.astropy.org/en/stable/api/astropy.io.votable.tree.Table.html#astropy.io.votable.tree.Table) objects - [Votable VOTableFile](https://docs.astropy.org/en/stable/api/astropy.io.votable.tree.VOTableFile.html#astropy.io.votable.tree.VOTableFile) objects (may contain multiple votable Tables) -Although some things can be done with generic astropy Tables, other VO operations can only be done with VO Tables or VOTableFile objects. +Although some things can be done with generic astropy Tables, other VO operations can only be done with VO Tables or VOTableFile objects. -This is easiest to see with an example. +This is easiest to see with an example. ```{code-cell} ipython3 import io, numpy @@ -81,7 +84,7 @@ myaptable=aptable.Table( [241.519, 20.8014], [317.088, 18.2002], [329.235, 6.64845], - [333.830, 37.3012] ]), + [333.830, 37.3012] ]), names=["RA","DEC"]) print(type(myaptable)) diff --git a/content/use_case_notebooks/candidate_list_exercise.md b/content/use_case_notebooks/candidate_list_exercise.md index f35033d..87c3aa3 100644 --- a/content/use_case_notebooks/candidate_list_exercise.md +++ b/content/use_case_notebooks/candidate_list_exercise.md @@ -22,17 +22,17 @@ language_info: version: 3.8.2 --- -# Science User Case - Inspecting a Candidate List +# Candidate List Exercise Ogle et al. (2016) mined the NASA/IPAC Extragalactic Database (NED) to identify a new type of galaxy: Superluminous Spiral Galaxies. -Here's the paper: https://ui.adsabs.harvard.edu/abs/2016ApJ...817..109O/abstract +Here's the paper: Table 1 lists the positions of these Super Spirals. Based on those positions, let's create multiwavelength cutouts for each super spiral to see what is unique about this new class of objects. +++ -## 1. Import the Python modules we'll be using. +## 1. Import the Python modules we'll be using ```{code-cell} ipython3 # Suppress unimportant warnings. @@ -60,10 +60,10 @@ import pyvo as vo The next cell prepares the notebook to display our visualizations. ```{code-cell} ipython3 -%matplotlib inline +%matplotlib inline ``` -## 2. Search NED for objects in this paper. +## 2. Search NED for objects in this paper Insert a Code Cell below by clicking on the "Insert" Menu and choosing "Insert Cell Below". Then consult QuickReference.md to figure out how to use astroquery to search NED for all objects in a paper, based on the refcode of the paper. Inspect the resulting astropy table. @@ -72,7 +72,7 @@ Insert a Code Cell below by clicking on the "Insert" Menu and choosing "Insert C #objects_in_paper = Ned.query_refcode('2018ApJ...858...62K') ``` -## 3. Filter the NED results. +## 3. Filter the NED results The results from NED will include galaxies, but also other kinds of objects (e.g. galaxy clusters, galaxy groups). Print the 'Type' column to see the full range of classifications and filter the results so that we only keep the galaxies in the list. @@ -80,7 +80,7 @@ The results from NED will include galaxies, but also other kinds of objects (e.g ``` -## 4. Search the NAVO Registry for image resources. +## 4. Search the NAVO Registry for image resources The paper selected super spirals using WISE, SDSS, and GALEX images. Search the NAVO registry for all image resources, using the 'service_type' search parameter. How many image resources are currently available? @@ -89,7 +89,7 @@ The paper selected super spirals using WISE, SDSS, and GALEX images. Search the # services = vo.regsearch(servicetype='conesearch', keywords=['swift']) ``` -## 5. Search the NAVO Registry for image resources that will allow you to search for AllWISE images. +## 5. Search the NAVO Registry for image resources that will allow you to search for AllWISE images There are hundreds of image resources...too many to quickly read through. Try adding the 'keywords' search parameter to your registry search, and find the image resource you would need to search the AllWISE images. Remember from the Known Issues that 'keywords' must be a list. @@ -98,13 +98,14 @@ There are hundreds of image resources...too many to quickly read through. Try ad # services = vo.regsearch(servicetype='conesearch', keywords=['swift']) ``` -## 6. Choose the AllWISE image service that you are interested in. +## 6. Choose the AllWISE image service that you are interested in ```{code-cell} ipython3 ``` -## 7. Choose one of the galaxies in the NED list. +## 7. Choose one of the galaxies in the NED list + What is the position of this galaxy? ```{code-cell} ipython3 @@ -112,7 +113,7 @@ What is the position of this galaxy? # m83_pos = SkyCoord('13h37m00.950s -29d51m55.51s') ``` -## 8. Search for a list of AllWISE images that cover this galaxy. +## 8. Search for a list of AllWISE images that cover this galaxy How many images are returned? Which are you most interested in? @@ -121,7 +122,7 @@ How many images are returned? Which are you most interested in? #results = services[1].search(pos=m83_pos, size=.2) ``` -## 9. Use the .to_table() method to view the results as an Astropy table. +## 9. Use the .to_table() method to view the results as an Astropy table ```{code-cell} ipython3 @@ -130,6 +131,7 @@ How many images are returned? Which are you most interested in? ## 10. From the result in 8., select the first record for an image taken in WISE band W1 (3.6 micron) Hints: + * Loop over records and test on the `.bandpass_id` attribute of each record * Print the `.title` and `.bandpass_id` of the record you find, to verify it is the right one. @@ -137,17 +139,18 @@ Hints: ``` -## 11. Visualize this AllWISE image. +## 11. Visualize this AllWISE image + Hint: Locate the galaxy in the image by overplotting a ring centered on the galaxy on the image ```{code-cell} ipython3 # Hint: the QuickReference has this example: -# file_name = download_file(results[0].getdataurl()) -# Hint: when displaying the fits file play with the vmax value, a +# file_name = download_file(results[0].getdataurl()) +# Hint: when displaying the fits file play with the vmax value, a # vmax value around 10 will display a nice image ``` -## 12. Plot a cutout of the AllWISE image, centered on your position. +## 12. Plot a cutout of the AllWISE image, centered on your position Try a 60 arcsecond cutout. @@ -155,7 +158,7 @@ Try a 60 arcsecond cutout. # Hint: using Cutout2D imported above from from astropy.nddata ``` -## 13. Try visualizing a cutout of a GALEX image that covers your position. +## 13. Try visualizing a cutout of a GALEX image that covers your position Repeat steps 5, 6, 8 through 12 for GALEX. @@ -172,12 +175,12 @@ Repeat steps 5, 6, 8 through 12 for GALEX. ``` ```{code-cell} ipython3 -# Step 10: Select one of the images and print the title and relevent +# Step 10: Select one of the images and print the title and relevent # info. For example you can select an image with an 'enrValue' of 2.35e-07 ``` ```{code-cell} ipython3 -# Step 11: Visualize the image and overplot a ring on the image centered +# Step 11: Visualize the image and overplot a ring on the image centered # on your galaxy. (A very small vmax value around 0.01 is recomended) ``` @@ -186,9 +189,10 @@ Repeat steps 5, 6, 8 through 12 for GALEX. # centered on the galaxy ``` -## 14. Try visualizing a cutout of an SDSS image that covers your position. +## 14. Try visualizing a cutout of an SDSS image that covers your position Hints: + * Search the registry using `keywords=['sloan'] * Find the service with a `short_name` of `'SDSS SIAP'` * From Known Issues, recall that an empty string must be specified to the `format` parameter dues to a bug in the service. @@ -219,7 +223,8 @@ Hints: # use Cutout2D to get a 60 arcsecond cutout centered on the galaxy ``` -## 15. Try looping over all positions and plotting multiwavelength cutouts. +## 15. Try looping over all positions and plotting multiwavelength cutouts + Hint: Gather the data in ALLWISE, GALEX, and SDSS for each galaxy and plot as seperate cutouts centered on the galaxy position +++ diff --git a/content/use_case_notebooks/candidate_list_solution.md b/content/use_case_notebooks/candidate_list_solution.md index a57f748..16fd9fe 100644 --- a/content/use_case_notebooks/candidate_list_solution.md +++ b/content/use_case_notebooks/candidate_list_solution.md @@ -19,7 +19,7 @@ language_info: name: python nbconvert_exporter: python pygments_lexer: ipython3 - version: 3.11.6 + version: 3.11.7 toc: base_numbering: 1 nav_menu: {} @@ -37,7 +37,7 @@ widgets: version: 1.1.1 --- -# Science User Case - Inspecting a Candidate List +# Candidate List Solution [Ogle et al. (2016)](https://ui.adsabs.harvard.edu/abs/2016ApJ...817..109O/abstract) mined the NASA/IPAC Extragalactic Database (NED) to identify a new type of galaxy: Superluminous Spiral Galaxies. @@ -45,7 +45,7 @@ Table 1 lists the positions of these Super Spirals. Based on those positions, le +++ -## 1. Import the Python modules we'll be using. +## 1. Import the Python modules we'll be using ```{code-cell} ipython3 # Suppress unimportant warnings. @@ -73,10 +73,10 @@ import pyvo as vo The next cell prepares the notebook to display our visualizations. ```{code-cell} ipython3 -%matplotlib inline +%matplotlib inline ``` -## 2. Search NED for objects in this paper. +## 2. Search NED for objects in this paper Insert a Code Cell below by clicking on the "Insert" Menu and choosing "Insert Cell Below". Then consult QuickReference.md to figure out how to use astroquery to search NED for all objects in a paper, based on the refcode of the paper. Inspect the resulting astropy table. @@ -87,7 +87,7 @@ objects_in_paper = Ned.query_refcode('2016ApJ...817..109O') objects_in_paper.show_in_notebook() ``` -## 3. Filter the NED results. +## 3. Filter the NED results The results from NED will include galaxies, but also other kinds of objects (e.g. galaxy clusters, galaxy groups). Print the 'Type' column to see the full range of classifications and filter the results so that we only keep the galaxies in the list. @@ -104,7 +104,7 @@ galaxies = objects_in_paper[np.array(objects_in_paper['Type']) == 'G'] galaxies.show_in_notebook() ``` -## 4. Search the NAVO Registry for image resources. +## 4. Search the NAVO Registry for image resources The paper selected super spirals using WISE, SDSS, and GALEX images. Search the NAVO registry for all image resources, using the 'service_type' search parameter. How many image resources are currently available? @@ -116,7 +116,7 @@ print(f'{len(image_services)} result(s) found.') image_services.to_table()['ivoid', 'short_name', 'res_title'] ``` -## 5. Search the NAVO Registry for image resources that will allow you to search for AllWISE images. +## 5. Search the NAVO Registry for image resources that will allow you to search for AllWISE images There are hundreds of image resources...too many to quickly read through. Try adding the 'keywords' search parameter to your registry search, and find the image resource you would need to search the AllWISE images. Remember from the Known Issues that 'keywords' must be a list. @@ -128,14 +128,14 @@ print(f'{len(allwise_image_services)} result(s) found.') allwise_image_services.to_table()['ivoid', 'short_name', 'res_title'] ``` -## 6. Choose the AllWISE image service that you are interested in. +## 6. Choose the AllWISE image service that you are interested in ```{code-cell} ipython3 allwise_image_service = allwise_image_services[0] allwise_image_service.service ``` -## 7. Choose one of the galaxies in the NED list. +## 7. Choose one of the galaxies in the NED list ```{code-cell} ipython3 ra = galaxies['RA'][0] @@ -147,7 +147,7 @@ pos = SkyCoord(ra, dec, unit = 'deg') ra,dec ``` -## 8. Search for a list of AllWISE images that cover this galaxy. +## 8. Search for a list of AllWISE images that cover this galaxy How many images are returned? Which are you most interested in? @@ -156,7 +156,7 @@ allwise_image_table = allwise_image_service.search(pos=pos, size=0) allwise_image_table ``` -## 9. Use the .to_table() method to view the results as an Astropy table. +## 9. Use the .to_table() method to view the results as an Astropy table ```{code-cell} ipython3 allwise_images = allwise_image_table.to_table() @@ -166,6 +166,7 @@ allwise_images ## 10. From the result in 8., select the first record for an image taken in WISE band W1 (3.6 micron) Hints: + * Loop over records and test on the `.bandpass_id` attribute of each record * Print the `.title` and `.bandpass_id` of the record you find, to verify it is the right one. @@ -176,15 +177,15 @@ for allwise_image_record in allwise_image_table: print(allwise_image_record.title, allwise_image_record.bandpass_id) ``` -## 11. Visualize this AllWISE image. +## 11. Visualize this AllWISE image ```{code-cell} ipython3 ## If you only run this once, you can do it in memory in one line: ## This fetches the FITS as an astropy.io.fits object in memory #allwise_w1_image = allwise_image_record.getdataobj() -## But if you might run this notebook repeatedly with limited bandwidth, -## download it once and cache it. -file_name = download_file(allwise_image_record.getdataurl(), cache=True) +## But if you might run this notebook repeatedly with limited bandwidth, +## download it once and cache it. +file_name = download_file(allwise_image_record.getdataurl(), cache=True) allwise_w1_image = fits.open(file_name) ``` @@ -197,7 +198,7 @@ ax.imshow(allwise_w1_image[0].data, cmap='gray_r', origin='lower', vmax = 10) ax.scatter(ra, dec, transform=ax.get_transform('fk5'), s=500, edgecolor='red', facecolor='none') ``` -## 12. Plot a cutout of the AllWISE image, centered on your position. +## 12. Plot a cutout of the AllWISE image, centered on your position Try a 60 arcsecond cutout. @@ -213,7 +214,7 @@ ax.imshow(cutout.data, cmap='gray_r', origin='lower', vmax = 10) ax.scatter(ra, dec, transform=ax.get_transform('fk5'), s=500, edgecolor='red', facecolor='none') ``` -## 13. Try visualizing a cutout of a GALEX image that covers your position. +## 13. Try visualizing a cutout of a GALEX image that covers your position Repeat steps 4, 5, 6, 8 through 12 for GALEX. @@ -242,9 +243,9 @@ print(galex_image_record.title, galex_image_record.bandpass_id) ``` ```{code-cell} ipython3 -## See above regarding two ways to do this: +## See above regarding two ways to do this: #galex_nuv_image = fits.open(galex_image_record.getdataurl()) -file_name = download_file(galex_image_record.getdataurl(), cache=True) +file_name = download_file(galex_image_record.getdataurl(), cache=True) galex_nuv_image=fits.open(file_name) ``` @@ -260,7 +261,7 @@ print('Stdev:', np.std(image_data)) fig = plt.figure() ax = fig.add_subplot(1, 1, 1, projection=WCS(galex_nuv_image[0].header)) ax.imshow(galex_nuv_image[0].data, cmap='gray_r', origin='lower', vmin=0.0, vmax=0.01) -ax.scatter(ra, dec, transform=ax.get_transform('fk5'), s=500, edgecolor='red', facecolor='none') +ax.scatter(ra, dec, transform=ax.get_transform('fk5'), s=500, edgecolor='red', facecolor='none') ``` ```{code-cell} ipython3 @@ -273,9 +274,10 @@ ax.imshow(cutout.data, cmap='gray_r', origin='lower', vmin = 0.0, vmax = 0.01) ax.scatter(ra, dec, transform=ax.get_transform('fk5'), s=500, edgecolor='red', facecolor='none') ``` -## 14. Try visualizing a cutout of an SDSS image that covers your position. +## 14. Try visualizing a cutout of an SDSS image that covers your position Hints: + * Search the registry using `keywords=['sloan'] * Find the service with a `short_name` of `'SDSS SIAP'` * After obtaining your search results, select r-band images using the `.title` attribute of the records that are returned, since `.bandpass_id` is not populated. @@ -287,7 +289,7 @@ sdss_image_services.to_table()['ivoid', 'short_name', 'res_title', 'source_value ```{code-cell} ipython3 # Use list comprehension to check each service's short_name attribute. -# Given the above, we know the first match is the right one. +# Given the above, we know the first match is the right one. sdss_image_service = [s for s in sdss_image_services if 'SDSS SIAP' in s.short_name ][0] sdss_image_service.short_name ``` @@ -307,7 +309,7 @@ print(sdss_rband_record.title, sdss_rband_record.bandpass_id) ```{code-cell} ipython3 ## See above regarding two ways to do this # sdss_rband_image = fits.open(sdss_rband_record.getdataurl()) -file_name = download_file(sdss_rband_record.getdataurl(), cache=True) +file_name = download_file(sdss_rband_record.getdataurl(), cache=True) sdss_rband_image=fits.open(file_name) ``` @@ -319,7 +321,7 @@ ax = fig.add_subplot(1, 1, 1, projection=WCS(sdss_rband_image[0].header)) interval = vis.PercentileInterval(99.9) vmin,vmax = interval.get_limits(sdss_rband_image[0].data) norm = vis.ImageNormalize(vmin=vmin, vmax=vmax, stretch=vis.LogStretch(1000)) -ax.imshow(sdss_rband_image[0].data, cmap = 'gray_r', norm = norm, origin = 'lower') +ax.imshow(sdss_rband_image[0].data, cmap = 'gray_r', norm = norm, origin = 'lower') ax.scatter(ra, dec, transform=ax.get_transform('fk5'), s=500, edgecolor='red', facecolor='none') ``` @@ -329,11 +331,11 @@ fig = plt.figure() ax = fig.add_subplot(1, 1, 1, projection=cutout.wcs) vmin,vmax = interval.get_limits(sdss_rband_image[0].data) norm = vis.ImageNormalize(vmin=vmin, vmax=vmax, stretch=vis.LogStretch(1000)) -ax.imshow(cutout.data, cmap = 'gray_r', norm = norm, origin = 'lower') +ax.imshow(cutout.data, cmap = 'gray_r', norm = norm, origin = 'lower') ax.scatter(ra, dec, transform=ax.get_transform('fk5'), s=500, edgecolor='red', facecolor='none') ``` -## 15. Try looping over all positions and plotting multiwavelength cutouts. +## 15. Try looping over all positions and plotting multiwavelength cutouts +++ @@ -349,18 +351,18 @@ for galaxy in galaxy_subset: # Establish the position. ra = galaxy['RA'] dec = galaxy['DEC'] - pos = SkyCoord(ra, dec, unit = 'deg') - + pos = SkyCoord(ra, dec, unit = 'deg') + # Set up the plot for this position. fig = plt.figure(figsize=(20,6)) plt.suptitle('POSITION = ' + str(ra) + ', ' + str(dec), fontsize=16) # GALEX - + # Find the GALEX images that overlap the position. galex_image_table = galex_image_service.search(pos=pos, size=0.25) - - # Find the GALEX All-Sky Image Survey (AIS) Near-UV FITS coadd. + + # Find the GALEX All-Sky Image Survey (AIS) Near-UV FITS coadd. galex_image_record = None for record in galex_image_table: if (('image/fits' in record.format) and @@ -368,10 +370,10 @@ for galaxy in galaxy_subset: (record['productType'] == 'SCIENCE')): galex_image_record = record break - + if galex_image_record is not None: # Create a cutout. - file_name = download_file(galex_image_record.getdataurl(), cache=True) + file_name = download_file(galex_image_record.getdataurl(), cache=True) gimage = fits.open(file_name) galex_cutout = Cutout2D(gimage[0].data, pos, size, wcs=WCS(gimage[0].header)) @@ -384,22 +386,22 @@ for galaxy in galaxy_subset: # We didn't find a suitable image, so leave that subplot blank. ax = fig.add_subplot(1, 3, 1, projection=galex_cutout.wcs) ax.set_title('GALEX image not found') - + # SDSS - + # Find the SDSS images that overlap the position. sdss_image_table = sdss_image_service.search(pos=pos, size=0) - + # Find the first SDSS r-band image. sdss_rband_record = None for record in sdss_image_table: if 'Sloan Digital Sky Survey - Filter r' in record.title: sdss_rband_record = record break - + if sdss_rband_record is not None: # Create a cutout. - file_name = download_file(sdss_rband_record.getdataurl(), cache=True) + file_name = download_file(sdss_rband_record.getdataurl(), cache=True) sdss_rband_image=fits.open(file_name) sdss_cutout = Cutout2D(sdss_rband_image[0].data, pos, size, @@ -409,29 +411,29 @@ for galaxy in galaxy_subset: vmin,vmax = interval.get_limits(sdss_cutout.data) norm = vis.ImageNormalize(vmin=vmin, vmax=vmax, stretch=vis.LogStretch(1000)) ax = fig.add_subplot(1, 3, 2, projection=sdss_cutout.wcs) - ax.imshow(sdss_cutout.data, cmap = 'gray_r', norm = norm, origin = 'lower') + ax.imshow(sdss_cutout.data, cmap = 'gray_r', norm = norm, origin = 'lower') ax.scatter(ra, dec, transform=ax.get_transform('fk5'), s=500, edgecolor='red', facecolor='none') ax.set_title(sdss_rband_record.title) else: # We didn't find a suitable image, so leave that subplot blank. ax = fig.add_subplot(1, 3, 2, projection=galex_cutout.wcs) ax.set_title('SDSS rband image not found') - + # AllWISE - + # Find the AllWISE images that overlap the position. allwise_image_table = allwise_image_service.search(pos=pos, size=0) - + # Find the first AllWISE W1 channel image. allwise_image_record = None for record in allwise_image_table: if 'W1' in record.bandpass_id: allwise_image_record = record break - + if allwise_image_record is not None: # Create a cutout. - file_name = download_file(allwise_image_record.getdataurl(), cache=True) + file_name = download_file(allwise_image_record.getdataurl(), cache=True) allwise_w1_image=fits.open(file_name) allwise_cutout = Cutout2D(allwise_w1_image[0].data, pos, (size, size), diff --git a/content/use_case_notebooks/hr_diagram_exercise.md b/content/use_case_notebooks/hr_diagram_exercise.md index 08c5d84..75155ea 100644 --- a/content/use_case_notebooks/hr_diagram_exercise.md +++ b/content/use_case_notebooks/hr_diagram_exercise.md @@ -32,18 +32,18 @@ toc: toc_window_display: true --- -# Creating a stellar color-magnitude (or Hertzsprung-Russell) diagram +# HR (Hertzsprung-Russell) Diagram Exercise -The [Hertzsprung-Russell diagram](https://en.wikipedia.org/wiki/Hertzsprung–Russell_diagram) is a fundamental diagram in astronomy that displays important relationships between the stellar color (or temperature) and absolute brightness (or luminosity). +The [Hertzsprung-Russell diagram](https://en.wikipedia.org/wiki/Hertzsprung–Russell_diagram) is a fundamental diagram in astronomy that displays important relationships between the stellar color (or temperature) and absolute brightness (or luminosity). -In this exercise, we will use existing stellar catalogs to produce the H-R diagram. +In this exercise, we will use existing stellar catalogs to produce the H-R diagram. ```{code-cell} ipython3 -# As a hint, we include the code block for Python modules that you will likely need to import: +# As a hint, we include the code block for Python modules that you will likely need to import: import matplotlib import matplotlib.pyplot as plt import numpy as np -%matplotlib inline +%matplotlib inline # For downloading files from astropy.utils.data import download_file @@ -52,7 +52,7 @@ from astropy.io import fits import pyvo as vo from pyvo import registry -## There are a number of relatively unimportant warnings that +## There are a number of relatively unimportant warnings that ## show up, so for now, suppress them: import warnings warnings.filterwarnings("ignore", module="astropy.io.votable.*") @@ -61,56 +61,56 @@ warnings.filterwarnings("ignore", module="pyvo.utils.xml.*") ## Step 1: Find appropriate catalogs -We want to find a star catalog that has the available data to produce the H-R diagram, i.e., the absolute magnitudes (or both apparent magnitudes AND distances, so we can calculate the absolute magnitudes) in two optical bands (e.g., B and V). This would give us color. Or we need B- OR V- band magnitude and the stellar temperature. +We want to find a star catalog that has the available data to produce the H-R diagram, i.e., the absolute magnitudes (or both apparent magnitudes AND distances, so we can calculate the absolute magnitudes) in two optical bands (e.g., B and V). This would give us color. Or we need B- OR V- band magnitude and the stellar temperature. -To simplify this problem, we want to find a catalog of an open cluster of stars, where all the stars were born around the same time and are located in one cluster. This simplifies the issue of getting accurate distances to the stars. One famous cluster is the Pleiades in the constellation of Taurus. So first we start by searching for an existing catalog with data on Pleiades that will provide the necessary information about the stars: magnitudes in two bands (e.g., B and V), which can be used to measure color, or temperature of the star plus one magnitude. +To simplify this problem, we want to find a catalog of an open cluster of stars, where all the stars were born around the same time and are located in one cluster. This simplifies the issue of getting accurate distances to the stars. One famous cluster is the Pleiades in the constellation of Taurus. So first we start by searching for an existing catalog with data on Pleiades that will provide the necessary information about the stars: magnitudes in two bands (e.g., B and V), which can be used to measure color, or temperature of the star plus one magnitude. +++ -### DATA DISCOVERY STEPS: +### DATA DISCOVERY STEPS Here is useful link for [how the pyvo registry search works](https://pyvo.readthedocs.io/en/latest/registry/index.html). ```{code-cell} ipython3 :tags: [output_scroll] -# Write some code to perform a registry search using -# appropriate keywords and specify the service type. -# Also apply the includeaux=True option to return -# the maximum number of services: +# Write some code to perform a registry search using +# appropriate keywords and specify the service type. +# Also apply the includeaux=True option to return +# the maximum number of services: # Print the size of the output table and get a summary of the sources: ``` -Note: The includeaux=True includes auxiliary services. +Note: The includeaux=True includes auxiliary services. -Because we specified the service type, this returns only the TAP services for each of the avalible resorces matching our search criteria. So, the 'interfaces' column will only show TAP service as an avalible service, even if other services are avalible from the same resource. We know we need the service to be a TAP service since we know that we want to eventually access the search using additional column information (i.e., beyond RA and Dec, which is all that Simple Cone Search returns). +Because we specified the service type, this returns only the TAP services for each of the avalible resorces matching our search criteria. So, the 'interfaces' column will only show TAP service as an avalible service, even if other services are avalible from the same resource. We know we need the service to be a TAP service since we know that we want to eventually access the search using additional column information (i.e., beyond RA and Dec, which is all that Simple Cone Search returns). +++ -#### Next, we need to find which of these has the columns of interest, i.e. magnitudes in two bands to create the color-magnitude diagram. +#### Next, we need to find which of these has the columns of interest, i.e. magnitudes in two bands to create the color-magnitude diagram We can re-run the registry search, but further restrict the results by column UCD. We want tables that have magnitude columns; the most basic UCD to describe a magnitude column is phot.mag ```{code-cell} ipython3 # Perform another registry search, this time searching -# tables that have at least 1 magnitude column. +# tables that have at least 1 magnitude column. # Note: '%' serves as a wild card when searching columns -# How many tables do you get? +# How many tables do you get? ``` Note: the '%' serves as a wild card when searching by UCD -The IOVA standard enables resources to be as sepecific as they would like when defining the UCD of columns. For example, 'phot.mag' and 'phot.mag;em.opt.V' can both be used to describe a column containing the V magnitudes of objects. If a resource uses the latter to describe a column, a search using 'phot.mag' will not return that resource. A wild card would need to be used or the exact column UCD. The UCD search requires an exact match for a resource to be returned, so using the wild card will make it easier to discover a wider variety of resources. +The IOVA standard enables resources to be as sepecific as they would like when defining the UCD of columns. For example, 'phot.mag' and 'phot.mag;em.opt.V' can both be used to describe a column containing the V magnitudes of objects. If a resource uses the latter to describe a column, a search using 'phot.mag' will not return that resource. A wild card would need to be used or the exact column UCD. The UCD search requires an exact match for a resource to be returned, so using the wild card will make it easier to discover a wider variety of resources. +++ -So using this we can reduce the matched tables to ones that are a bit more catered to our experiment. Note, that there is redundancy in some resources since these are available via multiple services and/or publishers. Therefore a bit more cleaning can be done to provide only the unique matches. +So using this we can reduce the matched tables to ones that are a bit more catered to our experiment. Note, that there is redundancy in some resources since these are available via multiple services and/or publishers. Therefore a bit more cleaning can be done to provide only the unique matches. ```{code-cell} ipython3 -# Print the 'ivoid' column for these tables +# Print the 'ivoid' column for these tables ``` ```{code-cell} ipython3 @@ -128,82 +128,82 @@ We can read more information about the results we found. For each resource elem RESULT: Based on these, the second one (by Eichhorn et al) looks like a good start. -### At this point, you can proceed to Step 2. +### At this point, you can proceed to Step 2 -- OR -- -### Try a different data discovery method! +### Try a different data discovery method +++ {"tags": ["output_scroll"]} -### Alternative Method: Use ADS to search for appropriate paper and access data via NED. +### Alternative Method: Use ADS to search for appropriate paper and access data via NED -There are multiple paths for the data discovery. So it may also be that you know the paper that has the data you are interested in and want to access via the bibcode or authors, etc. +There are multiple paths for the data discovery. So it may also be that you know the paper that has the data you are interested in and want to access via the bibcode or authors, etc. -In this case, let's assume that we have the information that the Eichhorn+1970 paper has the data that we need to create the H-R diagram: https://ui.adsabs.harvard.edu/abs/1970MmRAS..73..125E/abstract +In this case, let's assume that we have the information that the Eichhorn+1970 paper has the data that we need to create the H-R diagram: -We can either search by bibcode (1970MmRAS..73..125E) or "Eichhorn" to get the access_urls that will allow us to work with the data. +We can either search by bibcode (1970MmRAS..73..125E) or "Eichhorn" to get the access_urls that will allow us to work with the data. -Before this step, if may help to see the names of the fields available to use. Notice the following fields: +Before this step, if may help to see the names of the fields available to use. Notice the following fields: -"source_value" contains the bibcode information that we want; "creator_seq" lists the authors; +"source_value" contains the bibcode information that we want; "creator_seq" lists the authors; -and +and -"access_url" provides the url from where the data can be accessed. +"access_url" provides the url from where the data can be accessed. ```{code-cell} ipython3 -# Print the fieldnames for the tap services. +# Print the fieldnames for the tap services. ``` -First, Try using bibcode: +First, Try using bibcode: ```{code-cell} ipython3 bibcode = '1970MmRAS..73..125E' # Eichhorn -# -# print "short_name", "source_value" and "access_url" information -# for the table entry that matches this bibcode. # -# Note that using the to_table() lets you search the result +# print "short_name", "source_value" and "access_url" information +# for the table entry that matches this bibcode. +# +# Note that using the to_table() lets you search the result # easily using all columns. But in the end, you want to get # back not an astropy table row, which you cannot use, but the -# original RegistryResult that has the callable TAP service. +# original RegistryResult that has the callable TAP service. ``` Note that the URL is a generic TAP url for Vizier. All of its tables can be accessed by that same TAP services. It'll be in the ADQL query itself that you specify the table name. We'll see this below. +++ -Next, try using Author name: +Next, try using Author name: ```{code-cell} ipython3 author = 'Eichhorn' -# -# print "short_name", "reference_url" and "access_url" information -# for the table entry that matches this author name. +# +# print "short_name", "reference_url" and "access_url" information +# for the table entry that matches this author name. # # Hint: The pyVO Registry resource page may help: # https://pyvo.readthedocs.io/en/latest/api/pyvo.registry.regtap.RegistryResource.html#pyvo.registry.regtap.RegistryResource.search ``` -These examples provide a few ways to access the information of interest. +These examples provide a few ways to access the information of interest. -Below are a few other ways to see what the tap_service table contains. +Below are a few other ways to see what the tap_service table contains. -1. To view the column information: tap_services.to_table().columns() shows the metadata contained in the tap service. We will reference some of this columns below as we try to find the appropriate table. +1. To view the column information: tap_services.to_table().columns() shows the metadata contained in the tap service. We will reference some of this columns below as we try to find the appropriate table. -2. tap_services[index].describe(): The table with the tap_services output has, in our case, 83 tables listed and each includes metadata containing some human readable description. You can get the description for one case or for all the records by iterating through the resource. In the former case, we show the description for the Eichhorn data. The latter case also follows. - +2. tap_services[index].describe(): The table with the tap_services output has, in our case, 83 tables listed and each includes metadata containing some human readable description. You can get the description for one case or for all the records by iterating through the resource. In the former case, we show the description for the Eichhorn data. The latter case also follows. ```{code-cell} ipython3 :tags: [output_scroll] -# Print the full description for the Eichhorn+1970 example. +# Print the full description for the Eichhorn+1970 example. ``` -## Step 2: Acquire the relevant data and make a plot! +## Step 2: Acquire the relevant data and make a plot + In order to query the table, we need the table name, note this is NOT the same as the short name we found above: ```{code-cell} ipython3 @@ -211,36 +211,37 @@ In order to query the table, we need the table name, note this is NOT the same a ``` ```{code-cell} ipython3 -# Query the data to output the table information (i.e. the data in the +# Query the data to output the table information (i.e. the data in the # columns in the table) ``` -We can access the column data as array using the .getcolumn(colname) attribute, where the colname is given in the table above. In particular the "CI" is the color index and "Ptm" is the photovisual magnitude. See [here](https://vizier.u-strasbg.fr/viz-bin/VizieR?-source=I/90) for details about the columns. +We can access the column data as array using the .getcolumn(colname) attribute, where the colname is given in the table above. In particular the "CI" is the color index and "Ptm" is the photovisual magnitude. See [here](https://vizier.u-strasbg.fr/viz-bin/VizieR?-source=I/90) for details about the columns. ```{code-cell} ipython3 # get color and magnitude column information for -# stars in this table. +# stars in this table. ``` -### Plotting... -Note: The magnitudes here are apparent and therefore in plotting, the color-magnitude diagram is typically brightness increasing upwards (higher y-axis) so we will flip the y-axis here. +### Plotting + +Note: The magnitudes here are apparent and therefore in plotting, the color-magnitude diagram is typically brightness increasing upwards (higher y-axis) so we will flip the y-axis here. ```{code-cell} ipython3 plt.ylim(15, 0) plt.ylabel("V [apparent mag]") plt.xlabel("B-V") # -# plot color and magnitude data: +# plot color and magnitude data: ``` -## Step 3. Compare with other color-magnitude diagrams for Pleiades: +## Step 3. Compare with other color-magnitude diagrams for Pleiades -There is nice discussion here: http://www.southastrodel.com/Page03009a.htm about the color-magnitude diagram. Their Fig 4 looks slightly cleaner because part of this investigation was to select the 270 stars that are vetted members and restricted to stellar types more massive than K0. +There is nice discussion here: about the color-magnitude diagram. Their Fig 4 looks slightly cleaner because part of this investigation was to select the 270 stars that are vetted members and restricted to stellar types more massive than K0. The dataset is from Raboud+1998 (1998A&A...329..101R) -Therefore in this next step, we will use the bibcode to select this data and overplot with the previous data to compare. +Therefore in this next step, we will use the bibcode to select this data and overplot with the previous data to compare. ```{code-cell} ipython3 bibcode = '1998A&A...329..101R' # Raboud @@ -261,13 +262,13 @@ plt.ylim(15, 0) plt.ylabel("V [apparent mag]") plt.xlabel("B-V") -# Plot the Eichhorn data as black circles and -# Raboud data as red squares. +# Plot the Eichhorn data as black circles and +# Raboud data as red squares. ``` -## BONUS: Step 4: The CMD as a distance indicator! +## BONUS: Step 4: The CMD as a distance indicator -Since the y-axis above is apparent magnitude, we can use the obvious features (e.g., main sequence curve) to translate the apparent magnitudes to absolute magnitudes (by comparing to published H-R diagrams given in absolute magnitudes) and measure the distance to Pleiades! +Since the y-axis above is apparent magnitude, we can use the obvious features (e.g., main sequence curve) to translate the apparent magnitudes to absolute magnitudes (by comparing to published H-R diagrams given in absolute magnitudes) and measure the distance to Pleiades! ```{code-cell} ipython3 @@ -277,19 +278,19 @@ sun_color = 0.65 # from http://www.astro.ucla.edu/~wright/magcolor.htm # the Sun's apparent magnitude at the distance of the Pleiades? # i.e. if you overplot the Sun on the CMD above # what's the Sun's magnitude, corresponding to the sun's color -# given as B-V=0.65? +# given as B-V=0.65? -# Overplot the sun in the plot above for reference. +# Overplot the sun in the plot above for reference. ``` ```{code-cell} ipython3 # Measure the distance to the Pleaides, using the Sun -# as a reference. We know the Sun's absolute magnitude: +# as a reference. We know the Sun's absolute magnitude: Vabs = 4.8 ## Sun @ B-V = 0.65 (taken from Wikipedia) # Using the Sun's apparent magnitude from your estimate above -# what's the distance to Pleiades in pc? +# what's the distance to Pleiades in pc? ``` -True distance to Pleaides is 136.2 pc (https://en.wikipedia.org/wiki/Pleiades ). Not bad! +True distance to Pleaides is 136.2 pc ( ). Not bad! diff --git a/content/use_case_notebooks/hr_diagram_solution.md b/content/use_case_notebooks/hr_diagram_solution.md index c5f4c1d..af31f17 100644 --- a/content/use_case_notebooks/hr_diagram_solution.md +++ b/content/use_case_notebooks/hr_diagram_solution.md @@ -32,18 +32,18 @@ toc: toc_window_display: true --- -# Creating a stellar color-magnitude (or Hertzsprung-Russell) diagram +# HR (Hertzsprung-Russell) Diagram Solution -The [Hertzsprung-Russell diagram](https://en.wikipedia.org/wiki/Hertzsprung–Russell_diagram) is a fundamental diagram in astronomy that displays important relationships between the stellar color (or temperature) and absolute brightness (or luminosity). +The [Hertzsprung-Russell diagram](https://en.wikipedia.org/wiki/Hertzsprung–Russell_diagram) is a fundamental diagram in astronomy that displays important relationships between the stellar color (or temperature) and absolute brightness (or luminosity). -In this exercise, we will use existing stellar catalogs to produce the H-R diagram. +In this exercise, we will use existing stellar catalogs to produce the H-R diagram. ```{code-cell} ipython3 -# As a hint, we include the code block for Python modules that you will likely need to import: +# As a hint, we include the code block for Python modules that you will likely need to import: import matplotlib import matplotlib.pyplot as plt import numpy as np -%matplotlib inline +%matplotlib inline # For downloading files from astropy.utils.data import download_file @@ -52,7 +52,7 @@ from astropy.io import fits import pyvo as vo from pyvo import registry -## There are a number of relatively unimportant warnings that +## There are a number of relatively unimportant warnings that ## show up, so for now, suppress them: import warnings warnings.filterwarnings("ignore", module="astropy.io.votable.*") @@ -61,13 +61,13 @@ warnings.filterwarnings("ignore", module="pyvo.utils.xml.*") ## Step 1: Find appropriate catalogs -We want to find a star catalog that has the available data to produce the H-R diagram, i.e., the absolute magnitudes (or both apparent magnitudes AND distances, so we can calculate the absolute magnitudes) in two optical bands (e.g., B and V). This would give us color. Or we need B- OR V- band magnitude and the stellar temperature. +We want to find a star catalog that has the available data to produce the H-R diagram, i.e., the absolute magnitudes (or both apparent magnitudes AND distances, so we can calculate the absolute magnitudes) in two optical bands (e.g., B and V). This would give us color. Or we need B- OR V- band magnitude and the stellar temperature. -To simplify this problem, we want to find a catalog of an open cluster of stars, where all the stars were born around the same time and are located in one cluster. This simplifies the issue of getting accurate distances to the stars. One famous cluster is the Pleiades in the constellation of Taurus. So first we start by searching for an existing catalog with data on Pleiades that will provide the necessary information about the stars: magnitudes in two bands (e.g., B and V), which can be used to measure color, or temperature of the star plus one magnitude. +To simplify this problem, we want to find a catalog of an open cluster of stars, where all the stars were born around the same time and are located in one cluster. This simplifies the issue of getting accurate distances to the stars. One famous cluster is the Pleiades in the constellation of Taurus. So first we start by searching for an existing catalog with data on Pleiades that will provide the necessary information about the stars: magnitudes in two bands (e.g., B and V), which can be used to measure color, or temperature of the star plus one magnitude. +++ -### DATA DISCOVERY STEPS: +### DATA DISCOVERY STEPS Here is useful link for [how the pyvo registry search works](https://pyvo.readthedocs.io/en/latest/registry/index.html). @@ -79,31 +79,31 @@ print(len(tap_services)) tap_services.get_summary() ``` -Note: The includeaux=True includes auxiliary services. +Note: The includeaux=True includes auxiliary services. -Because we specified the service type, this returns only the TAP services for each of the avalible resorces matching our search criteria. So, the 'interfaces' column will only show TAP service as an avalible service, even if other services are avalible from the same resource. We know we need the service to be a TAP service since we know that we want to eventually access the search using additional column information (i.e., beyond RA and Dec, which is all that Simple Cone Search returns). +Because we specified the service type, this returns only the TAP services for each of the avalible resorces matching our search criteria. So, the 'interfaces' column will only show TAP service as an avalible service, even if other services are avalible from the same resource. We know we need the service to be a TAP service since we know that we want to eventually access the search using additional column information (i.e., beyond RA and Dec, which is all that Simple Cone Search returns). +++ -#### Next, we need to find which of these has the columns of interest, i.e. magnitudes in two bands to create the color-magnitude diagram. +#### Next, we need to find which of these has the columns of interest, i.e. magnitudes in two bands to create the color-magnitude diagram We can re-run the registry search, but further restrict the results by column UCD. We want tables that have magnitude columns; the most basic UCD to describe a magnitude column is phot.mag ```{code-cell} ipython3 :tags: [output_scroll] -tap_services = registry.search(servicetype='tap', keywords=['star pleiades'], +tap_services = registry.search(servicetype='tap', keywords=['star pleiades'], ucd = ['phot.mag%'], includeaux=True) print(len(tap_services)) ``` Note: the '%' serves as a wild card when searching by UCD -The IOVA standard enables resources to be as sepecific as they would like when defining the UCD of columns. For example, 'phot.mag' and 'phot.mag;em.opt.V' can both be used to describe a column containing the V magnitudes of objects. If a resource uses the latter to describe a column, a search using 'phot.mag' will not return that resource. A wild card would need to be used or the exact column UCD. The UCD search requires an exact match for a resource to be returned, so using the wild card will make it easier to discover a wider variety of resources. +The IOVA standard enables resources to be as sepecific as they would like when defining the UCD of columns. For example, 'phot.mag' and 'phot.mag;em.opt.V' can both be used to describe a column containing the V magnitudes of objects. If a resource uses the latter to describe a column, a search using 'phot.mag' will not return that resource. A wild card would need to be used or the exact column UCD. The UCD search requires an exact match for a resource to be returned, so using the wild card will make it easier to discover a wider variety of resources. +++ {"tags": ["output_scroll"]} -So using this we can reduce the matched tables to ones that are a bit more catered to our experiment. Note, that there can be redundancy in some resources since these are available via multiple services and/or publishers. Therefore a bit more cleaning can be done to provide only the unique matches. +So using this we can reduce the matched tables to ones that are a bit more catered to our experiment. Note, that there can be redundancy in some resources since these are available via multiple services and/or publishers. Therefore a bit more cleaning can be done to provide only the unique matches. ```{code-cell} ipython3 :tags: [output_scroll] @@ -118,13 +118,13 @@ def getunique( result ): short_name = [] unique_ind = [] for i in range(len(result)): - short = result[i].short_name - if short not in short_name: + short = result[i].short_name + if short not in short_name: short_name.append(short) unique_ind.append(i) else: print(i) - + return(unique_ind) ``` @@ -143,7 +143,7 @@ tap_services.to_table()[uniq_ind]['ivoid'] +++ {"tags": ["output_scroll"]} -This shows that in this case, all of our TAP results are unique. +This shows that in this case, all of our TAP results are unique. +++ @@ -154,50 +154,50 @@ We can read more information about the results we found. For each resource elem # To read the descriptions of the resulting matches: -for i in uniq_ind: +for i in uniq_ind: print(" *** \n") print(tap_services[i].creators) print(tap_services[i].res_description) - - + + ``` +++ {"tags": ["output_scroll"]} RESULT: Based on these, the second one (by Eichhorn et al) looks like a good start. -### At this point, you can proceed to Step 2. +### At this point, you can proceed to Step 2 -- OR -- -### Try a different data discovery method! +### Try a different data discovery method +++ {"tags": ["output_scroll"]} -### Alternative Method: Use ADS to search for appropriate paper and access data via NED. +### Alternative Method: Use ADS to search for appropriate paper and access data via NED -There are multiple paths for the data discovery. So it may also be that you know the paper that has the data you are interested in and want to access via the bibcode or authors, etc. +There are multiple paths for the data discovery. So it may also be that you know the paper that has the data you are interested in and want to access via the bibcode or authors, etc. -In this case, let's assume that we have the information that the Eichhorn+1970 paper has the data that we need to create the H-R diagram: https://ui.adsabs.harvard.edu/abs/1970MmRAS..73..125E/abstract +In this case, let's assume that we have the information that the Eichhorn+1970 paper has the data that we need to create the H-R diagram: -We can either search by bibcode (1970MmRAS..73..125E) or "Eichhorn" to get the access_urls that will allow us to work with the data. +We can either search by bibcode (1970MmRAS..73..125E) or "Eichhorn" to get the access_urls that will allow us to work with the data. -Before this step, if may help to see the names of the fields available to use. Notice the following fields: +Before this step, if may help to see the names of the fields available to use. Notice the following fields: -"source_value" contains the bibcode information that we want; "creator_seq" lists the authors; +"source_value" contains the bibcode information that we want; "creator_seq" lists the authors; -and +and -"access_url" provides the url from where the data can be accessed. +"access_url" provides the url from where the data can be accessed. ```{code-cell} ipython3 ## You already have this from above: -# tap_services = registry.search(servicetype='tap', keywords=['star pleiades'], +# tap_services = registry.search(servicetype='tap', keywords=['star pleiades'], # ucd = ['phot.mag%'], includeaux=True) print(tap_services.fieldnames) ``` -First, Try using bibcode: +First, Try using bibcode: ```{code-cell} ipython3 bibcode = '1970MmRAS..73..125E' # Eichhorn @@ -207,11 +207,11 @@ for s in tap_services: if bibcode in s['source_value']: myidx=idx print(f"{s.short_name}, {s.source_value}, {s.access_url}") - break -# Note that using the to_table() lets you search the result + break +# Note that using the to_table() lets you search the result # easily using all columns. But in the end, you want to get # back not an astropy table row, which you cannot use, but the -# original RegistryResult that has the callable TAP service. +# original RegistryResult that has the callable TAP service. myTAP=tap_services[myidx] ``` @@ -219,42 +219,40 @@ Note that the URL is a generic TAP url for Vizier. All of its tables can be acc +++ -Next, try using Author name: +Next, try using Author name: ```{code-cell} ipython3 :tags: [output_scroll] author = 'Eichhorn' -for record in tap_services: +for record in tap_services: names=record.creators - if 'Eichhorn' in names[0]: + if 'Eichhorn' in names[0]: print("For %s: " %record.short_name) - print(" Access URL: %s" %record['access_urls'][0]) + print(" Access URL: %s" %record['access_urls'][0]) print(" Reference URL: %s" %record.reference_url) ``` -In the code above, the record is a Registry Resource. You can access the attribute, "creators", from the resource, which is relevant for our example here since this is a direct way to get the author names. The other attributes, "access_url" and "reference_url", provides two types of URLs. The former can be used to access the service resource (as described above) and the latter points to a human-readable document describing this resource. +In the code above, the record is a Registry Resource. You can access the attribute, "creators", from the resource, which is relevant for our example here since this is a direct way to get the author names. The other attributes, "access_url" and "reference_url", provides two types of URLs. The former can be used to access the service resource (as described above) and the latter points to a human-readable document describing this resource. If you click on the Reference URL links, you can find the abstract, Readme file, Vizier table, and much more information associated with this catalog. -Additional information about PyVO Registry Resource and its attributes is available from this page. - +Additional information about PyVO Registry Resource and its attributes is available from this page.

***

+++ -These examples provide a few ways to access the information of interest. +These examples provide a few ways to access the information of interest. -Below are a few other ways to see what the tap_service table contains. +Below are a few other ways to see what the tap_service table contains. -1. To view the column information: tap_services.to_table().columns() shows the metadata contained in the tap service. We will reference some of this columns below as we try to find the appropriate table. +1. To view the column information: tap_services.to_table().columns() shows the metadata contained in the tap service. We will reference some of this columns below as we try to find the appropriate table. -2. tap_services[index].describe(): The table with the tap_services output has, in our case, 83 tables listed and each includes metadata containing some human readable description. You can get the description for one case or for all the records by iterating through the resource. In the former case, we show the description for the Eichhorn data, whose index is uniq_ind[1]. The latter case also follows. - +2. tap_services[index].describe(): The table with the tap_services output has, in our case, 83 tables listed and each includes metadata containing some human readable description. You can get the description for one case or for all the records by iterating through the resource. In the former case, we show the description for the Eichhorn data, whose index is uniq_ind[1]. The latter case also follows. ```{code-cell} ipython3 :tags: [output_scroll] @@ -265,21 +263,22 @@ print( tap_services.to_table().columns ) ```{code-cell} ipython3 :tags: [output_scroll] -tap_services[uniq_ind[1]].describe() # For Eichhorm+1970 example. +tap_services[uniq_ind[1]].describe() # For Eichhorm+1970 example. ``` ```{code-cell} ipython3 :tags: [output_scroll] -# To iterate over all the tables: -for tapsvc in tap_services: +# To iterate over all the tables: +for tapsvc in tap_services: print("--------------------------------------------- \n") tapsvc.describe() ``` +++ {"tags": ["output_scroll"]} -## Step 2: Acquire the relevant data and make a plot! +## Step 2: Acquire the relevant data and make a plot + In order to query the table, we need the table name, note this is NOT the same as the short name we found above: ```{code-cell} ipython3 @@ -288,24 +287,24 @@ In order to query the table, we need the table name, note this is NOT the same a tables = tap_services[uniq_ind[1]].service.tables short_name = "I/90" -# find table name: +# find table name: for name in tables.keys(): - if short_name in name: + if short_name in name: print(name) ``` +++ {"tags": ["output_scroll"]} -We can write code to eliminate the other cases (e.g., VI or VIII...) but we wanted to keep this cell to illustrate that the table name (which is required for the query) will likely include the short_name appended to "/catalog" (or "/table"). +We can write code to eliminate the other cases (e.g., VI or VIII...) but we wanted to keep this cell to illustrate that the table name (which is required for the query) will likely include the short_name appended to "/catalog" (or "/table"). -But the other roman numeral catalogs are obviously different catalogs. Therefore try the below for a better match: +But the other roman numeral catalogs are obviously different catalogs. Therefore try the below for a better match: ```{code-cell} ipython3 :tags: [output_scroll] -# find (more restricted) table name: +# find (more restricted) table name: for name in tables.keys(): - if name.startswith(short_name): + if name.startswith(short_name): print(name) tablename=name ``` @@ -319,15 +318,16 @@ results = tap_services[short_name].search(query) results.to_table() ``` -We can access the column data as array using the .getcolumn(colname) attribute, where the colname is given in the table above. In particular the "CI" is the color index and "Ptm" is the photovisual magnitude. See [here](https://vizier.u-strasbg.fr/viz-bin/VizieR?-source=I/90) for details about the columns. +We can access the column data as array using the .getcolumn(colname) attribute, where the colname is given in the table above. In particular the "CI" is the color index and "Ptm" is the photovisual magnitude. See [here](https://vizier.u-strasbg.fr/viz-bin/VizieR?-source=I/90) for details about the columns. ```{code-cell} ipython3 color = results.getcolumn('CI') -mag = results.getcolumn('Ptm') +mag = results.getcolumn('Ptm') ``` -### Plotting... -Note: The magnitudes here are apparent and therefore in plotting, the color-magnitude diagram is typically brightness increasing upwards (higher y-axis) so we will flip the y-axis here. +### Plotting + +Note: The magnitudes here are apparent and therefore in plotting, the color-magnitude diagram is typically brightness increasing upwards (higher y-axis) so we will flip the y-axis here. ```{code-cell} ipython3 plt.ylim(15, 0) @@ -337,13 +337,13 @@ plt.xlabel("B-V") plt.plot(color, mag, 'o', color='black') ``` -## Step 3. Compare with other color-magnitude diagrams for Pleiades: +## Step 3. Compare with other color-magnitude diagrams for Pleiades -There is nice discussion here: http://www.southastrodel.com/Page03009a.htm about the color-magnitude diagram. Their Fig 4 looks slightly cleaner because part of this investigation was to select the 270 stars that are vetted members and restricted to stellar types more massive than K0. +There is nice discussion here: about the color-magnitude diagram. Their Fig 4 looks slightly cleaner because part of this investigation was to select the 270 stars that are vetted members and restricted to stellar types more massive than K0. The dataset is from Raboud+1998 (1998A&A...329..101R) -Therefore in this next step, we will use the bibcode to select this data and overplot with the previous data to compare. +Therefore in this next step, we will use the bibcode to select this data and overplot with the previous data to compare. ```{code-cell} ipython3 bibcode = '1998A&A...329..101R' # Raboud @@ -352,7 +352,7 @@ all_shortnames = tap_services.getcolumn('short_name') match = np.where(all_bibcodes == bibcode) -# Show relevant short_name (for Raboud paper): +# Show relevant short_name (for Raboud paper): short_name = all_shortnames[match][0] print(short_name) print("----------") @@ -366,21 +366,21 @@ tap_services[ind].describe() # Doing steps above to view table from Raboud+1998 -# This 'tables' will return the same service tables as the 'tables' defined -# ealier in Step 2, since both the Eichhorn+1970 and Raboud+1998 come from the +# This 'tables' will return the same service tables as the 'tables' defined +# ealier in Step 2, since both the Eichhorn+1970 and Raboud+1998 come from the # same service. Therfore, we did not need to redefine 'tables', but we kept it -# for completion, as different tables don't always use the same service. -tables = tap_services[ind].service.tables +# for completion, as different tables don't always use the same service. +tables = tap_services[ind].service.tables -# find table name: +# find table name: for name in tables.keys(): - if short_name in name: + if short_name in name: tablename = name - + # Another way to find the table name: # Raboud_table = tap_services[short_name].get_tables() # tablename = [name for name in Raboud_table.keys()][0] - + query = 'SELECT * FROM "%s"' %tablename print(query) results = tap_services[ind].search(query) @@ -389,7 +389,7 @@ results.to_table() ```{code-cell} ipython3 R98_color = results.getcolumn('B-V') -R98_mag = results.getcolumn('Vmag') +R98_mag = results.getcolumn('Vmag') plt.ylim(15, 0) plt.ylabel("V [apparent mag]") @@ -398,16 +398,16 @@ plt.plot(color, mag, 'o', markersize=4.0, color='black') ## This is Eichhorn dat plt.plot(R98_color, R98_mag, 's', markersize=5.0, color='red') ## This is new data from Raboud+98 ``` -## BONUS: Step 4: The CMD as a distance indicator! +## BONUS: Step 4: The CMD as a distance indicator -Since the y-axis above is apparent magnitude, we can use the obvious features (e.g., main sequence curve) to translate the apparent magnitudes to absolute magnitudes (by comparing to published H-R diagrams given in absolute magnitudes) and measure the distance to Pleiades! +Since the y-axis above is apparent magnitude, we can use the obvious features (e.g., main sequence curve) to translate the apparent magnitudes to absolute magnitudes (by comparing to published H-R diagrams given in absolute magnitudes) and measure the distance to Pleiades! ```{code-cell} ipython3 R98_color = results.getcolumn('B-V') -R98_mag = results.getcolumn('Vmag') +R98_mag = results.getcolumn('Vmag') sun_color = 0.65 # from http://www.astro.ucla.edu/~wright/magcolor.htm -sun_mag = 10.4 # Played with this value until it looked centered in relation at the B-V color above (yellow star!) +sun_mag = 10.4 # Played with this value until it looked centered in relation at the B-V color above (yellow star!) plt.ylim(15, 0) plt.ylabel("V [apparent mag]") @@ -418,13 +418,13 @@ plt.plot(sun_color, sun_mag, '*', markersize=15.0, color='yellow') ## This is ou ``` ```{code-cell} ipython3 -# Another measure... use the Sun: +# Another measure... use the Sun: Vabs = 4.8 ## Sun @ B-V = 0.65 (taken from Wikipedia) Vapp = 10.4 ## Based on rough reading of plot above at B-V = 0.65 -dm= Vapp - Vabs # distance module = 5log d / 10pc. +dm= Vapp - Vabs # distance module = 5log d / 10pc. dist = 10. ** (dm / 5. + 1.) print("%10.1f pc " %dist) ``` -True distance to Pleaides is 136.2 pc ( https://en.wikipedia.org/wiki/Pleiades ). Not bad! +True distance to Pleaides is 136.2 pc ( ). Not bad! diff --git a/content/use_case_notebooks/proposal_prep_exercise.md b/content/use_case_notebooks/proposal_prep_exercise.md index a19ef2a..986f4a1 100644 --- a/content/use_case_notebooks/proposal_prep_exercise.md +++ b/content/use_case_notebooks/proposal_prep_exercise.md @@ -36,18 +36,18 @@ toc: toc_window_display: true --- -# Preparing a proposal +# Proposal Preparation Exercise -The Story: Suppose that you are preparing to write a proposal on NGC1365, aiming to investigate the intriguing black hole spin this galaxy with Chandra grating observations (see: https://www.space.com/19980-monster-black-hole-spin-discovery.html ) +The Story: Suppose that you are preparing to write a proposal on NGC1365, aiming to investigate the intriguing black hole spin this galaxy with Chandra grating observations (see: ) -In writing proposals, there are often the same tasks that are required: including finding and analyzing previous observations of the proposal, and creating figures that include, e.g., multiwavelength images and spectrum for the source. +In writing proposals, there are often the same tasks that are required: including finding and analyzing previous observations of the proposal, and creating figures that include, e.g., multiwavelength images and spectrum for the source. ```{code-cell} ipython3 -# As a hint, we include the code block for Python modules that you will likely need to import: +# As a hint, we include the code block for Python modules that you will likely need to import: import matplotlib import matplotlib.pyplot as plt import numpy as np -%matplotlib inline +%matplotlib inline # For downloading files from astropy.utils.data import download_file @@ -55,52 +55,53 @@ from astropy.io import fits import pyvo as vo -## There are a number of relatively unimportant warnings that +## There are a number of relatively unimportant warnings that ## show up, so for now, suppress them: import warnings warnings.filterwarnings("ignore", module="astropy.io.votable.*") warnings.filterwarnings("ignore", module="pyvo.utils.xml.*") ``` -## Step 1: Find out what the previously quoted Chandra 2-10 keV flux of the central source is for NGC 1365. +## Step 1: Find out what the previously quoted Chandra 2-10 keV flux of the central source is for NGC 1365 Hint: Do a Registry search for tables served by the HEASARC (where high energy data are archived) to find potential table with this information ```{code-cell} ipython3 # Start with a Registry query to find table services for the HEASARC. # Then get the list of tables that this service serves. -# +# # Hint: the QuickReference has this example: #services = vo.regsearch(servicetype='tap', keywords=['heasarc']) -#tables = services[0].service.tables +#tables = services[0].service.tables # # Hint2: the QuickReference also has this example: #for c in tables['zcat'].columns: # print(f'{c.name:30s} - {c.description}') ``` -Hint: The Chansngcat ( https://heasarc.gsfc.nasa.gov/W3Browse/chandra/chansngcat.html ) table is likely the best table. Create a table with ra, dec, exposure time, and flux (and flux errors) from the public.chansngcat catalog for Chandra observations matched within 0.1 degree. +Hint: The Chansngcat ( ) table is likely the best table. Create a table with ra, dec, exposure time, and flux (and flux errors) from the public.chansngcat catalog for Chandra observations matched within 0.1 degree. ```{code-cell} ipython3 -# Get the coordinate for NGC 1365 with astropy. +# Get the coordinate for NGC 1365 with astropy. ``` ```{code-cell} ipython3 -# Construct a query that will get the ra, dec, exposure time, flux, and flux errors -# from this catalog in the region around this source and submit the query. +# Construct a query that will get the ra, dec, exposure time, flux, and flux errors +# from this catalog in the region around this source and submit the query. # (See the CS_Catalog_queries.md ) # Hint: the QuickReference has this example: #coord = SkyCoord.from_name("m83") #query = f''' -#SELECT ra, dec, Radial_Velocity, radial_velocity_error, bmag, morph_type FROM public.zcat as cat where +#SELECT ra, dec, Radial_Velocity, radial_velocity_error, bmag, morph_type FROM public.zcat as cat where #contains(point('ICRS',cat.ra,cat.dec),circle('ICRS',{coord.ra.deg},{coord.dec.deg},1.0))=1 #''' #results = services[0].service.run_async(query) ``` -## Step 2: Make Images +## Step 2: Make Images ### Create ultraviolet and X-ray images + Hint: Start by checking what UV image services exist (e.g., GALEX?) ```{code-cell} ipython3 @@ -116,10 +117,10 @@ The keyword search for 'galex' returned a bunch of things that may have mentione Though using the result as an Astropy Table makes it easier to look at the contents, to call the service itself, we cannot use the row of that table. You have to use the entry in the service result list itself. So use the table to browse, but select the list of services itself using the properties that have been defined as attributes such as short_name and ivoid: ```{code-cell} ipython3 -# You may find more than one service. Look at both. +# You may find more than one service. Look at both. ``` -Hint: Next create a UV image for the source +Hint: Next create a UV image for the source ```{code-cell} ipython3 # Do an image search for NGC 1365 in the UV services found above @@ -129,21 +130,22 @@ Hint: Next create a UV image for the source ``` ```{code-cell} ipython3 -# Get a FITS file and visualize the image +# Get a FITS file and visualize the image # # Hint: the QuickReference has this example: -#file_name = download_file(results[0].getdataurl()) +#file_name = download_file(results[0].getdataurl()) ``` -Hint: Repeat steps for X-ray image. (Note: Ideally, we would find an image in the Chandra 'cxc' catalog) +Hint: Repeat steps for X-ray image. (Note: Ideally, we would find an image in the Chandra 'cxc' catalog) ```{code-cell} ipython3 ``` -## Step 3: Make a spectrum +## Step 3: Make a spectrum + +### Find what Chandra spectral observations exist already for this source -### Find what Chandra spectral observations exist already for this source. Hint: try searching for X-ray spectral data tables using the registry query ```{code-cell} ipython3 @@ -156,7 +158,7 @@ Hint 2: Take a look at what data exist for our candidate, NGC 1365. ``` -Hint 3: Download the data to make a spectrum. Note: you might end here and use Xspec to plot and model the spectrum. Or ... you can also try to take a quick look at the spectrum. +Hint 3: Download the data to make a spectrum. Note: you might end here and use Xspec to plot and model the spectrum. Or ... you can also try to take a quick look at the spectrum. ```{code-cell} ipython3 # Get it and look at it: @@ -169,10 +171,10 @@ Hint 3: Download the data to make a spectrum. Note: you might end here and use X Extension: Making a "quick look" spectrum. For our purposes, the 1st order of the HEG grating data would be sufficient. ```{code-cell} ipython3 -# Hint: You'll have to look into the details of the spectra. +# Hint: You'll have to look into the details of the spectra. ``` -This can then be analyzed in your favorite spectral analysis tool, e.g., [pyXspec](https://heasarc.gsfc.nasa.gov/xanadu/xspec/python/html/index.html). (For the winter 2018 AAS workshop, we demonstrated this in a [notebook](https://github.com/NASA-NAVO/aas_workshop_2018/blob/master/heasarc/heasarc_Spectral_Access.md) that you can consult for how to use pyXspec, but the pyXspec documentation will have more information.) +This can then be analyzed in your favorite spectral analysis tool, e.g., [pyXspec](https://heasarc.gsfc.nasa.gov/xanadu/xspec/python/html/index.html). (For the winter 2018 AAS workshop, we demonstrated this in a [notebook](https://github.com/NASA-NAVO/aas_workshop_2018/blob/master/heasarc/heasarc_Spectral_Access.md) that you can consult for how to use pyXspec, but the pyXspec documentation will have more information.) +++ diff --git a/content/use_case_notebooks/proposal_prep_solution.md b/content/use_case_notebooks/proposal_prep_solution.md index da865f9..a00bcf2 100644 --- a/content/use_case_notebooks/proposal_prep_solution.md +++ b/content/use_case_notebooks/proposal_prep_solution.md @@ -39,18 +39,18 @@ widgets: version: 1.1.1 --- -# Preparing a proposal +# Proposal Preparation Solution -The Story: Suppose that you are preparing to write a proposal on NGC1365, aiming to investigate the intriguing black hole spin this galaxy with Chandra grating observations (see: [Monster Blackhole Spin Revealed](https://www.space.com/19980-monster-black-hole-spin-discovery.html)) +The Story: Suppose that you are preparing to write a proposal on NGC1365, aiming to investigate the intriguing black hole spin this galaxy with Chandra grating observations (see: [Monster Blackhole Spin Revealed](https://www.space.com/19980-monster-black-hole-spin-discovery.html)) -In writing proposals, there are often the same tasks that are required: including finding and analyzing previous observations of the proposal, and creating figures that include, e.g., multiwavelength images and spectrum for the source. +In writing proposals, there are often the same tasks that are required: including finding and analyzing previous observations of the proposal, and creating figures that include, e.g., multiwavelength images and spectrum for the source. ```{code-cell} ipython3 -# As a hint, we include the code block for Python modules that you will likely need to import: +# As a hint, we include the code block for Python modules that you will likely need to import: import matplotlib import matplotlib.pyplot as plt import numpy as np -%matplotlib inline +%matplotlib inline # For downloading files from astropy.utils.data import download_file @@ -58,14 +58,14 @@ from astropy.io import fits import pyvo as vo -## There are a number of relatively unimportant warnings that +## There are a number of relatively unimportant warnings that ## show up, so for now, suppress them: import warnings warnings.filterwarnings("ignore", module="astropy.io.votable.*") warnings.filterwarnings("ignore", module="pyvo.utils.xml.*") ``` -## Step 1: Find out what the previously quoted Chandra 2-10 keV flux of the central source is for NGC 1365. +## Step 1: Find out what the previously quoted Chandra 2-10 keV flux of the central source is for NGC 1365 Hint: Do a Registry search for tables served by the HEASARC (where high energy data are archived) to find potential table with this information @@ -80,7 +80,7 @@ Hint: The [Chansngcat](https://heasarc.gsfc.nasa.gov/W3Browse/chandra/chansngcat ```{code-cell} ipython3 for tablename in heasarc_tables.keys(): - if "chansng" in tablename: + if "chansng" in tablename: print("Table {} has columns={}\n".format( tablename, sorted([k.name for k in heasarc_tables[tablename].columns ]))) @@ -93,10 +93,10 @@ pos=coord.SkyCoord.from_name("ngc1365") ``` ```{code-cell} ipython3 -# Construct a query that will get the ra, dec, exposure time, flux, and flux errors +# Construct a query that will get the ra, dec, exposure time, flux, and flux errors # from this catalog in the region around this source: -query="""SELECT ra, dec, exposure, flux, flux_lower, flux_upper FROM public.chansngcat as cat - where contains(point('ICRS',cat.ra,cat.dec),circle('ICRS',{},{},0.1))=1 +query="""SELECT ra, dec, exposure, flux, flux_lower, flux_upper FROM public.chansngcat as cat + where contains(point('ICRS',cat.ra,cat.dec),circle('ICRS',{},{},0.1))=1 and cat.exposure > 0 order by cat.exposure""".format(pos.ra.deg, pos.dec.deg) # Submit the query. (See the CS_Catalog_queries.md for # information about these two search options.) @@ -106,9 +106,10 @@ results=tap_services[0].service.run_async(query) results.to_table() ``` -## Step 2: Make Images: +## Step 2: Make Images ### Create ultraviolet and X-ray images + Hint: Start by checking what UV image services exist (e.g., GALEX?) ```{code-cell} ipython3 @@ -132,7 +133,7 @@ galex_stsci=[s for s in uv_services if 'GALEX' in s.short_name and 'stsci' in s. galex_heasarc=[s for s in uv_services if 'GALEX' in s.short_name and 'heasarc' in s.ivoid][0] ``` -Hint: Next create a UV image for the source +Hint: Next create a UV image for the source ```{code-cell} ipython3 # Do an image search for NGC 1365 in the UV service found above @@ -141,7 +142,7 @@ im_table_stsci.to_table() ``` ```{code-cell} ipython3 -# Let's see what HEASARC offers, and this time limit it to FITS +# Let's see what HEASARC offers, and this time limit it to FITS # this option doesn't currently work for STScI's service) im_table_heasarc=galex_heasarc.search(pos=pos,size=0.1,format='image/fits') im_table_heasarc.to_table() @@ -151,8 +152,8 @@ im_table_heasarc.to_table() ## If you only run this once, you can do it in memory in one line: ## This fetches the FITS as an astropy.io.fits object in memory #dataobj=im_table_heasarc[0].getdataobj() -## But if you might run this notebook repeatedly with limited bandwidth, -## download it once and cache it. +## But if you might run this notebook repeatedly with limited bandwidth, +## download it once and cache it. file_name = download_file(im_table_heasarc[0].getdataurl(), cache=True, timeout=600) dataobj=fits.open(file_name) print(type(dataobj)) @@ -165,7 +166,7 @@ from matplotlib.colors import LogNorm plt.matshow(dataobj[0].data, origin='lower', cmap=cm.gray_r, norm=LogNorm(vmin=0.005, vmax=0.3)) ``` -Hint: Repeat steps for X-ray image. (Note: Ideally, we would find an image in the Chandra 'cxc' catalog) +Hint: Repeat steps for X-ray image. (Note: Ideally, we would find an image in the Chandra 'cxc' catalog) ```{code-cell} ipython3 x_services=vo.regsearch(servicetype='image',keywords=['chandra'], waveband='x-ray') @@ -177,7 +178,7 @@ print(x_services.to_table()['short_name','ivoid']) xim_table=x_services[0].search(pos=pos,size=0.2) ## Some of these are FITS and some JPEG. Look at the columns: print( xim_table.to_table().columns ) -first_fits_image_row = [x for x in xim_table if 'image/fits' in x.format][0] +first_fits_image_row = [x for x in xim_table if 'image/fits' in x.format][0] ``` ```{code-cell} ipython3 @@ -193,9 +194,10 @@ plt.xlim(460, 560) plt.ylim(460, 560) ``` -## Step 3: Make a spectrum: +## Step 3: Make a spectrum + +### Find what Chandra spectral observations exist already for this source -### Find what Chandra spectral observations exist already for this source. Hint: try searching for X-ray spectral data tables using the registry query ```{code-cell} ipython3 @@ -211,7 +213,7 @@ spec_tables=xsp_services[0].search(pos=pos,radius=0.2,verbose=True) spec_tables.to_table() ``` -Hint 3: Download the data to make a spectrum. Note: you might end here and use Xspec to plot and model the spectrum. Or ... you can also try to take a quick look at the spectrum. +Hint 3: Download the data to make a spectrum. Note: you might end here and use Xspec to plot and model the spectrum. Or ... you can also try to take a quick look at the spectrum. ```{code-cell} ipython3 # Get it and look at it: @@ -253,7 +255,7 @@ for i in range(len(spectra)): j=j+1 ``` -This can then be analyzed in your favorite spectral analysis tool, e.g., [pyXspec](https://heasarc.gsfc.nasa.gov/xanadu/xspec/python/html/index.html). (For the winter 2018 AAS workshop, we demonstrated this in a [notebook](https://github.com/NASA-NAVO/aas_workshop_2018/blob/master/heasarc/heasarc_Spectral_Access.md) that you can consult for how to use pyXspec, but the pyXspec documentation will have more information.) +This can then be analyzed in your favorite spectral analysis tool, e.g., [pyXspec](https://heasarc.gsfc.nasa.gov/xanadu/xspec/python/html/index.html). (For the winter 2018 AAS workshop, we demonstrated this in a [notebook](https://github.com/NASA-NAVO/aas_workshop_2018/blob/master/heasarc/heasarc_Spectral_Access.md) that you can consult for how to use pyXspec, but the pyXspec documentation will have more information.) +++ diff --git a/index.md b/index.md index 8c7d8ca..2ffc05c 100644 --- a/index.md +++ b/index.md @@ -1,28 +1,46 @@ # NASA-NAVO notebooks -## Content +## Reference Notebooks ```{toctree} --- maxdepth: 2 --- -content/reference_notebooks/catalog_queries +content/reference_notebooks/basic_reference content/reference_notebooks/image_access +content/reference_notebooks/catalog_queries content/reference_notebooks/spectral_access -content/reference_notebooks/ucds_unified_content_descriptors content/reference_notebooks/votables +content/reference_notebooks/ucds_unified_content_descriptors + +``` + +## Use Case Exercises + +```{toctree} +--- +maxdepth: 2 +--- + content/use_case_notebooks/candidate_list_exercise content/use_case_notebooks/proposal_prep_exercise content/use_case_notebooks/hr_diagram_exercise -content/reference_notebooks/basic_reference +``` + +## Use Case Solutions + +```{toctree} +--- +maxdepth: 2 +--- + content/use_case_notebooks/candidate_list_solution content/use_case_notebooks/proposal_prep_solution content/use_case_notebooks/hr_diagram_solution ``` - -## Additional resources +## Additional Resources ```{toctree} --- @@ -31,4 +49,4 @@ maxdepth: 1 00_SETUP KNOWN_ISSUES -``` \ No newline at end of file +```