-

Accessing astronomical catalogs#

+

Accessing astronomical catalogs#

There are two ways to access astronomical data catalogs that are provided as table data with a VO API.

First, there is a Simple Cone Search (SCS) protocol used to search a given table with a given position and radius, getting back a table of results. The interface requires only a position and search radius.

For more complicated searches, the Table Access Protocol (TAP) protocol is a powerful tool to search any VO table. Here, we expand on its usage and that of the Astronomical Data Query Language (ADQL) that it uses.

-

-
+ +
# suppress some specific warnings that are not important
 import warnings
@@ -480,7 +518,6 @@ 
Accessing  astronomical catalogs
-

-

3. Using the TAP to cross-correlate and combine#

+

3. Using the TAP to cross-correlate and combine#

-

3.1 Cross-correlating to combine catalogs#

+

3.1 Cross-correlating to combine catalogs#

TAP can also be a powerful way to collect a lot of useful information from existing catalogs in one quick step. For this exercise, we will start with a list of sources, uploaded from our own table, and do a ‘cross-correlation’ with the zcat table.

-

For more on creating and working with VO tables, see that notebook. Here, we just read one in that’s already prepared:

+

For more on creating and working with VO tables, see that notebook. Here, we just read one in that’s already prepared:

First, check that this service can handle uploaded tables. Not all do.

@@ -938,7 +966,7 @@

3.1 Cross-correlating to combine catalogs 
query="""
     SELECT cat.ra, cat.dec, Radial_Velocity, bmag, morph_type
-    FROM zcat cat, tap_upload.mysources mt 
+    FROM zcat cat, tap_upload.mysources mt
     WHERE
     contains(point('ICRS',cat.ra,cat.dec),circle('ICRS',mt.ra,mt.dec,0.01))=1
     and Radial_Velocity > 0
@@ -952,30 +980,30 @@ 
3.1  Cross-correlating to combine catalogs
 
Table length=14
-
+

+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+

 
radecradial_velocitybmagmorph_type
degdeg

 
float64float64int32float32int16
136.0007437121.96791867309313.820
146.7033430822.01827217744614.6-1
146.7033430822.01827217759715.0-2
148.7780563114.29613296719415.12
175.0394016215.32725392332514.43
191.5419864330.73226569665115.020
191.5473991130.72338192651714.7-2
194.9129447528.89537435609315.00
206.5716382343.85050581222915.0-1
206.5799642643.843857072586418.5--
209.9626024438.18183343275914.93
213.5560119915.62222407464414.03
219.9664999742.74233535251714.720
333.8216972537.29922141520715.31
136.0007421.96792309313.8020
146.7033422.01827744614.60-1
146.7033422.01827759715.00-2
148.7780614.29613719415.102
175.0394015.32725332514.403
191.5419930.73227665115.0020
191.5474030.72338651714.70-2
194.9129428.89537609315.000
206.5716443.85051222915.00-1
206.5799643.843862586418.50--
209.9626038.18183275914.903
213.5560115.62222464414.003
219.9665042.74234251714.7020
333.8217037.29922520715.301

Therefore we now have the Bmag, morphological type and radial velocities for all the sources in our list with a single TAP query.

-

-

3.2 Cross-correlating with user-defined columns#

+

3.2 Cross-correlating with user-defined columns#

Our input list of sources contains galaxy pair candidates that may be interacting with each other. Therefore it would be interesting to know what the morphological type and the Bmagnitude are for the potential companions.

In this advanced example, we want our search to be physically motivated since the criterion for galaxy interaction depends on the physical separation of the galaxies. Unlike the previous case, the search radius is not a constant, but varies for each candidate by the distance to the source. Specifically, we want to search for companions that are within 50 kpc of the candidate and therefore first need to find the angular diameter distance that corresponds to galaxy’s distance (in our case the radial velocity).

Therefore, we begin by taking our table of objects and adding an angDdeg column:

@@ -983,8 +1011,8 @@

3.2 Cross-correlating with user-defined columns 
## The column 'radial_velocity' is c*z but doesn't include the unit; it is km/s
 ## Get the speed of light from astropy.constants and express in km/s
-c = const.c.to(u.km/u.s).value 
-redshifts = mytable['radial_velocity']/c 
+c = const.c.to(u.km/u.s).value
+redshifts = mytable['radial_velocity']/c
 mytable['redshift'] = redshifts
 physdist = 0.05*u.Mpc # 50 kpc physical distance
 
@@ -997,37 +1025,37 @@ 
3.2  Cross-correlating with user-defined columns
 
Table length=14
-
+

-
+
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+
+
+
+
+
+

 
radecradial_velocitybmagmorph_typeredshiftangDdeg
deg
degdegdeg

 
float64float64int32float32int16float64float64
136.0007437121.96791867309313.8200.0103171374644788430.06354147623734327
146.7033430822.01827217744614.6-10.0248371825284544050.026865139047937883
146.7033430822.01827217759715.0-20.025340864312203610.026347225109192074
148.7780563114.29613296719415.120.023996601008555060.027777901534139626
175.0394016215.32725392332514.430.0110910061653385560.0591638655187596
191.5419864330.73226569665115.0200.0221853479716290940.02997982632048326
191.5473991130.72338192651714.7-20.021738372084063570.030579669183669427
194.9129447528.89537435609315.000.0203240603204234050.032651528395800614
206.5716382343.85050581222915.0-10.0074351436819668090.08786069920287261
206.5799642643.843857072586418.5--0.086273017582050050.008319382638265264
209.9626024438.18183343275914.930.0092030333865170160.07113666025554453
213.5560119915.62222407464414.030.0154907165810021820.042588105321019934
219.9664999742.74233535251714.7200.0083958082761374870.0778991456401862
333.8216972537.29922141520715.310.0173686824369677780.03807030457673168
136.0007421.96792309313.80200.0103171374644788430.06354147623734327
146.7033422.01827744614.60-10.0248371825284544050.026865139047937883
146.7033422.01827759715.00-20.025340864312203610.026347225109192074
148.7780614.29613719415.1020.023996601008555060.027777901534139626
175.0394015.32725332514.4030.0110910061653385560.0591638655187596
191.5419930.73227665115.00200.0221853479716290940.02997982632048326
191.5474030.72338651714.70-20.021738372084063570.030579669183669427
194.9129428.89537609315.0000.0203240603204234050.032651528395800614
206.5716443.85051222915.00-10.0074351436819668090.08786069920287261
206.5799643.843862586418.50--0.086273017582050050.008319382638265264
209.9626038.18183275914.9030.0092030333865170160.07113666025554453
213.5560115.62222464414.0030.0154907165810021820.042588105321019934
219.9665042.74234251714.70200.0083958082761374870.0778991456401862
333.8217037.29922520715.3010.0173686824369677780.03807030457673168

Now we construct and run a query that uses the new angDdeg column in every row search. Note, we also don’t want to list the original candidates since we know these are in the catalog and we want rather to find any companions. Therefore, we exclude the match if the radial velocities match exactly.

This time, rather than write the table to disk, we’ll keep it in memory and give Tap.query() a “file-like” object using io.BytesIO(). This can take half a minute:

-
## In memory only, use an IO stream. 
+
## In memory only, use an IO stream.
 vot_obj=io.BytesIO()
 apvot.writeto(apvot.from_table(mytable),vot_obj)
 ## (Reset the "file-like" object to the beginning.)
 vot_obj.seek(0)
-query="""SELECT mt.ra, mt.dec, cat.ra, cat.dec, cat.Radial_Velocity, cat.morph_type, cat.bmag 
-    FROM zcat cat, tap_upload.mytable mt 
+query="""SELECT mt.ra, mt.dec, cat.ra, cat.dec, cat.Radial_Velocity, cat.morph_type, cat.bmag
+    FROM zcat cat, tap_upload.mytable mt
     WHERE
     contains(point('ICRS',cat.ra,cat.dec),circle('ICRS',mt.ra,mt.dec,mt.angDdeg))=1
     and cat.Radial_Velocity > 0 and cat.radial_velocity != mt.radial_velocity
@@ -1042,26 +1070,26 @@ 
3.2  Cross-correlating with user-defined columns
 
Table length=9
-
+

+
-
-
-
-
-
-
-
-
-
+
+
+
+
+
+
+
+
+

 
radecra2dec2radial_velocitymorph_typebmag
degdegdegdeg

 
float64float64float64float64int32int16float32
146.7033430822.01827217146.7033430822.018272177597-215.0
146.7033430822.01827217146.7033430822.018272177446-114.6
175.0394016215.32725392175.0552212615.342802433299313.1
191.5473991130.72338192191.5419864330.7322656966512015.0
191.5419864330.73226569191.5473991130.723381926517-214.7
206.5716382343.85050581206.5799642643.8438570725864--18.5
206.5716382343.85050581206.5981029943.872229742420412.72
219.9664999742.74233535220.0385384142.778948852545714.0
333.8216972537.29922141333.8468365837.282608985984-315.06
146.7033422.01827146.7033422.018277597-215.00
146.7033422.01827146.7033422.018277446-114.60
175.0394015.32725175.0552215.342803299313.10
191.5474030.72338191.5419930.7322766512015.00
191.5419930.73227191.5474030.723386517-214.70
206.5716443.85051206.5799643.8438625864--18.50
206.5716443.85051206.5981043.872232420412.72
219.9665042.74234220.0385442.778952545714.00
333.8217037.29922333.8468437.282615984-315.06

 

 

 
Therefore, by adding new information to our original data table, we could cross-correlate with the TAP.  We find that, in our candidate list, there is one true pair of galaxies.

-

-

4 Synchronous versus asynchronous queries#

+

4. Synchronous versus asynchronous queries#

There is one technical detail about TAP queries that you will need to know. In the code cells above, there are two commands for sending the query, one of which is commented out. This is because, with the TAP, there are two ways to send such queries. The default when you use the search() method is to us a synchronous query, which means that the query is sent and the client waits for the response. For large and complicated queries, this may time out, or you may want to run several in parallel. So there are other options.

The method service.run_async() uses an asynchronous query, which means that the query is sent, and then (under the hood without you needing to do anything), the method checks for a response. From your point of view, these methods look the same; PyVO is doing different things under the hood, but the method will not return until it has your result.

You need to know about these two methods for a couple of reasons. First, some services will limit synchronous queries, i.e. they will not necessarily return all the results if there are too many of them. An asynchronous query should have no such restrictions. In the case of the HEASARC service that we use above, it does not matter, but you should be aware of this and be in the habit of using the asynchronous queries for complete results after an initial interactive exploration.

@@ -1083,7 +1111,7 @@

4 Synchronous versus asynchronous queries
@@ -1092,7 +1120,7 @@

4 Synchronous versus asynchronous queries

next

@@ -1119,19 +1147,19 @@

4 Synchronous versus asynchronous queries

@@ -1179,8 +1207,8 @@

4 Synchronous versus asynchronous queries - + +
diff --git a/CS_Image_Access.html b/content/reference_notebooks/image_access.html similarity index 80% rename from CS_Image_Access.html rename to content/reference_notebooks/image_access.html index 721e201..7467b7d 100644 --- a/CS_Image_Access.html +++ b/content/reference_notebooks/image_access.html @@ -19,37 +19,37 @@ - - - + + + - - - - + + + + - - - - + + + + - - - - - - - - - - - - - - - + + + + + + + + + + + + + + + @@ -77,7 +77,7 @@
-
@@ -220,7 +220,7 @@ -
  • - + Binder @@ -238,7 +238,7 @@ -
  • - + Colab @@ -286,7 +286,7 @@ -
  • + + + + + +.ipynb + +
    • + + + + +
  • -.ipynb +.md
    • @@ -425,7 +442,7 @@

    Contents