Modify the schema with a new Regimes table (#471)

* add Regimes to Reference Tables * made Regimes table by hand * Spectra table now tied to Regimes * SpectralTypes table: regime now foreign key * Gravities: regime now foreign key to Regimes table * docs updates --------- Co-authored-by: Will Cooper <[email protected]>
SIMPLE-AstroDB · Mar 5, 2024 · bef48f0 · bef48f0
1 parent 6c46a09
commit bef48f0
Show file tree

Hide file tree

Showing 16 changed files with 749 additions and 563 deletions.
diff --git a/data/Regimes.json b/data/Regimes.json
@@ -0,0 +1,18 @@
+[
+    {
+        "regime": "nir",
+        "description": "Near-infrared 1-5 microns"
+    },
+    {
+        "regime": "optical",
+        "description": "Optical 3000-10000 Angstroms"
+    },
+    {
+        "regime": "mir",
+        "description": "Mid-infrared 5-30 microns"
+    },
+    {
+        "regime": "unknown",
+        "description": "Used in Spectral Types table. Delete in #309"
+    }
+]
diff --git a/data/Versions.json b/data/Versions.json
@@ -66,8 +66,14 @@
         "description": "Added JWST spectrum for WISE 1935-1546"
     },
     {
-        "version": "latest",
+        "version": "2024.2",
         "start_date": "2024-02-15",
+        "end_date": "2024-03-05",
+        "description": "Regime cleanup"
+    },
+    {
+        "version": "latest",
+        "start_date": "2024-03-05",
         "end_date": null,
         "description": "Version in development"
     }

diff --git a/documentation/Gravities.md b/documentation/Gravities.md
@@ -2,16 +2,16 @@
 
 The Gravities table contains gravity measurements for sources listed in the Sources table. 
 The combination of *source*, *regime*, and *reference* is expected to be unique.
-Note that *gravity* and *regime* are strings constrained from a list of enumerated values. 
+Note that *gravity* is a string constrained from a list of enumerated values (see below table). 
 Columns marked with an asterisk (*) may not be empty.
 
 | Column Name | Description  | Unit  | Data Type | Key Type  |
 |---|---|---|---|---|
 | *source    | Unique identifier for the source |   | String(100)  | primary and foreign: Sources.source   |
 | gravity | Gravity value |  | Enumeration  |   |
-| *regime | Regime for gravity value |  | Enumeration  | primary |
+| *regime | Regime for gravity value |  |  | primary and foreign: Regimes.regime |
 | comments  | Free form comments |   | String(1000) |   |
-| *reference | Reference |   | String(30) | primary and foreign: Publications.name |
+| *reference | Reference |   | String(30) | primary and foreign: Publications.reference |
 
 Enumeraions for gravity include:
  - alpha
@@ -23,12 +23,3 @@ Enumeraions for gravity include:
  - vl-g
  - int-g
  - fld-g
-
-Enumerations for regime include:
- - gammaray
- - xray
- - ultraviolet
- - optical
- - infrared
- - millimeter
- - radio
diff --git a/documentation/Spectra.md b/documentation/Spectra.md
@@ -11,36 +11,24 @@ Columns marked with an asterisk (*) may not be empty.
 | *spectrum         | URL of spectrum location |   | String(1000) | primary |
 | original_spectrum | URL of original spectrum location, if applicable |   | String(1000) |  |
 | local_spectrum    | Local path of spectrum   |   | String(1000) |  |
-| *regime           | Regime of the spectrum, eg Optical, Infrared, etc |  | Enumeration  | primary |
-| *telescope        | Name of telescope |  | String(30)   | foreign: Telescopes.name |
-| *instrument       | Name of instrument |  | String(30)   | foreign: Instruments.name |
-| *mode             | Mode of spectrum  |  | String(30)   | foreign: Modes.name |
+| *regime           | Regime of the spectrum, eg Optical, Infrared, etc |  |  | foreign: Regimes.regime |
+| *telescope        | Name of telescope |  | String(30)   | foreign: Telescopes.telescope |
+| *instrument       | Name of instrument |  | String(30)   | foreign: Instruments.instrument |
+| *mode             | Mode of spectrum  |  | String(30)   | foreign: Instruments.mode |
 | *observation_date | Observation date  |  | DateTime     | primary |
 | comments          | Free form comments |   | String(1000) |   |
-| *reference        | Primary Reference |   | String(30)   | primary and foreign: Publications.name |
+| *reference        | Primary Reference |   | String(30)   | primary and foreign: Publications.reference |
 | other_references  | Other References |   | String(100)  |   |
 
+Relevant functions: `spectra.ingest_spectrum`, `spectra.spectrum_plottable`, `spectra.find_spectra`
+
 If the spectrum provided has been modified from the author-provided one, 
 a link to the original spectrum can be provided in the `original_spectrum` column.
 
 The local_spectrum is meant to store the path to a local copy of the spectrum with an 
 environment variable to define part of the path (so it can be shared among other users). 
 For example: `$ASTRODB_SPECTRA/infrared/filename.fits`
 
-Enumerations for regime should be [UCDs](https://www.ivoa.net/documents/UCD1+/20210616/EN-UCDlist-1.4-20210616.html#tth_sEc2). 
-They currently  include:
- - em.UV
- - em.opt
- - optical (*deprecated, do not use*)
- - em.IR.NIR
- - nir (*deprecated, do not use*)
- - em.IR
- - em.IR.MIR 
- - mir (*deprecated, do not use*)
- - em.mm
- - em.radio 
- - unknown
-
 # Notes
  - An accurate observation date is required for a spectrum to be ingested.
  - Data based on data from multiple observation dates has 'Multiple observation dates' 

diff --git a/documentation/SpectralTypes.md b/documentation/SpectralTypes.md
@@ -1,17 +1,16 @@
 # SpectralTypes
 
 The SpectralTypes table contains spectral type measurements for sources listed in the Sources table. 
-The combination of *source*, *regime*, and *reference* is expected to be unique.
-Note that *regime* is a string constrained from a list of enumerated values.
+The combination of *source*, *spectral_type_code*, *regime*, and *reference* is expected to be unique.
 Columns marked with an asterisk (*) may not be empty.
 
 | Column Name | Description  | Unit  | Data Type | Key Type  |
 |---|---|---|---|---|
 | *source    | Unique identifier for the source |   | String(100)  | primary and foreign: Sources.source   |
 | spectral_type_string | Spectral type string |  | String(10)  |   |
-| spectral_type_code | Numeric code corresponding to spectral type |  | Float  |   |
+| *spectral_type_code | Numeric code corresponding to spectral type |  | Float  | primary  |
 | spectral_type_error | Uncertainty of spectral type |  | Float  |   |
-| *regime | Regime for spectral type value |  | Enumeration  | primary |
+| regime | Regime for spectral type value |  |  | primary and foreign:Regimes.regime |
 | adopted    | Flag indicating if this is the adopted measurement |  | Boolean  |   |
 | comments  | Free form comments |   | String(1000) |   |
 | *reference | Reference |   | String(30) | primary and foreign: Publications.name |
@@ -21,16 +20,4 @@ Spectral Type Codes:
  - 69 = M9
  - 70 = L0 
  - 80 = T0
- - 90 = Y0
-
-Enumerations for regime include:
- - gammaray
- - xray
- - ultraviolet
- - optical
- - nir
- - infrared
- - millimeter
- - radio
- - unknown
-
+ - 90 = Y0
diff --git a/scripts/__init__.py b/scripts/__init__.py
diff --git a/scripts/updates/2023/update_regimes.py → scripts/updates/update_regimes.py b/scripts/updates/2023/update_regimes.py → scripts/updates/update_regimes.py
@@ -177,6 +177,16 @@
     )
     conn.commit()
 
+with db.engine.connect() as conn:
+    conn.execute(
+        db.SpectralTypes.update()
+        .where(
+            db.SpectralTypes.c.regime == "unknown",
+        )
+        .values(regime=None)
+    )
+    conn.commit()
+
 
 # Populate the regimes table
 with db.engine.connect() as conn:
@@ -197,6 +207,15 @@
     )
     conn.commit()
 
+with db.engine.connect() as conn:
+    conn.execute(
+        db.Regimes.insert().values(
+            regime="unknown", description="Used in Spectral Types table. Delete in #309"
+        )
+    )
+    conn.commit()
+
+# Get list of regimes in the Spectral Typtes table
 
 # Save database
 db.save_database("data/")