Re-processing of gPhoton DB (v2)

[scfleming]

** Make sure the files are generated with Python 3.x **

• Re-examine the data types of existing gPhoton columns to make smaller if possible.
• Think about new columns to add from RAW6, including eclipse numbers.
• Fix eclipses that failed originally.
• Fix null and non-null photon events being in wrong places (both directions).

[bshiao]

here's a list of the eclipse that didn't process, i.e. died in gPipe
eclipse tilename

---

38150 MISGCSN2_22274_0339_css38150
38151 MISGCSN3_23778_0484
38152 MISGCSN3_11396_0115
38153 MISGCSN3_11396_0115
38154 MISGCSN3_21639_0484
38155 MISGCSN3_21639_0484
38156 MISGCSN3_21847_0337
38157 PS_ELAISN1_MOS16
38158 PS_ELAISN1_MOS15
38159 PS_ELAISN1_MOS14
38166 MISGCSN3_21903_0136
38167 MISGCSN3_22003_0137
38168 MISGCSN3_22003_0137
38169 MISGCSN3_21967_0137
38170 MISGCSN3_21967_0137
38182 MISGCSN3_22184_0137
38183 MISGCSN3_22184_0137
38184 MISGCSN3_22224_0137
38185 MISGCSN_21958_1658
38186 MISGCSN3_22224_0137
38187 MISGCSN3_22536_0138
38188 MISGCSN3_22536_0138
38189 MISGCSN3_22306_0137
38190 MISGCSN3_22306_0137
38191 MISGCSN3_22305_0137
38192 MISGCSN2_22096_0131_css38192
38193 MISGCSN3_22305_0137
38194 MISGCSN3_22097_0131
38195 MISGCSN3_22097_0131
38196 GI6_026004_NGC6789
38197 MISGCSN3_22292_0131
38198 MISGCSN3_11763_0017
38199 MISGCSN3_22054_0017
38204 MISGCSN2_22259_0137_css38204
38205 MISGCSN3_22299_0132
38206 MISGCSN3_22299_0132
38207 MISGCSN3_22343_0132
38208 MISGCSN1_22631_0138
38209 MISGCSN3_22343_0132
38210 MISGCSN3_22431_0132
38211 MISGCSN3_22431_0132
38212 MISGCSN3_23042_0133
38213 MISGCSN3_23042_0133
38214 MISGCSN2_22288_0127_css38214
38215 MISGCSN4_22429_0132_css38215
38216 MISGCSN4_22429_0132_css38216
38218 MISGCSN4_22283_0017_css38218
38219 MISGCSN2_22281_0018_css38219
38220 MISGCSN4_22323_0018_css38220
38221 MISGCSN2_11012_0014_css38221
38222 MISGCSN2_22985_0133_css38222
38223 MISGCSN2_22985_0133_css38223
38225 MISGCSN1_22375_0127
38226 MISGCSN2_22325_0018_css38226
38227 MISGCSN4_22699_0124_css38227
38228 MISGCSN1_23041_0133
38229 MISGCSN1_22369_0018
38233 GI6_079001_Kepler_UZ_Lyr
38234 GI6_079001_Kepler_UZ_Lyr
38235 GI6_079001_Kepler_UZ_Lyr
38236 GI6_079001_Kepler_UZ_Lyr
38237 GI6_079001_Kepler_UZ_Lyr
38238 GI6_079001_Kepler_UZ_Lyr
38239 GI6_079001_Kepler_UZ_Lyr
38240 GI6_079001_Kepler_UZ_Lyr
38241 GI6_079001_Kepler_UZ_Lyr
38242 GI6_079001_Kepler_UZ_Lyr
38243 GI6_079001_Kepler_UZ_Lyr
38244 GI6_079001_Kepler_UZ_Lyr
38245 GI6_079001_Kepler_UZ_Lyr
38246 GI6_079001_Kepler_UZ_Lyr
38247 GI6_079001_Kepler_UZ_Lyr
38248 GI6_079001_Kepler_UZ_Lyr
38250 GI6_079001_Kepler_UZ_Lyr_css3825
38252 GI6_079001_Kepler_UZ_Lyr
38253 GI6_079001_Kepler_UZ_Lyr
38254 GI6_079001_Kepler_UZ_Lyr
38255 GI6_079001_Kepler_UZ_Lyr
38256 GI6_079001_Kepler_UZ_Lyr
38257 GI6_079001_Kepler_UZ_Lyr
38258 GI6_079001_Kepler_UZ_Lyr
38259 GI6_079001_Kepler_UZ_Lyr
38261 GI6_079002_Kepler_BR_Cyg
38262 GI6_079002_Kepler_BR_Cyg
38263 GI6_079002_Kepler_BR_Cyg
38264 GI6_079002_Kepler_BR_Cyg
38265 GI6_079002_Kepler_BR_Cyg
38267 GI6_079002_Kepler_BR_Cyg
38916 PS_VVDS22H_MOS01
39893 GI5_028106_W1_18139_0274o
40326 GI6_060015_ESSENCE_D4
40330 GI6_060013_ESSENCE_D2
40427 PS_CDFS_MOS00
40442 PS_CDFS_MOS00
40497 PS_CDFS_MOS04
41840 GI6_061021_F_VOID_241
41866 DLS_F2_MOS02
42369 MISGCSN1_14141_0225
42428 MISGCSN1_24506_0317
42435 DLS_F5_MOS01
44600 SMC_42591_0399
44615 SMC_42592_0399
45369 AIS_419
45369 AIS_420
45431 MIS2DFSGP_38117_0396
45481 MISGCSS1_38363_0424
46388 MISGCSN1_03193_0069

(107 row(s) affected)

[bshiao]

current table schema description. initial thoughts are to change the datatypes of xa, ya, q to tinyint since the range is between 0 and 255, this has has to be confirmed though. Will shrink the row size by 9 bytes.
If a unique id and eclipse are added it will increase the row by 12 bytes, 8 for a unique id and 4 for eclipse. So a overall increase of 3 bytes so far.
CREATE TABLE [dbo].[FUVPhotonsPOZ](
[zoneID] [int] NOT NULL,
[time] [bigint] NOT NULL,
[cx] [float] NOT NULL,
[cy] [float] NOT NULL,
[cz] [float] NOT NULL,
[x] [real] NOT NULL,
[y] [real] NOT NULL,
[xa] [int] NOT NULL,
[ya] [int] NOT NULL,
[q] [int] NOT NULL,
[xi] [real] NOT NULL,
[eta] [real] NOT NULL,
[ra] [float] NOT NULL,
[dec] [float] NOT NULL,
[flag] [tinyint] NOT NULL
)

confirmed the min/max valus for xa, ya, q
select min(xa), max(xa), min(ya), max(ya), min(q), max(q)
from FUVPhotonsPOZ

---

0 31 0 31 0 31

(1 row(s) affected)