Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Gaia column confusion for few sources #715

Open
araichoor opened this issue Oct 19, 2022 · 2 comments
Open

Gaia column confusion for few sources #715

araichoor opened this issue Oct 19, 2022 · 2 comments

Comments

@araichoor
Copy link

[copying pieces of an email thread with @geordie666 and @dstndstn]

it appears that there are two different objects in the Legacy Surveys with the same REF_ID:

from @geordie666 :

a = fitsio.read("/global/cfs/cdirs/cosmo/data/legacysurvey/dr9/south/sweep/9.0/sweep-020p030-030p035.fits")
a[a["REF_ID"] == 303315950562852864][["RA", "DEC", "REF_ID", "PARALLAX", "PMRA", "PMDEC"]]
Out[]:
array([(23.88419642, 30.37504982, 303315950562852864, 0., 0., 0.),
       (23.97793216, 30.21423356, 303315950562852864, 0., 0., 0.)],

These correspond to your "correct" source and your "incorrect" source:

303293444934132224      G2 23.97793215834139   30.214233564409696     
303315950562852864      G2 23.884196422341706 30.375049822000054

The duplicate is also present in the tractor catalogs, one in brick 0238p302 and one in brick 0238p305:

a = fitsio.read("/global/cfs/cdirs/cosmo/data/legacysurvey/dr9/south/tractor/023/tractor-0238p302.fits", upper=True)
a[a["REF_ID"] == 303315950562852864][["RA", "DEC", "REF_ID", "PARALLAX", "PMRA", "PMDEC"]]
Out[]:
array([(23.97793216, 30.21423356, 303315950562852864, 0., 0., 0.)],
      dtype={'names':['RA','DEC','REF_ID','PARALLAX','PMRA','PMDEC'], 'formats':['>f8','>f8','>i8','>f4','>f4','>f4'], 'offsets':[59,67,139,155,147,151], 'itemsize':2164})

a = fitsio.read("/global/cfs/cdirs/cosmo/data/legacysurvey/dr9/south/tractor/023/tractor-0238p305.fits", upper=True)
a[a["REF_ID"] == 303315950562852864][["RA", "DEC", "REF_ID", "PARALLAX", "PMRA", "PMDEC"]]
Out[]:
array([(23.88419642, 30.37504982, 303315950562852864, 0., 0., 0.)],
      dtype={'names':['RA','DEC','REF_ID','PARALLAX','PMRA','PMDEC'], 'formats':['>f8','>f8','>i8','>f4','>f4','>f4'], 'offsets':[59,67,139,155,147,151], 'itemsize':2164})

from @dstndstn :

That Gaia star is right on a brick edge. Not sure if that's relevant or not.

This looks like an error in the tractor catalogs...

That star appears with the same values in the "reference" source tables:

> tablist /global/cfs/cdirs/cosmo/data/legacysurvey/dr9/south/metrics/023/reference-0238p302.fits"[REF_ID==303315950562852864]" | head

                         ra                     dec re               ref_id            mag
   1        23.8841964223417        30.3750498220001 G2   303315950562852864        20.7093

    i i i i i i f p d i             ba             pa    decam_mag_g    decam_mag_r    decam_mag_z
   1 F T F F F F F F F T        0.00000        0.00000        21.5519        20.5525        20.0744

            zguess       mask_mag         radius    keep_radius           radius_pix
   1        20.0744        20.7093    0.000452284     0.00180914                    7

> tablist /global/cfs/cdirs/cosmo/data/legacysurvey/dr9/south/metrics/023/reference-0238p305.fits"[REF_ID==303315950562852864]" | head

                         ra                     dec re               ref_id            mag
   1        23.8841964223417        30.3750498220001 G2   303315950562852864        20.7093

    i i i i i i f p d i             ba             pa    decam_mag_g    decam_mag_r    decam_mag_z
   1 F T F F F F F F F T        0.00000        0.00000        21.5519        20.5525        20.0744

            zguess       mask_mag         radius    keep_radius           radius_pix
   1        20.0744        20.7093    0.000452284     0.00180914                    7

But then it appears in the tractor catalogs with incorrect RA,Dec, and different flux_r values,

> tablist /global/cfs/cdirs/cosmo/data/legacysurvey/dr9/south/tractor/023/tractor-0238p305.fits"[REF_ID==303315950562852864][col ref_id;brickname;maskbits;type;ra;dec;bx;by;ref_cat;gaia_phot_g_mean_mag;flux_r]" | head

    bricknam maskbi typ                      ra                     dec             bx
   1 0238p305      0 PSF        23.8841964223417        30.3750498220001        1685.72

                by re               ref_id gaia_phot_g_me         flux_r
   1        82.6294 G2   303315950562852864        20.7093        4.98087


> tablist /global/cfs/cdirs/cosmo/data/legacysurvey/dr9/south/tractor/023/tractor-0238p302.fits"[REF_ID==303315950562852864][col ref_id;brickname;maskbits;type;ra;dec;bx;by;ref_cat;gaia_phot_g_mean_mag;flux_r]" | head

    bricknam maskbi typ                      ra                     dec             bx
   1 0238p302      0 PSF        23.9779321583414        30.2142335644097        117.500

                by re               ref_id gaia_phot_g_me         flux_r
   1        1309.10 G2   303315950562852864        20.7093        5.69361

The bad RA,Dec does correspond to a Gaia star that is itself missing from the catalog:

> tablist -w 160 /global/cfs/cdirs/cosmo/data/legacysurvey/dr9/south/tractor/023/tractor-0238p302.fits"[REF_ID==303293444934132224]" | less -S
(none)

https://www.legacysurvey.org/viewer/?ra=23.9779&dec=30.2164&layer=ls-dr9&zoom=16&bricks&gaia-dr2

so it looks like the REF_ID got scrambled or something.
@araichoor
Copy link
Author

if useful, below are few more examples (found while running some checks on desi fiberassign patched tiles):
the GAIA_ columns are from desitarget.gaiamatch.match_gaia_to_primary(), the ORIG_VAL should come from tractor.
I ve just investigated the one reported above.

TILEID      TARGETID          GAIA_RA            GAIA_DEC            GAIA_VAL            ORIG_VAL     
------ ----------------- ------------------ ------------------ ------------------- -------------------
  3370 39628376683578739 34.031842136570866  25.14932607678862  105875964663651584  105875857291264640
  3371 39628483692859235 23.910033773471664 30.186978366553923  303295884475547776  303309147334594048
  3371 39628483692858723 23.878016000059542 30.134438063571963  303291284570189568  303201193333294848
  3371 39628483692859800  23.94468612421525 30.241997474876417  303296507250414464  303308150902171904
  3371 39628483692860390  23.97793215834139 30.214233564409696  303293444934132224  303315950562852864
  8053 39628483692860390  23.97793215834139 30.214233564409696  303293444934132224  303315950562852864
  8973 39627473511518217 21.660460334630915 -13.04582292468445 2457118771061174016 2457105950583223296
 20305 39628342567112294  325.5384490476321 23.610040205966566 1794517089518000256 1794515710833412992
 20305 39628342567110737 325.49177306155127  23.49398234455707 1794513962781805440 1794515951351442688
 20305 39628342567109290  325.4406691692023 23.394957599508295 1794489425633289088 1794518734490262144
 20305 39628342567108740  325.4180316183054  23.39370266702708 1794489391279175040 1794526843387435520
 20305 39628342567115467  325.6378307669439   23.6016457985784 1794521934241014400 1794516260588009472
 23926 39628478441591216  23.28040085802485  29.91215089247258  303182256821674624  303256375073080448
 23926 39628478441590838 23.262013842103137 29.948855913262943  303205999400904448  303183047095661824
 23926 39628478441587112 23.056279881760517  29.93504095346095  303228745546814080  303254309193090944
 23926 39628483680274814 22.948170151322838  30.13189176730603  303247712122471680  303276746102554752
 23926 39628483692859790  23.94406896619156 30.143191949771303  303292590240273920  303315710044670080
 23926 39628483692857959   23.8344256279959  30.13325760091827  303293891610707200  303311969132673536
 23926 39628483692857457  23.80739131842835 30.173254875551176  303294961062192640  303302790783003392
 24769 39628342567110160  325.4719299182065 23.445087848091013 1794513133851773312 1794462178360716032
 24769 39628342567110993  325.4994771837178  23.39686471211951 1794509942692212096 1794516054429599744
 24769 39628342567109785   325.459159935496 23.416351252633135 1794512927694355840 1794510419433482496
 24769 39628342567110874 325.49575098504164 23.377183104808797 1794509697878893568 1794490559503661952
 24769 39628342567116008 325.65602709540036  23.47584762572384 1794511553304879104 1794511037908939648
 24769 39628342567111694  325.5193126754916  23.61107930031608 1794517153941249024 1794512549740280832
 24769 39628342567108713  325.4168926927513 23.548443735387504 1794526778964046336 1794522307903082368
 24769 39628342567110612  325.4878742719738  23.54159993046535 1794516294947748096 1794510213275029888
 24769 39628342567109340  325.4422308630944 23.505102853767326 1794514649976391552 1794509942692212096
 24769 39628342567110070  325.4687699531766 23.474802176117713 1794513821046552448 1794522209118825088
 24769 39628342567112417 325.54200592248395  23.55667087095232 1794515642112727680 1794512412298359296
 24769 39628342567114457  325.6040754517045  23.55730423337604 1794515882631971584 1794512038635420800
 24769 39628342567116997 325.68296995167356 23.621271321027855 1794522307903079296 1794509835320366976
 80615 39628478441591047 23.272506724074518  30.01500969028378  303207064554972544  303254377912562304
 80615 39628478441591047 23.272506724074518  30.01500969028378  303207064554972544  303254377912562304
 80615 39628483692859660 23.935153208764465 30.139727939463295  303291177191378176  303291795666665472

@moustakas
Copy link
Contributor

This was documented as a known issue for DR9 and DR10 in legacysurvey/legacysurvey@b709ef8.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@moustakas @araichoor