Fix the Links of Spectra for Suarez22

[SherelynA]

The links for the spectra for the Suarez22 ingest don't work and need to be fixed to have the correct ones.

[kelle]

Please be more specific about what "doesn't work" mean and give a couple examples.

[SherelynA]

The issue lies in that the sources which have a plus in the name like 2MASS J21481633+4003594 have the url name of https://bdnyc.s3.amazonaws.com/IRS/2148%252B4003_IRS_spectrum.fits which does not work because the actual working link is https://bdnyc.s3.amazonaws.com/IRS/2MASS+J21481633%2B4003594.fits. This is the google doc link to the correct spectra urls : https://docs.google.com/spreadsheets/d/1ojsNF59GvdFhf-lgwJaCwiHPfaRx4fmCpyzKEonrVbw/edit?usp=sharing

[kelle]

We should write a test which finds these problems before we fix them. Issue opened #369

[kelle]

another example: https://simple-bd-archive.org/load_solo/2MASS%20J12130336-0432437

[kelle]

There are 149 URLs giving 404s:
['https://bdnyc.s3.amazonaws.com/IRS/0837-0000_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0937%252B2931_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1916%252B0509_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/vB10_SL.txt', 'https://bdnyc.s3.amazonaws.com/IRS/1051%252B5613_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1707-0558_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0144-0716_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20990.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0034-0052_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1615%252B3559_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS1615%2B3558_SL.txt', 'https://bdnyc.s3.amazonaws.com/XShooter/J11123099-7653342.txt', 'https://bdnyc.s3.amazonaws.com/IRS/2204-5646_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20989_n3.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20989_n4.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40018_ppl1_davy.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1022%252B5825_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1045-0149_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2224-0158_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1048%252B0111_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1239%252B5515_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21011.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1335%252B1130_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0929%252B3429_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1217-0311_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1624%252B0029_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0921-2104_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0912%252B1459_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1028%252B5654_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0255-4700_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/RCspec%20%28CTIO%204m%29/L1_OPT_2MASS_J10595138-2113082_Cruz2003.txt', 'https://bdnyc.s3.amazonaws.com/IRS/0000%252B2554_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0746%252B2000_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS0746%2B2000AB_SL.txt', 'https://bdnyc.s3.amazonaws.com/IRS/1155%252B0559_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1106%252B2754_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2057-0252_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20906.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1439%252B1839_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1237%252B6526_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1507-1627_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2152%252B0937_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1052%252B4422_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1731%252B2721_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21010.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1013-1356_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1108%252B6830_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1728%252B3948_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1254-0122_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2244%252B2043_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2132%252B1341_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1828%252B1229_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0830%252B4828_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0523-1403_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0911%252B7401_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2148%252B4003_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1503%252B2525_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS1503%2B2525_SL.txt', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21002.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0004-4044_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20907.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0805%252B4812_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40106_0406-3812.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1457-2121_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1110%252B0116_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0036%252B1821_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS0036%2B1821_SL.txt', 'https://bdnyc.s3.amazonaws.com/IRS/0429-3123_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U50933_1118-0856_K10.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0857%252B5708_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1658%252B7027_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS1658%2B7027_SL.txt', 'https://bdnyc.s3.amazonaws.com/IRS/0939-2448_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0920%252B3517_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1626%252B3925_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1456-2809_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1448%252B1031_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21009.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0624-4521_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U51060_1302%2B5650_notell.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0059-0114_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0355%252B1133_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0024-0158_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/BRI0021-0214_SL.txt', 'https://bdnyc.s3.amazonaws.com/IRS/0700%252B3157_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0139-1757_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1506%252B1321_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20976.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2139%252B0220_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0908%252B5032_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2144%252B1446_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40005_2249%2B0044.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1305-2541_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1425-3650_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1331-0116_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U50174_1331-0116_Hawley02.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1610-0040_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0415-0935_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20909.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20979.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0539-0059_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1126-5003_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1807%252B5015_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1523%252B3014_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2146-0010_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1753-6559_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40021_roque25_davy.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0025%252B4759_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0825%252B2115_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1225-2739_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/u12003.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20993_2315%2B0617.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0251-0352_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1520%252B3546_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1021-0304_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21018.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1936-5502_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1207%252B0244_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2351-2537_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0758%252B3247_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0136%252B0933_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1821%252B1414_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21004.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0727%252B1710_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20994.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0034%252B0523_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1036-3441_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U11993.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1721%252B3344_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1114-2618_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0423-0414_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0532%252B8246_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1526%252B2043_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1213-0432_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1017%252B1308_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0559-1404_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1441-0945_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0445-3048_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21005.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0501-0010_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1439%252B1929_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1516%252B3053_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1515%252B4847_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/1112%252B3548_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21001.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0829%252B2646_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2254%252B3123_IRS_spectrum.fits', 'https://bdnyc.s3.amazonaws.com/IRS/0439-2353_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40068_0534-0631.fits']

[kelle]

Progress. After #404 , here are the ones left:

assert len(broken_urls) == 149, f'found {len(broken_urls)} broken spectra urls: {broken_urls}, {codes}'
E AssertionError: found 39 broken spectra urls: ['https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/vB10_SL.txt', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20990.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS1615%2B3558_SL.txt', 'https://bdnyc.s3.amazonaws.com/XShooter/J11123099-7653342.txt', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20989_n3.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20989_n4.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40018_ppl1_davy.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21011.fits', 'https://s3.amazonaws.com/bdnyc/RCspec%20%28CTIO%204m%29/L1_OPT_2MASS_J10595138-2113082_Cruz2003.txt', 'https://bdnyc.s3.amazonaws.com/IRS/0000%252B2554_IRS_spectrum.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS0746%2B2000AB_SL.txt', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20906.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21010.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS1503%2B2525_SL.txt', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21002.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20907.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40106_0406-3812.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS0036%2B1821_SL.txt', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U50933_1118-0856_K10.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/2MASS1658%2B7027_SL.txt', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21009.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U51060_1302%2B5650_notell.fits', 'https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/BRI0021-0214_SL.txt', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20976.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40005_2249%2B0044.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U50174_1331-0116_Hawley02.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20909.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20979.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40021_roque25_davy.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/u12003.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20993_2315%2B0617.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21018.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21004.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20994.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U11993.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21005.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U21001.fits', 'https://bdnyc.s3.amazonaws.com/IRS/2MASS+J22541892%2B3123498.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U40068_0534-0631.fits'], [404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404]
E assert 39 == 149
E + where 39 = len(['https://s3.amazonaws.com/bdnyc/IRS/IRS_DimSuns_S12/vB10_SL.txt', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20...zonaws.com/bdnyc/optical_spectra/U20989_n3.fits', 'https://s3.amazonaws.com/bdnyc/optical_spectra/U20989_n4.fits', ...])