Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overview about MAGIC articles #157

Open
pdeiml opened this issue Aug 29, 2017 · 19 comments
Open

Overview about MAGIC articles #157

pdeiml opened this issue Aug 29, 2017 · 19 comments
Assignees
Milestone

Comments

@pdeiml
Copy link
Collaborator

pdeiml commented Aug 29, 2017

This issue gives an overview about all MAGIC publication which can be found here:
https://magic.mpp.mpg.de/backend/publications
i will update the list during the next days.

Firstly, the articles:

Reference relevant? info.yaml? yaml file?
2006ApJ...648L.105A y y y
2006ApJ...642L.119A y y y
2006ApJ...637L..41A y y y
2006ApJ...638L.101A y y y
2006ApJ...639..761A y y y
2006ApJ...643L..53A y y y
2006Sci...312.1771A y y y
2006ApJ...641L...9A ? n n
2007ApJ...665L..51A y n n
2007ApJ...669..862A ? n n
2007NIMPA.583..494A ? n n
2007ApJ...663..125A ? n n
2007A&A...474..937A y y y
2007ApJ...662..892A y y y
2007ApJ...667..358A ? n n
2007ApJ...658..245A ? n n
2007ApJ...664L..87A y y y
2007ApJ...667L..21A y y y
2007ApJ...666L..17A y y y
2007ApJ...654L.119A y y y
2007ApJ...669.1143A ? n n
2008ApJ...674.1037A y y y
2008Sci...320.1752M y y y
2008ApJ...685L..23A y y y
2008ApJ...679..428A ? n n
2008NIMPA.595..200D ? n n
2008ApJ...681..944A ? n n
2008ApJ...679.1029T y y y
2008PhLB..668..253M ? n n
2008Sci...322.1221A y n n
2008ApJ...684.1351A y y y
2008ApJ...675L..25A y n n
2008NIMPA.588..424A ? n n
2008ApJ...685L..71A ? n n
2008NIMPA.594..407A ? n n
2009A&A...493..467A y y y
2009ApJ...697.1299A ? n n
2009ApJ...691L..13D y y y
2009PASJ...61.1011S y n n
2009ApJ...703..169A y y y
2009ApJ...705.1624A y y y
2009ApJ...702..266A y n n
2009Sci...325..444A ? n n
2009ApJ...693..303A y y y
2009A&A...498...83A ? n n
2009APh....30..293A ? n n
2009ApJ...704L.129A y y y
2009ApJ...692L..29A y y y
2009ApJ...706L..27A y y y
2010A&A...515A..76A y y y
2010ApJ...723L.207A y y y
2010ApJ...721..843A ? n n
2010ApJ...710..634A y y y
2010A&A...517A...5A ? n n
2010A&A...519A..32A y y y
2010ApJ...725.1629A ? n n
2010A&A...524A..77A y y y
2010ApJ...710..828A ? n n
2011ApJ...729....2A y y y
2011ApJ...735L...5A y n n
2011ApJ...736..131A y y y
2011ApJ...729..115A ? n n
2011ApJ...727..129A y y y
2011ApJ...730L...8A y y y
2011A&A...530A...4A y y y
2011ApJ...726...58A y y y
2011ApJ...742...43A y n n
2011JCAP...06..035A ? n n
2012ApJ...746..151A y y y
2012A&A...541A..99A ? n n
2012ApJ...746...80A y y y
2012A&A...539L...2A y y y
2012ApJ...754L..10A y y y
2012A&A...539A.118A y y y
2012A&A...544A.142A y y y
2012A&A...544A..75A y y y
2012A&A...544A..96A y y y
2012A&A...541A..13A y y y
2012A&A...542A.100A y y y
2012APh....35..435A ? n n
2012ApJ...748...46A y y y
2012A&A...540A..69A y ? ?
2012NIMPA.692..201D ? n n
2013A&A...552A.112A ? n n
2013A&A...556A..67A y y y
2013A&A...549A..23A y n n
2014Sci...346.1080A y y y
2014A&A...563A..91A y y y
2014ApJ...786..157A y n n
2014MNRAS.437.3103A ? n n
2014A&A...571A..96M y y y
2014A&A...567A..41A ? n n
2014A&A...567A.135A y y y
2014A&A...563A..90A y y y
2014A&A...565L..12A y n n
2014A&A...564A...5A y y y
2014A&A...568A.109A y n n
2014NIMPA.742...99D n n n
2014JCAP...02..008A y n n
2014A&A...567L...8A ? n n
2014MNRAS.440..530A ? n n
2014A&A...569A..46A ? n n
2014A&A...572A.121A ? n n
2015ApJ...815L..23A y n n
2015A&A...576A.126A y y y
2015MNRAS.450.4399A y y y
2015A&A...573A..50A y y y
2015JHEAp...5...30A y y y
2015A&A...576A..36A ? n n
2015MNRAS.451..739A y y y
2015MNRAS.446..217A y y y
2015A&A...578A..22A ? n n
2015A&A...582A..67A ? n n
2015ApJ...812...65F n n n
2015arXiv150902391P n n n
2016APh....72...61A n n n
2016APh....72...76A n n n
2016A&A...585A.133A y n n
2016A&A...591A..76A y y y
2016A&A...591A.138A ? n n
2016ApJ...819..156B y y y
2016MNRAS.459.2286A y y y
2016A&A...590A..24A y y y
2016A&A...593A..91A ? n n
2016JCAP...02..039M ? n n
2016MNRAS.459.3271A ? n n
2016A&A...591A..10A y y y
2016A&A...589A..33A y y y
2016A&A...595A..98A ? ? ?
2016JInst..1111009. n ? ?

| 2017MNRAS.470.4608A | ? | n | n |
| 2017A&A...603A..29A | ? | n | n |
| 2017A&A...603A..31A | y | y | y |
| 2017A&A...602A..98A | y | n | n |
| 2017MNRAS.471.1688A | ? | n | n |
| 2017arXiv170803689M | y | n | n |
| 2017A&A...601A..33A | n | n | n |

These articles do not have a Reference_id:
https://magic.mpp.mpg.de/backend/publication/show/361
https://magic.mpp.mpg.de/backend/publication/show/369

I propose that someone should double check the papers I marked with 'not relevant' or with the questionmark which means that I am not sure whether the paper is relevant or not.
Additionally, one should check whether for the relevant papers data entry is done or not, if not, please make at least a dummy yaml file with a TODO remark in the corresponding folder.
Of course, edit the table entry when you did something.

@pdeiml pdeiml changed the title Overview about MAGIC publications Overview about MAGIC articles Aug 30, 2017
@cdeil cdeil added this to the wishlist milestone Aug 31, 2017
@cdeil
Copy link
Contributor

cdeil commented Aug 31, 2017

@pdeiml - Thanks for doing the overview and starting the MAGIC data entry!

I'm not familiar with their papers, and have no time to look, so I won't be able to help you if you have any question which papers are relevant or not. Personally I'm only interested in Galactic sources, and would give priority to sources (if any left) that don't have a single measurement in gamma-cat yet: https://github.com/cdeil/gamma-cat-status/blob/master/cat_table_printout.csv

@pdeiml
Copy link
Collaborator Author

pdeiml commented Aug 31, 2017

Ok, but as soon as I start to add any data problems may arise and then I will write you my questions in the corresponding PR :-)
At some point I may will have a deeper look into the papers which I marked as 'not relevant' and if am not sure, we will have to discuss.

But don't worry, this will work in some way at some point.

@cdeil
Copy link
Contributor

cdeil commented Aug 31, 2017

Just to note here what we discussed on Skype.

IMO this huge task / TODO table in this issue is not maintainable. I did the same when I added HESS data, then I moved it to lists in the todo folder, then I gave up because it's hard to always remember to update.

Suggest to instead transfer the info from the table above into info.yaml files and then generate such tables and status info on the gamma-cat webpage:
#155 (comment)

For hand-written notes like "we should have all MAGIC papers up to 2016 according to this list" or something like this, we should use some docs page.
Feel free to add a section to either of these pages
https://gamma-cat.readthedocs.io/changes.html
https://gamma-cat.readthedocs.io/stats/index.html
or make a new page status_notes.rst or something like that any time.

Suggest to do this work of getting set up in a good way first (and closing this issue) before spending time doing more MAGIC data entry.

@cdeil
Copy link
Contributor

cdeil commented Sep 6, 2017

@pdeiml - Just as an example I've added a few info.yaml for MAGIC publications, as well as a file other_data_collections/MAGIC/publications.yaml where we can keep track of MAGIC papers. See 7586417 .

I'm not sure if we should try to be complete in the list of MAGIC publications. There will always be many "nice to have" papers like conference proceedings and papers with upper limits, so I think the attempt to add info.yaml for all MAGIC papers or to draw a strict line which might be relevant in gamma-cat is hard to draw and we could just skip on that and simply add info.yaml for the ones we clearly want (i.e. refereed publications with good measurements that can be entered into gamma-cat).

I'll stop with gamma-cat again now. @pdeiml - Please transfer the list above into info.yaml files.

@pdeiml
Copy link
Collaborator Author

pdeiml commented Sep 11, 2017

@cdeil I will be going through the input folders and check whether the reference already have a folder/ info.yaml/ dummy yaml-file.
Do you prefer big PRs with creation of e.g. all 2006 references or do you prefer one PR per reference?

@cdeil
Copy link
Contributor

cdeil commented Sep 11, 2017

Please make one PR adding all info.yaml files. Also I have close to zero time to review, so please check the diff yourself before submitting the PR. The main things to check are that the reference_id is a valid ADS identifier, and that the source_id is the correct one for the given paper. I usually use https://gammapy.github.io/gamma-cat/sources.html to look up source names vs source_id in gamma-cat.

@pdeiml
Copy link
Collaborator Author

pdeiml commented Sep 13, 2017

I need help with these two articles:

https://arxiv.org/abs/0706.1505
There the black hole cygnus x-1 is mentioned and I don't find this source in the defined sources in gamma-cat.

https://arxiv.org/abs/astro-ph/0602231
I am not sure, whether this paper is relevant for gamma-cat. It mentions only an 'upper limit' to the flux.

@cdeil
Copy link
Contributor

cdeil commented Sep 13, 2017

@pdeiml - Please add what you can, and leave the few you're unsure about somewhere in a reminder issue. Leave both of those out for now. Then we can look at / discuss them individually, or if needed talk to someone from MAGIC about.

These are upper limit papers on sources that weren't detected. So for Cygnus X-1 we'd have to add a "detected" flag to our basic source info schema first. I.e. best to postpone this to a separate issue (or just not add it for now, focus on the meat of available measurements and add those first; that's what Gernot is doing for VERITAS).

@pdeiml
Copy link
Collaborator Author

pdeiml commented Sep 13, 2017

Ok, I make here an overview about the articles I am not sure about and then we can discuss later on and close this issue as soon as EVERY article was being looked at. :-)

@pdeiml
Copy link
Collaborator Author

pdeiml commented Sep 13, 2017

Overwiew about publications that need further discussion:

2006ApJ...641L...9A
2007ApJ...665L..51A
2007ApJ...669..862A
2007NIMPA.583..494A
2007ApJ...663..125A
2007ApJ...667..358A
2007ApJ...658..245A
2007ApJ...669.1143A
2008ApJ...679..428A
2008NIMPA.595..200D
2008ApJ...681..944A
2008PhLB..668..253M
2008Sci...322.1221A (I am not sure, is Crab pulsar equal to Crab nebula?)
2008ApJ...675L..25A (What is the correct source_id?)
2008NIMPA.588..424A
2008ApJ...685L..71A
2008NIMPA.594..407A
2009ApJ...697.1299A
2009PASJ...61.1011S (The source is not defined in gamma-cat, I think)
2009ApJ...702..266A (No source that matches the one in the paper)
2009A&A...498...83A
2009APh....30..293A
2010ApJ...721..843A
2010A&A...517A...5A
2010ApJ...725.1629A
2010ApJ...710..828A
2011ApJ...735L...5A (Don't know which source is meant)
2011ApJ...729..115A
2011ApJ...742...43A (Crab pulsar equal to Crab Nebula?)
2011JCAP...06..035A
2012A&A...541A..99A
2012APh....35..435A
2012A&A...540A..69A (Crab pulsar equal to Crab Nebula?)
2012NIMPA.692..201D
2013A&A...552A.112A
2013A&A...549A..23A (no matching source given in gamma-cat)
2014ApJ...786..157A (no matching source given in gamma-cat)
2014MNRAS.437.3103A
2014A&A...567A..41A
2014A&A...565L..12A (Crab pulsar equal to Crab Nebula?)
2014A&A...568A.109A (no matching source given in gamma-cat)
2014NIMPA.742...99D
2014JCAP...02..008A (no matching source given in gamma-cat)
2014A&A...567L...8A
2014MNRAS.440..530A
2014A&A...569A..46A
2014A&A...572A.121A
2015ApJ...815L..23A (no matching source given in gamma-cat)
2015A&A...576A..36A
2015A&A...578A..22A
2015A&A...582A..67A
2015ApJ...812...65F
2015arXiv150902391P
2016APh....72...61A
2016APh....72...76A
2016A&A...585A.133A (Crab pulsar equal to Crab nebula?)
2016A&A...591A.138A
2016A&A...593A..91A
2016JCAP...02..039M
2016MNRAS.459.3271A
2016A&A...595A..98A
2016JInst..1111009.
2017MNRAS.470.4608A
2017A&A...603A..29A
2017A&A...602A..98A (no matching source in gamma-cat)
2017MNRAS.471.1688A
2017A&A...601A..33A
2017arXiv170803689M (no matching source in gamma-cat)

@micheledoro
Copy link
Collaborator

micheledoro commented Sep 19, 2017

Hi, this is Michele Doro (currently publication manager of MAGIC). I can help you with MAGIC papers (or find somebody). Note you can cross-check MAGIC Publication information with the NASA ADS archive: ui.adsabs.harvard.edu/#/public-libraries/7QbEh28fSY-6jzxej_D7mw

@micheledoro
Copy link
Collaborator

Hi again, I hope somebody is reading this. I discussed yesterday with people from the SSDC (ASI science data center). They told me they will contact you. They started a similar catalog some time ago: http://www.asdc.asi.it/tgevcat/. To me it looks like they have already developed the "graphical" part (table and skymap), while gamma-cat is best for the repository part.

@micheledoro
Copy link
Collaborator

In addition, were the HESS and VERITAS data information cross-checked before entering the gamma-cat database?

@GernotMaier
Copy link
Collaborator

Some of the VERITAS data has been cross checked, but not all. This is indeed important.

@micheledoro
Copy link
Collaborator

MAGIC is going to a) cross-check previous generate fits file, and possibly regenerate fits file, b) input the re-generate fits file in the format for gamma-cat. We discussed that we would like to define precisely (more than in the past) what and how to write. Things like energy threshold, moon status, tag for spectral fit, etc. All this needs definition.

@pdeiml
Copy link
Collaborator Author

pdeiml commented Sep 30, 2017

I think, we have to find an easy and in particular reliable way to maintain the overview about all relevant papers/ data and which of them have been cross checked and which not.

I thought about writing an list like:
https://github.com/gammapy/gamma-cat/blob/master/todo/todo_hess_aux.md
for VERITAS and MAGIC publications to have a general overview. In my opinion the list of all relevant papers should not be hided in an issue like this one, it should be somewhere in the code itself.
Unfortunately, I do not have a good solution for the "what has been cross-checked" problem.

I do not have any experience with such an project and how to maintain the overview as soon as many people start to development, hence: What do you think?

@micheledoro
Copy link
Collaborator

@elisaprandini will also contribute to this issue with MAGIC

@cdeil
Copy link
Contributor

cdeil commented Oct 16, 2017

@micheledoro @elisaprandini - It's great that you want to contribute MAGIC data to gamma-cat.

In addition, were the HESS and VERITAS data information cross-checked before entering the gamma-cat database?

Data entry goes via "pull requests", so there is some review as data is added.

Things like energy threshold, moon status, tag for spectral fit, etc. All this needs definition.

Ideally, yes, as much information as possible would be available in the files. But there also has to be a balance between usefulness of the information for most users of gamma-cat, and time spent to define what each piece of information means and to do and review the data entry.

What we've done so far is to start simple, and keep adding fields of information as the use cases arise. We can and should add more information for a given measurement / paper, I just want to caution against the goal to be complete and get the data format schema right from the start. I think we have to improve iteratively and it's best to start small and simple.

@pdeiml and I have started to write some information with the goal to help people get started with data entry for gamma-cat:
http://gamma-cat.readthedocs.io/contribute/introduction.html
http://gamma-cat.readthedocs.io/contribute/input.html
It's still very rough, but maybe you could read over those two pages and let us know if it's enough to get your started or what is unclear.

If you want, we could also do a short Skype call where we discuss a bit and help you get started doing a first "pull request" with data entry for one MAGIC paper / source?

@micheledoro
Copy link
Collaborator

@elisaprandini and I have checked all MAGIC TeVCat sources against those in gamma-cat, and found that the following published papers were not included:

Additionally we found the following unpublished discoveries:

  • S2 0109+22
  • OT 081
  • 1ES 2037+521
  • TON 0599
  • TXS 0506+056

@elisaprandini and I will try to include these one first. We will later on continue with:
1/ check whether multiple papers on sources are properly accounted for
2/ check all MAGIC entries
3/ add upper limits papers

@cdeil cdeil assigned micheledoro and unassigned pdeiml Apr 3, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants