Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make "always read EventHeader" collection more robust #161

Merged
merged 6 commits into from
Nov 21, 2023

Conversation

tmadlener
Copy link
Contributor

@tmadlener tmadlener commented Nov 16, 2023

BEGINRELEASENOTES

  • Make sure that the EventHeader is always read when it is available, but also to not read it if it is not available.
  • Add a test case that checks that the EventHeaderFiller (introduced in Add a Gaudi Algorithm filling EventHeader eventNumber and runNumber #133) are correct when reading back.
  • Make sure that the PodioDataSvc only reads and registers collections to the EDS once even in case there are duplicates.

ENDRELEASENOTES

#160 introduced a bug that went undetected in tests, because all the tests apparently override the list of collections that is set in the PodioInput constructor. Using an empty (default) list and an input file that doesn't contain the EventHeader collection leads to Gaudi exiting early because it cannot retrieve the collection from the event store.

This PR adds a simple consumer that makes sure that things run, even if the input file does not have an EventHeader collection.

Overall there is a discussion to be head whether making the EventHeader a datatype like all the rest is really the best design, as it is technically rather part of the "metadata" that describe an Event rather than content of it.

  • Actually implement the fix
  • Release Notes

}

void PodioInput::operator()() const {
if (m_podioDataSvc->hasCollection(edm4hep::EventHeaderName)) {
m_readers[edm4hep::EventHeaderCollection::typeName](edm4hep::EventHeaderName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about doing it this way initially, what happens if the EventHeader collection is also in the list of collections to read? Will it be read twice?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would query the underlying Frame again for a second time. However, the second time around that effectively is a map lookup and return. Technically, hasCollection does all the "heavy lifting" of prompting the Frame to actually read the data and populate it's internal map. Most importantly you will always get the same collection.

I quickly checked what happens if I explicitly make the test read the EventHeader collection and nothing seems to have broken, but it looks like we would be putting two DataHandles pointing to the same collection into the EDS at the moment. I have to check why that seems to work without breaking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We indeed do get two DataHandles (and corresponding objects in the EDS) if EventHeader is explicitly passed in. However, since the collection is owned by the Frame and the DataSvc defers the cleanup to that, there is at least no double free.

I will add a simple filter to clean out duplicate collection names as well as "EventHeader". Additionally, I will adapt the PodioDataSvc::readCollection to check for existence in the EDS before populating it twice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't do the filtering in the end, because it is effectively unnecessary with the guard against duplicate entries in the EDS.

@tmadlener
Copy link
Contributor Author

Unless there are any complaints until this evening I would merge this and then tag k4FWCore

@@ -95,6 +95,7 @@ add_test_with_env(TestDataHandleUniquePtr options/TestDataHandleUniquePtr.py)
add_test_with_env(TestUniqueIDGenSvc options/TestUniqueIDGenSvc.py)
add_test_with_env(CreateLegacyExampleEventData options/createLegacyExampleEventData.py)
add_test_with_env(TestEventHeaderFiller options/createEventHeader.py)
add_test_with_env(EventHeaderCheck options/runEventHeaderCheck.py PROPERTIES DEPENDS TestEventHeaderFiller)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look at how easy it is to add one test now...

}

void PodioInput::operator()() const {
if (m_podioDataSvc->hasCollection(edm4hep::EventHeaderName)) {
Copy link
Member

@jmcarcell jmcarcell Nov 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it only happens one the hasCollection function can be removed, I think this should work:

Suggested change
if (m_podioDataSvc->hasCollection(edm4hep::EventHeaderName)) {
if (m_podioDataSvc->getEventFrame().get(edm4hep::EventHeaderName)) {

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@jmcarcell
Copy link
Member

I don't have any more comments; I think many parts need to be reworked anyway to be able to use the PodioHiveWhiteBoard so that is something we can change after the release

@tmadlener tmadlener merged commit bc8c064 into key4hep:main Nov 21, 2023
2 of 5 checks passed
@tmadlener tmadlener deleted the robust-event-header branch November 21, 2023 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants