Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make "always read EventHeader" collection more robust #161

Merged
merged 6 commits into from
Nov 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions k4FWCore/components/PodioInput.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -184,14 +184,15 @@ PodioInput::PodioInput(const std::string& name, ISvcLocator* svcLoc) : Consumer(
error() << "Could not get PodioDataSvc" << endmsg;
}
fillReaders();

auto key = edm4hep::EventHeaderName;
if (std::find(m_collectionNames.begin(), m_collectionNames.end(), key) == m_collectionNames.end()) {
m_collectionNames.value().push_back(key);
}
}

void PodioInput::operator()() const {
if (m_podioDataSvc->getEventFrame().get(edm4hep::EventHeaderName)) {
m_readers[edm4hep::EventHeaderCollection::typeName](edm4hep::EventHeaderName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about doing it this way initially, what happens if the EventHeader collection is also in the list of collections to read? Will it be read twice?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would query the underlying Frame again for a second time. However, the second time around that effectively is a map lookup and return. Technically, hasCollection does all the "heavy lifting" of prompting the Frame to actually read the data and populate it's internal map. Most importantly you will always get the same collection.

I quickly checked what happens if I explicitly make the test read the EventHeader collection and nothing seems to have broken, but it looks like we would be putting two DataHandles pointing to the same collection into the EDS at the moment. I have to check why that seems to work without breaking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We indeed do get two DataHandles (and corresponding objects in the EDS) if EventHeader is explicitly passed in. However, since the collection is owned by the Frame and the DataSvc defers the cleanup to that, there is at least no double free.

I will add a simple filter to clean out duplicate collection names as well as "EventHeader". Additionally, I will adapt the PodioDataSvc::readCollection to check for existence in the EDS before populating it twice.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't do the filtering in the end, because it is effectively unnecessary with the guard against duplicate entries in the EDS.

} else {
info() << "No EventHeader collection found in the event. Not reading it" << endmsg;
}

for (auto& collName : m_collectionNames) {
debug() << "Registering collection to read " << collName << endmsg;
auto type = m_podioDataSvc->getCollectionType(collName);
Expand Down
5 changes: 5 additions & 0 deletions k4FWCore/include/k4FWCore/PodioDataSvc.h
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@ class PodioDataSvc : public DataSvc {
const std::string_view getCollectionType(const std::string& collName);

template <typename T> StatusCode readCollection(const std::string& collName) {
DataObject* objectPtr = nullptr;
if (DataSvc::findObject("/Event", "/" + collName, objectPtr)) {
debug() << "Collection " << collName << " already read, not reading it again" << endmsg;
return StatusCode::SUCCESS;
}
const T* collection(nullptr);
collection = static_cast<const T*>(m_eventframe.get(collName));
if (collection == nullptr) {
Expand Down
1 change: 1 addition & 0 deletions test/k4FWCoreTest/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ add_test_with_env(TestDataHandleUniquePtr options/TestDataHandleUniquePtr.py)
add_test_with_env(TestUniqueIDGenSvc options/TestUniqueIDGenSvc.py)
add_test_with_env(CreateLegacyExampleEventData options/createLegacyExampleEventData.py)
add_test_with_env(TestEventHeaderFiller options/createEventHeader.py)
add_test_with_env(EventHeaderCheck options/runEventHeaderCheck.py PROPERTIES DEPENDS TestEventHeaderFiller)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look at how easy it is to add one test now...

add_test(NAME TestExec WORKING_DIRECTORY ${CMAKE_CURRENT_LIST_DIR} COMMAND python options/TestExec.py)
set_test_env(TestExec)

Expand Down
43 changes: 43 additions & 0 deletions test/k4FWCoreTest/options/runEventHeaderCheck.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
#!/usr/bin/env python3
#
# Copyright (c) 2014-2023 Key4hep-Project.
#
# This file is part of Key4hep.
# See https://key4hep.github.io/key4hep-doc/ for further info.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

from Gaudi.Configuration import INFO
from Configurables import k4DataSvc
from Configurables import PodioInput
from Configurables import ExampleEventHeaderConsumer
from Configurables import ApplicationMgr

podioevent = k4DataSvc("EventDataSvc")
podioevent.input = "eventHeader.root"

inp = PodioInput()
inp.collections = []

consumer = ExampleEventHeaderConsumer(
"EventHeaderCheck", runNumber=42, eventNumberOffset=42
)

ApplicationMgr(
TopAlg=[inp, consumer],
EvtSel="NONE",
EvtMax=-1,
ExtSvc=[podioevent],
OutputLevel=INFO,
)
68 changes: 68 additions & 0 deletions test/k4FWCoreTest/src/components/ExampleEventHeaderConsumer.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/*
* Copyright (c) 2014-2023 Key4hep-Project.
*
* This file is part of Key4hep.
* See https://key4hep.github.io/key4hep-doc/ for further info.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

#include "edm4hep/Constants.h"
#include "edm4hep/EventHeaderCollection.h"

#include "k4FWCore/BaseClass.h"

#include <Gaudi/Property.h>
#include <GaudiAlg/Consumer.h>
#include <GaudiKernel/ISvcLocator.h>

#include <fmt/core.h>
#include <fmt/format.h>

#include <atomic>
#include <stdexcept>
#include <string>

struct ExampleEventHeaderConsumer final
: Gaudi::Functional::Consumer<void(const edm4hep::EventHeaderCollection&), BaseClass_t> {
ExampleEventHeaderConsumer(const std::string& name, ISvcLocator* svcLoc)
: Consumer(name, svcLoc, {KeyValue("EventHeaderName", edm4hep::EventHeaderName)}) {}

void operator()(const edm4hep::EventHeaderCollection& evtHeaderColl) const {
if (evtHeaderColl.empty()) {
throw std::runtime_error("EventHeader collection is empty");
}
const auto evtHeader = evtHeaderColl[0];
if (!evtHeader.isAvailable()) {
throw std::runtime_error("Cannot get a valid EventHeader");
}

if (evtHeader.getRunNumber() != m_runNumber) {
throw std::runtime_error(fmt::format("Run number is not set correctly (expected {}, actual {})",
m_runNumber.value(), evtHeader.getRunNumber()));
}

const auto expectedEvent = m_evtCounter++ + m_eventNumberOffset;
if (evtHeader.getEventNumber() != expectedEvent) {
throw std::runtime_error(fmt::format("Event number is not set correctly (expected {}, actual {})", expectedEvent,
evtHeader.getEventNumber()));
}
}

Gaudi::Property<int> m_runNumber{this, "runNumber", 0, "The expected run number"};
Gaudi::Property<int> m_eventNumberOffset{this, "eventNumberOffset", 0,
"The event number offset where events will start counting from"};
mutable std::atomic<unsigned> m_evtCounter{0};
};

DECLARE_COMPONENT(ExampleEventHeaderConsumer)
Loading