Skip to content

Commit

Permalink
Overhaul python configuration of EDM conversion tools (#91)
Browse files Browse the repository at this point in the history
* Add control of conversion with flag and name mapping

Make it possible to convert everything but still change mapped names

* Update examples, tests and documentation
  • Loading branch information
tmadlener authored Nov 15, 2022
1 parent 4de0b14 commit 8c21c30
Showing 9 changed files with 260 additions and 251 deletions.
17 changes: 3 additions & 14 deletions doc/starterkit/k4MarlinWrapperCLIC/CEDViaWrapper.md
Original file line number Diff line number Diff line change
@@ -83,7 +83,8 @@ MyCEDViewer.Parameters = {

# EDM4hep to LCIO converter
edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio")
edmConvTool.Parameters = ["*"]
edmConvTool.convertAll = True
edmConvTool.collNameMapping = {'MCParticles': 'MCParticle'}
edmConvTool.OutputLevel = DEBUG
MyCEDViewer.EDM4hep2LcioTool = edmConvTool

@@ -117,16 +118,4 @@ The `event_display.py` options file that is used above and that is present in th
- Exchange the LCIO input reading by the podio input reading (see above)
- Attach the `EDM4hep2LcioTool` to the wrapped `CEDViewer` processor

This should allow to arrive at a similar steering file even for slightly different configurations. One potential pitfal is the slightly different naming of the `ddsim` outputs between LCIO and EDM4hep (see [this issue](https://github.com/AIDASoft/DD4hep/issues/921)). For the `event_display.py` in the `examples` drawing the `MCParticle`s in the event display required to change the name of the collection that is used in the `DrawInLayers` configuration parameter of the `CEDViewer`:

```diff
@@ -156,7 +156,7 @@ MyCEDViewer.Parameters = {
"MarlinTrkTracks", "0", "6", "7",
"PandoraClusters", "0", "3", "8",
"PandoraPFOs", "0", "3", "9",
- "MCParticle", "0", "3", "0",
+ "MCParticles", "0", "3", "0",
"VertexBarrelHits", "0", "5", "11",
"VertexEndcapHits", "0", "5", "11",
"InnerTrackerBarrelHits", "0", "5", "11",
```
This should allow one to arrive at a similar steering file even for slightly different configurations. One potential pitfall is the slightly different naming of the `ddsim` outputs between LCIO and EDM4hep (see [this issue](https://github.com/AIDASoft/DD4hep/issues/921)). This can be addressed by configuring the EDM4hep to LCIO converter (`edmConvTool` in the code above) to map the names accordingly.
44 changes: 23 additions & 21 deletions doc/starterkit/k4MarlinWrapperCLIC/edmConverters.md
Original file line number Diff line number Diff line change
@@ -117,17 +117,18 @@ Note and review the following when using the converters:
- If using the `PodioInput` to read events: collection names must be indicated both in the `PodioInput` and the `EDM4hep2LcioTool`.
- Collections not indicated to be converted **will not** be converted even if its a dependency from an indicated collection to be converted.
- If a converted collection is used later by a Gaudi Algorithm, and this Gaudi Algorithm indicates the use of that collection in the `Parameters`, the converted collection name must match the name indicated in the Gaudi Algorithm `Parameters`.
+ For example: A collection may be converted with the following parameters: `ReconstructedParticles", "ReconstructedParticleLCIO"`
+ For example: A collection may be converted with the following parameters: `"ReconstructedParticles": "ReconstructedParticleLCIO"`
+ A Gaudi Algorithm may indicate in their `Parameters`: `"PFOCollection": ["ReconstructedParticleLCIO"]`

### EDM4hep to LCIO converter

Collections from events that are already read, or are produced by a Gaudi Algorithm can be converted from EDM4hep to LCIO format:

1. Instantiate the `EDM4hep2LcioTool` Gaudi Tool.
2. Indicate the collections to convert in `Parameters`.
+ To convert all available collections write an asterisk, like follows: `edmConvTool.Parameters = ["*"]`
+ Arguments are read in groups of 2: name of the EDM4hep collection, name of the LCIO converted collection.
1. Instantiate the `EDM4hep2LcioTool` Gaudi Tool, e.g. as `edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio")`
2. Indicate the collections to convert using the options of the tool.
+ To simply convert all available collections use `edmConvTool.convertAll = True` (this is also the default!)
+ If you want to convert all available collections but want to rename some of them, e.g. because an algorithm expects a different name, use the `collNameMapping` option. This maps the input name to the output name; `edmConvTool.collNameMapping = {'MCParticles': 'MCParticle'}` will convert the input `'MCParticles'` into the `'MCParticle'` collection.
+ To convert only a subset of all available collections, set the `convertAll` option to `False` and indicate the collections to convert via the `collNameMapping` option.
3. Select the Gaudi Algorithm that will convert the indicated collections.
4. Add the Tool to the Gaudi Algorithm.

@@ -137,10 +138,11 @@ from Configurables import ToolSvc, EDM4hep2LcioTool
# 1
edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio")
# 2
edmConvTool.Parameters = [
"EFlowTrack", "EFlowTrack_LCIO",
"ReconstructedParticles", "ReconstructedParticle_LCIO"
]
edmConvToll.convertAll = False
edmConvTool.collNameMapping = {
"EFlowTrack": "EFlowTrack_LCIO",
"ReconstructedParticles": "ReconstructedParticle_LCIO"
}
edmConvTool.OutputLevel = DEBUG

# 3
@@ -154,10 +156,11 @@ InitDD4hep.EDM4hep2LcioTool=edmConvTool

Collections from events that are already read, or are produced by a gaudi Algorithm can be converted from LCIO to EDM4hep format:

1. Instantiate the `Lcio2EDM4hepTool` Gaudi Tool.
2. Indicate the collections to convert in `Parameters`.
+ To convert all available collections write an asterisk, like follows: `lcioConvTool.Parameters = ["*"]`
+ Arguments are read in groups of 2: name of the LCIO collection, name of the EDM4hep converted collection.
1. Instantiate the `Lcio2EDM4hepTool` Gaudi Tool, e.g. as `lcioConvTool = Lcio2EDM4hepTool("LCIO2EDM4hep")`.
2. Indicate the collections to convert in the options of the tool.
+ To simply convert all available collections use `lcioConvTool.convertAll = True` (this is also the default!)
+ If you want to convert all available collections but want to rename some of them, e.g. because an algorithm expects a different name, use the `collNameMapping` option. This maps the input name to the output name; `lcioConvTool.collNameMapping = {'MCParticles': 'MCParticle'}` will convert the input `'MCParticles'` into the `'MCParticle'` collection.
+ To convert only a subset of all available collections, set the `convertAll` option to `False` and indicate the collections to convert via the `collNameMapping` option.
3. Select the Gaudi Algorithm that will convert the indicated collections.
4. Add the Tool to the Gaudi Algorithm.

@@ -167,12 +170,13 @@ from Configurables import ToolSvc, Lcio2EDM4hepTool
# 1
lcioConvTool = Lcio2EDM4hepTool("LCIO2EDM4hep")
# 2
lcioConvTool.Parameters = [
"EFlowTrackConv", "EFlowTrackEDM4hep",
"ReconstructedParticle", "ReconstructedParticlesEDM4hep",
"BuildUpVertices", "BuildUpVerticesEDM4hep",
"PrimaryVertices", "PrimaryVerticesEDM4hep"
]
lcioConvTool.convertAll = False
lcioConvTool.collNameMapping = {
"EFlowTrackConv": "EFlowTrackEDM4hep",
"ReconstructedParticle": "ReconstructedParticlesEDM4hep",
"BuildUpVertices": "BuildUpVerticesEDM4hep",
"PrimaryVertices": "PrimaryVerticesEDM4hep"
}
lcioConvTool.OutputLevel = DEBUG

# 3
@@ -181,5 +185,3 @@ JetClusteringAndRefiner = MarlinProcessorWrapper("JetClusteringAndRefiner")
# 4
JetClusteringAndRefiner.Lcio2EDM4hepTool=lcioConvTool
```


5 changes: 3 additions & 2 deletions k4MarlinWrapper/examples/event_display.py
Original file line number Diff line number Diff line change
@@ -156,7 +156,7 @@
"MarlinTrkTracks", "0", "6", "7",
"PandoraClusters", "0", "3", "8",
"PandoraPFOs", "0", "3", "9",
"MCParticles", "0", "3", "0",
"MCParticle", "0", "3", "0",
"VertexBarrelHits", "0", "5", "11",
"VertexEndcapHits", "0", "5", "11",
"InnerTrackerBarrelHits", "0", "5", "11",
@@ -193,7 +193,8 @@

# EDM4hep to LCIO converter
edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio")
edmConvTool.Parameters = ["*"]
edmConvTool.convertAll = True
edmConvTool.collNameMapping = {'MCParticles': 'MCParticle'}
edmConvTool.OutputLevel = DEBUG
MyCEDViewer.EDM4hep2LcioTool = edmConvTool

28 changes: 15 additions & 13 deletions k4MarlinWrapper/k4MarlinWrapper/converters/EDM4hep2Lcio.h
Original file line number Diff line number Diff line change
@@ -1,25 +1,26 @@
#ifndef K4MARLINWRAPPER_EDM4HEP2LCIO_H
#define K4MARLINWRAPPER_EDM4HEP2LCIO_H

// std
#include <algorithm>
#include <bitset>
#include <string>
#include <vector>

// GAUDI
#include <GaudiAlg/GaudiTool.h>
// k4MarlinWrapper
#include "k4MarlinWrapper/LCEventWrapper.h"
#include "k4MarlinWrapper/converters/IEDMConverter.h"
#include "k4MarlinWrapper/util/k4MarlinWrapperUtil.h"

// FWCore
#include <k4FWCore/DataHandle.h>

//k4EDM4hep2LcioConv
#include "k4EDM4hep2LcioConv/k4EDM4hep2LcioConv.h"

// k4MarlinWrapper
#include "k4MarlinWrapper/LCEventWrapper.h"
#include "k4MarlinWrapper/converters/IEDMConverter.h"
#include "k4MarlinWrapper/util/k4MarlinWrapperUtil.h"
// GAUDI
#include <GaudiAlg/GaudiTool.h>

// std
#include <algorithm>
#include <bitset>
#include <map>
#include <string>
#include <vector>

class EDM4hep2LcioTool : public GaudiTool, virtual public IEDMConverter {
public:
@@ -31,7 +32,8 @@ class EDM4hep2LcioTool : public GaudiTool, virtual public IEDMConverter {
StatusCode convertCollections(lcio::LCEventImpl* lcio_event);

private:
Gaudi::Property<std::vector<std::string>> m_edm2lcio_params{this, "Parameters", {}};
Gaudi::Property<std::map<std::string, std::string>> m_collNames{this, "collNameMapping", {}};
Gaudi::Property<bool> m_convertAll{this, "convertAll", true};

PodioDataSvc* m_podioDataSvc;
ServiceHandle<IDataProviderSvc> m_eventDataSvc;
6 changes: 5 additions & 1 deletion k4MarlinWrapper/k4MarlinWrapper/converters/Lcio2EDM4hep.h
Original file line number Diff line number Diff line change
@@ -15,6 +15,9 @@
// Converter Interface
#include "k4MarlinWrapper/converters/IEDMConverter.h"

#include <map>
#include <string>

class Lcio2EDM4hepTool : public GaudiTool, virtual public IEDMConverter {
public:
Lcio2EDM4hepTool(const std::string& type, const std::string& name, const IInterface* parent);
@@ -25,7 +28,8 @@ class Lcio2EDM4hepTool : public GaudiTool, virtual public IEDMConverter {
StatusCode convertCollections(lcio::LCEventImpl* lcio_event);

private:
Gaudi::Property<std::vector<std::string>> m_params{this, "Parameters", {}};
Gaudi::Property<std::map<std::string, std::string>> m_collNames{this, "collNameMapping", {}};
Gaudi::Property<bool> m_convertAll{this, "convertAll", true};

std::map<std::string, DataObjectHandleBase*> m_dataHandlesMap;

41 changes: 19 additions & 22 deletions k4MarlinWrapper/src/components/EDM4hep2Lcio.cpp
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#include "k4MarlinWrapper/converters/EDM4hep2Lcio.h"
#include <unordered_map>

DECLARE_COMPONENT(EDM4hep2LcioTool);

@@ -270,34 +271,30 @@ void EDM4hep2LcioTool::convertAdd(const std::string& e4h_coll_name, const std::s
// Parse property parameters and convert the indicated collections.
// Use the collection names in the parameters to read and write them
StatusCode EDM4hep2LcioTool::convertCollections(lcio::LCEventImpl* lcio_event) {
CollectionsPairVectors collection_pairs{};
const auto collections = m_podioDataSvc->getCollections();

// Convert collections from parameters
if (m_edm2lcio_params.size() > 1 && m_edm2lcio_params.size() % 2 == 0) {
for (int i = 0; i < m_edm2lcio_params.size(); i = i + 2) {
if (!collectionExist(m_edm2lcio_params[i + 1], lcio_event)) {
convertAdd(m_edm2lcio_params[i], m_edm2lcio_params[i + 1], lcio_event, collection_pairs);
} else {
debug() << " Collection " << m_edm2lcio_params[i + 1] << " already in place, skipping conversion. " << endmsg;
}
// Start off with the pre-defined collection name mappings
auto collsToConvert{m_collNames.value()};
if (m_convertAll) {
info() << "Converting all collections from EDM4hep to LCIO" << endmsg;
// And simply add the rest, exploiting the fact that emplace will not
// replace existing entries with the same key
for (const auto& [name, _] : collections) {
collsToConvert.emplace(name, name);
}

FillMissingCollections(collection_pairs);
}

// Convert all collections if the only parameter is "*"
else if (m_edm2lcio_params.size() == 1 && m_edm2lcio_params[0] == "*") {
info() << "Converting all collections from EDM4hep to LCIO" << endmsg;
CollectionsPairVectors collection_pairs{};
for (const auto& [edm4hepName, lcioName] : collsToConvert) {
if (!collectionExist(lcioName, lcio_event)) {
convertAdd(edm4hepName, lcioName, lcio_event, collection_pairs);
} else {
debug() << " Collection " << lcioName << " already in place, skipping conversion. " << endmsg;
}

const auto& collections = m_podioDataSvc->getCollections();
for (auto& [name, collection] : collections) {
convertAdd(name, name, lcio_event, collection_pairs);
if (!m_convertAll) {
FillMissingCollections(collection_pairs);
}
} else {
error() << " Error processing conversion parameters. 2 arguments (EDM4hep "
"name, LCIO name) per collection, or 1 argument \"*\" to convert all collections expected. "
<< endmsg;
return StatusCode::FAILURE;
}

return StatusCode::SUCCESS;
Loading

0 comments on commit 8c21c30

Please sign in to comment.