Skip to content

Commit

Permalink
Reverse the direction of the ParticleID relation(s) (#268)
Browse files Browse the repository at this point in the history
* Remove the ParticlID from the Cluster

* Remove the ParticleIDUsed from the ReconstructedParticle

* Remove the ParticleIDs from the reco particles

* Add a relation from a PID to a reco particle

* Add a first version of a PIDHandler

* Fix typo

Co-authored-by: Juan Miguel Carceller <[email protected]>

* Add basic tests for PIDHandler

* Add a bit of metadata handling to PIDHandler

* Add testing for PIDHandler metadata handling

* Simplify signatures a bit by introducing helper struct

* Fix README links once again

* Add helper function to get all meta info in one go

* Make it possible to add meta information post-hoc

* Make retrieving param indices free function

* More verbose docstrings

* Add short document describing PIDHandler and its usage

* Fix grammar and be more specific

Co-authored-by: Andre Sailer <[email protected]>

* fix typos in README

Co-authored-by: Mateusz Jakub Fila <[email protected]>

* Fix usage example code

* Do not create an unnecessary intermediate vector

* Add section about how (not) to use the PIDHandler

* Try to not advertise optional misuse

---------

Co-authored-by: Juan Miguel Carceller <[email protected]>
Co-authored-by: Andre Sailer <[email protected]>
Co-authored-by: Mateusz Jakub Fila <[email protected]>
  • Loading branch information
4 people authored May 1, 2024
1 parent 162c858 commit 9878b16
Show file tree
Hide file tree
Showing 9 changed files with 727 additions and 9 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,9 @@ A generic event data model for future HEP collider experiments.
|-|-|-|
| [EventHeader](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L172) | [MCParticle](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L184) | [SimTrackerHit](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L252) |
| [CaloHitContribution](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L294) | [SimCalorimeterHit](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L306) | [RawCalorimeterHit](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L318) |
| [CalorimeterHit](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L327) | [ParticleID](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L339) | [Cluster](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L352) |
| [TrackerHit3D](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L373) | [TrackerHitPlane](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L388) | [RawTimeSeries](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L407) |
| [Track](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L420) | [Vertex](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L439) | [ReconstructedParticle](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L456) |
| [CalorimeterHit](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L327) | [ParticleID](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L339) | [Cluster](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L354) |
| [TrackerHit3D](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L375) | [TrackerHitPlane](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L390) | [RawTimeSeries](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L409) |
| [Track](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L422) | [Vertex](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L441) | [ReconstructedParticle](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L458) |
| [SimPrimaryIonizationCluster](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L564) | [TrackerPulse](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L598) | [RecIonizationCluster](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L611) |
| [TimeSeries](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L622) | [RecDqdx](https://github.com/key4hep/EDM4hep/blob/main/edm4hep.yaml#L634) | |

Expand Down
239 changes: 239 additions & 0 deletions doc/PIDHandler.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
# PIDHandler introduction and usage

This page contains some bigger picture introduction for the utilities that are
available to work with `ParticleID`s and its related metadata. It also contains
examples for the most common usage patterns.

## PIDHandler basics
The `PIDHandler` can be use to work with `ParticleIDCollection`s. The main
features are
- the retrieval meta information for a `ParticleIDCollection`, making it
possible to e.g. retrieve the indices of parameters from parameter names
- the possibility to invert the relation to `ReconstructedParticle`s, which
allows one to, e.g. get all `ParticleID`s that point to a specific
`ReconstructedParticle`.

### When (not) to use the PIDHandler
NOTE: Depending on how you use the `PIDHandler` it might incur an unnecessary
performance overhead. The main purpose is to use it in cases, where the relation
between `ParticleID`s and `ReconstructedParticle`s is not trivial, e.g.
- When the two collections do **not** run in parallel
- When not all `ReconstructedParticle`s have a `ParticleID` attached
- When you want to have access to several `ParticleID`s for a given
`ReonstructedParticle` and there is no trivial relation between them.

In case your collections run in parallel it will be much quicker to just loop
over them in parallel.

In case you just want look at one `ParticleID` collection, but need to look at
some properties of the `ReconstructedParticle`, simply use the existing relation
to do that.


## ParticleIDMeta basics
`ParticleIDMeta` is a simple struct that bundles all ParticleID meta information
for one collection together. Whenever metadata is involved for ParticleIDs this
will be the thing to use.

## General considerations

There is no strictly enforced coupling between the meta information that is set
for a collection and the contents of that collection. In order for everything to
work as expected the following assumptions have to be met
- The collection name that is passed to the `setAlgoInfo` methods to set meta
information has to match the collection name used for putting collections into
events.
- All elements of a `ParticleIDCollection` have the same value set for
`algorithmType`. Additionally, it is usually assumed that this is a unique
value.
- The `ParticleIDMeta::paramNames` is assumed to match the parameters as they
are set in the `ParticleID` elements.

Additionally there are a few usage considerations to be kept in mind
- The `PIDHandler` can be used without any meta data about the contained
`ParticleID`s. However, in this case it will only be useful for inverting
relations and for getting `ParticleID`s that point to a given
`ReconstructedParticle`.
- There is no guarantee that the meta information that is set in a `PIDHandler`
is consistent with the elements it knows about. Some constructors attempt to
construct a consistent handler, but it is always possible to set meta
information via `setMetaInfo`, which might be completely unrelated to the
actual contents of a handler.

Given that there is no enforced consistency the interface and the utility
functionality makes heavy use of
[`std::optional`](https://en.cppreference.com/w/cpp/utility/optional) in its
return values / types. This makes it possible to check whether the desired
information was actually present, without having to reserve some special values
to indicate that fact.

## Usage examples

**The following examples assume the following includes** (which will not be visible in the examples)

```cpp
#include <edm4hep/utils/ParticleIDUtils.h>
#include <edm4hep/ParticleIDCollection.h>

#include <podio/Frame.h>
```

**The following examples additionally assume, that there is a `metadata` and an
`event` Frame present.** The `metadata` Frame can be obtained like this from
standard EDM4hep files.

```cpp
#include <podio/ROOTReader.h>
#include <podio/Frame.h>

podio::Frame getMetadataFrame(const std::string& filename) {
podio::ROOTReader reader{};
reader.openFile(filename);
return podio::Frame(reader.readNextEntry(podio::Category::Metadata));
}
```
Alternatively it can be created as an empty Frame and then just written using
the `podio::Category::Metadata` category name.
**Finally most of the examples assume that the desired values were found and
simply get the `value` from the returned `std::optional` directly.** This means
that most of the times you will see lines like this
```cpp
const auto value = pidHandler.getAlgoType("someAlgo").value();
```

This will throw an exception if some information is not available. Check if the
optional has a value when actually using these utilities.

### Creating a PIDHandler for a complete event

If you want to work with a `PIDHandler` that has a somewhat global view of all
`ParticleID`s that are present you can simply construct one from an event Frame

```cpp
const auto pidHandler = edm4hep::utils::PIDHandler::from(event);
```

You can also construct a `pidHandler` that populates some meta information
internally by also passing in the `metadata` Frame to this constructor
```cpp
const auto pidHandler = edm4hep::utils::PIDHandler::from(event, metadata);
```

### Creating a PIDHandler from an arbitrary number of collections

If you simply want to use a `PIDHandler` to invert the relations for a given set
of `ParticleIDCollection`s you can do that via

```cpp
const auto& tofPID = event.get<edm4hep::ParticleIDCollection>("ToFPID");
const auto& dNdXPID = event.get<edm4hep::ParticleIDCollection>("dNdXPID");
const auto& mvaPID = event.get<edm4hep::ParticleIDCollection>("mvaPID");

const auto pidHandler = edm4hep::utils::PIDHandler::from(tofPID, dNdXPID, mvaPID);
```

**This handler will not have any access to metadata!** If you need metadata as
well, you can add that after the fact using the `addMetaInfo` method.

### Using a PIDHandler to get the ParticleIDs for a reco particle

If you have constructed a `PIDHandler` you can use it to get all or specific
related `ParticleID` objects of a `ReconstructedParticle`

```cpp
const auto pidHandler = /* constructed as above somehow */;

const auto recos = event.get<edm4hep::ReconstructedParticle>("recos");
const auto reco = recos[0];

// Get all related ParticleIDs
const auto pids = pidHandler.getPIDs(reco);

// Only get a specific ParticleID
const auto tofAlgoType = pidHandler.getAlgoType("ToFPID").value();
const auto tofPID = pidHandler.getPID(reco, tofAlgoType).value();
```

### Retrieving meta information for a collection

If you simply want to get the meta information for a collection (called "ToFPID"
in this example) do

```cpp
const auto algoInfo = edm4hep::utils::PIDHandler::getAlgoInfo(metadata, "ToFPID").value();
```


### Using meta information of a collection

If you have retrieved the meta information you can directly use that to learn
something about the `ParticleID`s it describes

```cpp
const auto& tofPIDs = event.get<edm4hep::ParticleIDCollection>("ToFPID");
const auto algoInfo = edm4hep::utils::PIDHandler::getAlgoInfo(metadata, "ToFPID").value();
const auto timeIndex = edm4hep::utils::getParamIndex(algoInfo, "time").value();

for (const auto tof : tofPIDs) {
const auto time = tof.getParameters(timeIndex);
// ...
}
```

If you have a `PIDHandler` with enough meta information set it is also possible
to retrieve the `timeIndex` from that

```cpp
const auto pidHandler = /* construct somehow see above */
const auto tofAlgoType = pidHandler.getAlgoType("ToFPID").value();
const auto timeIndex = pidHandler.getParamIndex("time").value();

// ... as above
```


### Setting meta information for a collection

If you have a `ParticleIDCollection` and want to persist some meta information
for it the following snippet shows you how to do that

```cpp
// Create ParticleID meta information for an algorithm
const auto tofMeta = edm4hep::utils::ParticleIDMeta{"TimeOfFlight", 42, {"time", "time_error"}};

auto tofPIDs = edm4hep::ParticleIDCollection{};
// ... fill collection

edm4hep::utils::PIDHandler::setAlgoInfo(metadata, tofPIDs, "ToFPID", tofMeta);
event.put(std::move(tofPIDs), "ToFPID");
```
This will put the necessary metadata into the `metadata` Frame and also take
care of setting the `algorithmType` field of all elements of the `tofPIDs`
collection to `42` (the value it is set to in the `tofMeta` object).
**Things to note:**
- It is important to use the same names for putting the collection into the
`event` Frame and the one that is passed to `PIDHandler::setAlgoInfo`!
### Setting meta information using a collection name
If you want to set the meta information for a `ParticleIDCollection` that you no
longer have mutable access to, this can be done via
```cpp
// Create ParticleID meta information for an algorithm
const auto tofMeta = edm4hep::utils::ParticleIDMeta{"TimeOfFlight", 42, {"time", "time_error"}};
edm4hep::utils::PIDHandler::setAlgoInfo(metadata, "ToFPID", tofMeta);
```

**Things to note:**
- In this case it is user responsibility to set the `algoType` of the
`ParticleIDMeta` to the same value as the one that is used for the elements of
the collection.
- It is important to use the same names for putting the collection into the
`event` Frame and the one that is passed to `PIDHandler::setAlgoInfo`!
6 changes: 3 additions & 3 deletions edm4hep.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,8 @@ datatypes:
- float likelihood // likelihood of this hypothesis - in a user defined normalization
VectorMembers:
- float parameters // parameters associated with this hypothesis
OneToOneRelations:
- edm4hep::ReconstructedParticle particle // the particle from which this PID has been computed


#------ Cluster
Expand All @@ -367,7 +369,7 @@ datatypes:
OneToManyRelations:
- edm4hep::Cluster clusters // clusters that have been combined to this cluster
- edm4hep::CalorimeterHit hits // hits that have been combined to this cluster
- edm4hep::ParticleID particleIDs // particle IDs (sorted by their likelihood)


#------------- TrackerHit
edm4hep::TrackerHit3D:
Expand Down Expand Up @@ -467,12 +469,10 @@ datatypes:
- std::array<float,10> covMatrix // cvariance matrix of the reconstructed particle 4vector (10 parameters). Stored as lower triangle matrix of the four momentum (px,py,pz,E), i.e. cov(px,px), cov(py,##
OneToOneRelations:
- edm4hep::Vertex startVertex // start vertex associated to this particle
- edm4hep::ParticleID particleIDUsed // particle Id used for the kinematics of this particle
OneToManyRelations:
- edm4hep::Cluster clusters // clusters that have been used for this particle
- edm4hep::Track tracks // tracks that have been used for this particle
- edm4hep::ReconstructedParticle particles // reconstructed particles that have been combined to this particle
- edm4hep::ParticleID particleIDs // particle Ids (not sorted by their likelihood)
ExtraCode:
declaration: "
bool isCompound() const { return particles_size() > 0 ;}\n
Expand Down
6 changes: 6 additions & 0 deletions include/edm4hep/Constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ static constexpr const char* CellIDEncoding = "CellIDEncoding";
static constexpr const char* EventHeaderName = "EventHeader";
static constexpr const char* EventWeights = "EventWeightNames";
static constexpr const char* shapeParameterNames = "shapeParameterNames";

/// The collection parameter name for accessing the names of the parameters for
/// a ParticleID collection
static constexpr const char* pidParameterNames = "ParameterNames";
static constexpr const char* pidAlgoName = "AlgoName";
static constexpr const char* pidAlgoType = "AlgoType";
} // namespace edm4hep

#endif // EDM4HEP_CONSTANTS_H
2 changes: 1 addition & 1 deletion test/utils/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ endif()
include(Catch)

add_executable(unittests_edm4hep
test_kinematics.cpp test_vector_utils.cpp)
test_kinematics.cpp test_vector_utils.cpp test_PIDHandler.cpp)
target_link_libraries(unittests_edm4hep edm4hep EDM4HEP::utils Catch2::Catch2 Catch2::Catch2WithMain)

option(SKIP_CATCH_DISCOVERY "Skip the Catch2 test discovery" OFF)
Expand Down
Loading

0 comments on commit 9878b16

Please sign in to comment.