Introduce a new links category for the YAML definitions

README.md

@@ -34,7 +34,7 @@ version and build that automatically. This behavior is controlled via the
 `USE_EXTERNAL_CATCH2` cmake variable. It defaults to `AUTO` but can also be set
 to `ON` or `OFF` to fully control the desired behavior.
 
-### Python > 3.6
+### Python >= 3.8
 
 Check your Python version by doing:
 

doc/datamodel_syntax.md

@@ -155,6 +155,38 @@ define which `Types` can be used with this interface class, in this case the
 not allow for mutable access to their data.** They can be used in relations
 between objects, just like normal `datatypes`.
 
+## Definition of links
+Podio offers a templated `Link` class ([see here for more details](links.md))
+that allows one to link two arbitrary datatypes without having to introduce a
+`OneToOneRelation` or `OneToManyRelation` inside the corresponding datatypes. In
+order to keep the full definition of a datamodel in the YAML file it is possible
+to declare `links` in the YAML file:
+
+```yaml
+    links:
+      ExampleLink:
+        Description: "A link between two (podio generated) objects"
+        Author: "It could be you"
+        From: ExampleHit
+        To: TypeWithEnergy
+```
+
+This definition will yield the following typedefs
+```cpp
+using ExampleLinkCollection = podio::LinkCollection<ExampleHit, TypeWithEnergy>;
+
+using ExampleLink = typename ExampleLinkCollection::value_type;
+// this is equivalent to
+// using ExampleLink = podio::Link<ExampleHit, TypeWithEnergy>;
+
+using MutableExampleLink = typename ExampleLinkCollection::mutable_type;
+// this is equivalent to
+// using MutableExampleLink = podio::MutableLink<ExampleHit, TypeWithEnergy>;
+```
+
+additionally, this will generate the necessary code to enable I/O for this link
+type.
+
 ### Assigning to interface types
 
 Interface types support the same functionality as normal (immutable) datatypes.

doc/index.rst

@@ -15,6 +15,7 @@ Welcome to PODIO's documentation!
    examples.md
    frame.md
    userdata.md
+   links.md
    storage_details.md
    cmake.md
    advanced_topics.md

doc/links.md

@@ -45,12 +45,16 @@ For a more detailed explanation of the internals and the actual implementation
 see [the implementation details](#implementation-details).
 
 ## How to use `Link`s
-Using `Link`s is quite simple. In line with other datatypes that are
-generated by podio all the functionality can be gained by including the
-corresponding `Collection` header. After that it is generally recommended to
-introduce a type alias for easier usage. **As a general rule `Links` need
-to be declared with the default (immutable) types.** Trying to instantiate them
-with `Mutable` types will result in a compilation error.
+Using `Link`s is quite simple. The most straight forward way is to simply
+declare them as part of the datamodel, [as described
+here](datamodel_syntax.md#definition-of-links). That will result in code
+generation that effectively does what is described here. However, it's not
+strictly necessary to do that in case non-generated code is preferred. In line
+with other datatypes that are generated by podio all the functionality can be
+gained by including the corresponding `Collection` header. After that it is
+generally recommended to introduce a type alias for easier usage. **As a general
+rule `Links` need to be declared with the default (immutable) types.** Trying to
+instantiate them with `Mutable` types will result in a compilation error.
 
 ```cpp
 #include "podio/LinkCollection.h"

doc/templates.md

@@ -26,19 +26,22 @@ Note that some of the information below will only apply to either of these gener
 Currently PODIO loads templates that are placed in [`<prefix>/python/templates`](/python/templates).
 They are broadly split along the classes that are generated for each datatype or component from the EDM definition:
 
-| template file(s)                | content                                                                                                                                                    | generated file(s)                                                                     |
-|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|
-| `Component.h.jinja2`            | Definition for each component                                                                                                                              | `[<package>/]<component-name>.h`                                                      |
-| `Data.h.jinja2`                 | POD struct of each datatype (living in the POD layer)                                                                                                      | `[<package>/]<datatype-name>Data.h`                                                   |
-| `Obj.{h,cc}.jinja2`             | `Obj` class for each datatype (living in the object layer) and managing resources                                                                          | `[<package>/]<datatype-name>Obj.h`,  `src/<datatype-name>Obj.cc`                      |
-| `[Mutable]Object.{h,cc}.jinja2` | The user facing interfaces for each datatype (living in the user layer)                                                                                    | `[<package>/][Mutable]<datatype-name>.h`, `src/[Mutable]<datatype-name>.cc`           |
-| `Collection.{h,cc}.jinja2`      | The user facing collection interface (living in the user layer)                                                                                            | `[<package>/]<datatype-name>Collection.h`, `src/<datatype-name>Collection.cc`         |
-| `CollectionData.{h,cc}.jinja2`  | The classes managing the collection storage (not user facing!)                                                                                             | `[<package>/]<datatype-name>CollectionData.h`, `src/<datatype-name>CollectionData.cc` |
-| `datamodel.h.jinja2`            | The *full datamodel header* that includes everything of a generated EDM (via including all generated `Collections`).                                        | `[<package>]/<package>.h`                                                             |
-| `selection.xml.jinja2`          | The `selection.xml` file that is necessary for generating a root dictionary for the generated datamodel                                                    | `src/selection.xml`                                                                   |
-| `SIOBlock.{h,cc}.jinja2`        | The SIO blocks that are necessary for the SIO backend                                                                                                      | `[<package>/]<datatype-name>SIOBlock.h`, `src/<datatype-name>SIOBlock.cc`             |
-| `MutableStruct.jl.jinja2`       | The mutable struct definitions of components and datatypes for julia                                                                                       | `[<package>/]<datatype-name>Struct.jl`, `[<package>/]<component-name>Struct.jl`       |
-| `ParentModule.jl.jinja2`        | The constructor and collection definitions of components and datatypes in the data model are contained within a single module named after the package-name | `[<package>/]<package-name>.jl`                                                       |
+| template file(s)                   | content                                                                                                                                                    | generated file(s)                                                                     |
+|------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|
+| `Component.h.jinja2`               | Definition for each component                                                                                                                              | `[<package>/]<component-name>.h`                                                      |
+| `Data.h.jinja2`                    | POD struct of each datatype (living in the POD layer)                                                                                                      | `[<package>/]<datatype-name>Data.h`                                                   |
+| `Obj.{h,cc}.jinja2`                | `Obj` class for each datatype (living in the object layer) and managing resources                                                                          | `[<package>/]<datatype-name>Obj.h`,  `src/<datatype-name>Obj.cc`                      |
+| `[Mutable]Object.{h,cc}.jinja2`    | The user facing interfaces for each datatype (living in the user layer)                                                                                    | `[<package>/][Mutable]<datatype-name>.h`, `src/[Mutable]<datatype-name>.cc`           |
+| `Collection.{h,cc}.jinja2`         | The user facing collection interface (living in the user layer)                                                                                            | `[<package>/]<datatype-name>Collection.h`, `src/<datatype-name>Collection.cc`         |
+| `CollectionData.{h,cc}.jinja2`     | The classes managing the collection storage (not user facing!)                                                                                             | `[<package>/]<datatype-name>CollectionData.h`, `src/<datatype-name>CollectionData.cc` |
+| `datamodel.h.jinja2`               | The *full datamodel header* that includes everything of a generated EDM (via including all generated `Collections`).                                       | `[<package>]/<package>.h`                                                             |
+| `selection.xml.jinja2`             | The `selection.xml` file that is necessary for generating a root dictionary for the generated datamodel                                                    | `src/selection.xml`                                                                   |
+| `SIOBlock.{h,cc}.jinja2`           | The SIO blocks that are necessary for the SIO backend                                                                                                      | `[<package>/]<datatype-name>SIOBlock.h`, `src/<datatype-name>SIOBlock.cc`             |
+| `LinkCollection.h.jinja2`          | The header that is generated for each *Link* containing effectively typedefs only                                                                          |                                                                                       |
+| `DatamodelLinksSIOBlock.cc.jinja2` | The .cc file that is necessary for enabling SIO based I/O for *Link*s                                                                                      |                                                                                       |
+| `DatamodelLinks.cc.jinja2`         | The global .cc file that is necessary to enable I/O for all *Link*s                                                                                        |                                                                                       |
+| `MutableStruct.jl.jinja2`          | The mutable struct definitions of components and datatypes for julia                                                                                       | `[<package>/]<datatype-name>Struct.jl`, `[<package>/]<component-name>Struct.jl`       |
+| `ParentModule.jl.jinja2`           | The constructor and collection definitions of components and datatypes in the data model are contained within a single module named after the package-name | `[<package>/]<package-name>.jl`                                                       |
 
 
 The presence of a `[<package>]` subdirectory for the header files is controlled by the `includeSubfolder` option in the yaml definition file.

include/podio/LinkCollection.h

@@ -8,30 +8,17 @@
   #define PODIO_ENABLE_SIO 0
 #endif
 
-/// Macro for registering links at the CollectionBufferFactory by injecting the
-/// corresponding buffer creation function.
-#define PODIO_REGISTER_LINK_BUFFERFACTORY(FromT, ToT)                                                                  \
+/// Main macro for declaring links. Takes care of registering the necessary
+/// buffer creation functionality with the CollectionBufferFactory.
+#define PODIO_DECLARE_LINK(FromT, ToT)                                                                                 \
   const static auto PODIO_PP_CONCAT(REGISTERED_LINK_, __COUNTER__) =                                                   \
       podio::detail::registerLinkCollection<FromT, ToT>(podio::detail::linkCollTypeName<FromT, ToT>());
 
-/// Macro for registering the necessary SIOBlock for a Link with the SIOBlock factory
-#define PODIO_REGISTER_LINK_SIOFACTORY(FromT, ToT)                                                                     \
-  const static auto PODIO_PP_CONCAT(LINK_SIO_BLOCK_, __COUNTER__) = podio::LinkSIOBlock<FromT, ToT>{};
-
 #if PODIO_ENABLE_SIO && __has_include("podio/detail/LinkSIOBlock.h")
-  #include "podio/detail/LinkSIOBlock.h"
-  /// Main macro for declaring links. Takes care of the following things:
-  /// - Registering the necessary buffer creation functionality with the
-  ///   CollectionBufferFactory.
-  /// - Registering the necessary SIOBlock with the SIOBlock factory
-  #define PODIO_DECLARE_LINK(FromT, ToT)                                                                               \
-    PODIO_REGISTER_LINK_BUFFERFACTORY(FromT, ToT)                                                                      \
-    PODIO_REGISTER_LINK_SIOFACTORY(FromT, ToT)
-#else
-  /// Main macro for declaring links. Takes care of the following things:
-  /// - Registering the necessary buffer creation functionality with the
-  ///   CollectionBufferFactory.
-  #define PODIO_DECLARE_LINK(FromT, ToT) PODIO_REGISTER_LINK_BUFFERFACTORY(FromT, ToT)
+  #include <podio/detail/LinkSIOBlock.h>
+  /// Macro for registering the necessary SIOBlock for a Link with the SIOBlock factory
+  #define PODIO_DECLARE_LINK_SIO(FromT, ToT)                                                                           \
+    const static auto PODIO_PP_CONCAT(LINK_SIO_BLOCK_, __COUNTER__) = podio::LinkSIOBlock<FromT, ToT>{};
 #endif
 
 #endif // PODIO_LINKCOLLECTION_H

python/podio_gen/cpp_generator.py

@@ -91,14 +91,16 @@ def pre_process(self):
 
         return {}
 
-    def post_process(self, _):
+    def post_process(self, datamodel):
         """Do the cpp specific post processing"""
         self._write_edm_def_file()
 
         if "ROOT" in self.io_handlers:
             self._prepare_iorules()
             self._create_selection_xml()
 
+        if the_links := datamodel["links"]:
+            self._write_links_registration_file(the_links)
         self._write_all_collections_header()
         self._write_cmake_lists_file()
 
@@ -207,6 +209,23 @@ def do_process_interface(self, _, interface):
         self._fill_templates("Interface", interface)
         return interface
 
+    def do_process_link(self, _, link):
+        """Process a link definition and generate the necessary code"""
+        link["include_types"] = []
+        for rel in ("From", "To"):
+            rel_type = link[rel]
+            include_header = f"{rel_type.bare_type}Collection"
+            if self._is_interface(rel_type.full_type):
+                # Interfaces do not have a Collection header
+                include_header = rel_type.bare_type
+            link["include_types"].append(
+                self._build_include_for_class(
+                    include_header, self._needs_include(rel_type.full_type)
+                )
+            )
+        self._fill_templates("LinkCollection", link)
+        return link
+
     def print_report(self):
         """Print a summary report about the generated code"""
         if not self.verbose:
@@ -506,8 +525,10 @@ def _write_list(name, target_folder, files, comment):
 
     def _write_all_collections_header(self):
         """Write a header file that includes all collection headers"""
-
-        collection_files = (x.split("::")[-1] + "Collection.h" for x in self.datamodel.datatypes)
+        collection_files = (
+            x.split("::")[-1] + "Collection.h"
+            for x in list(self.datamodel.datatypes.keys()) + list(self.datamodel.links.keys())
+        )
         self._write_file(
             os.path.join(self.install_dir, self.package_name, f"{self.package_name}.h"),
             self._eval_template(
@@ -520,6 +541,20 @@ def _write_all_collections_header(self):
             ),
         )
 
+    def _write_links_registration_file(self, links):
+        """Write a .cc file that registers all the link collections that were
+        defined with this datamodel"""
+        link_data = {"links": links, "incfolder": self.incfolder}
+        self._write_file(
+            "DatamodelLinks.cc",
+            self._eval_template("DatamodelLinks.cc.jinja2", link_data),
+        )
+        if "SIO" in self.io_handlers:
+            self._write_file(
+                "DatamodelLinkSIOBlock.cc",
+                self._eval_template("DatamodelLinksSIOBlock.cc.jinja2", link_data),
+            )
+
     def _write_edm_def_file(self):
         """Write the edm definition to a compile time string"""
         model_encoder = DataModelJSONEncoder()