diff --git a/README.md b/README.md index b0686edd1..f13420804 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ version and build that automatically. This behavior is controlled via the `USE_EXTERNAL_CATCH2` cmake variable. It defaults to `AUTO` but can also be set to `ON` or `OFF` to fully control the desired behavior. -### Python > 3.6 +### Python >= 3.8 Check your Python version by doing: diff --git a/doc/datamodel_syntax.md b/doc/datamodel_syntax.md index 6386187f7..8f9615c1c 100644 --- a/doc/datamodel_syntax.md +++ b/doc/datamodel_syntax.md @@ -155,6 +155,38 @@ define which `Types` can be used with this interface class, in this case the not allow for mutable access to their data.** They can be used in relations between objects, just like normal `datatypes`. +## Definition of links +Podio offers a templated `Link` class ([see here for more details](links.md)) +that allows one to link two arbitrary datatypes without having to introduce a +`OneToOneRelation` or `OneToManyRelation` inside the corresponding datatypes. In +order to keep the full definition of a datamodel in the YAML file it is possible +to declare `links` in the YAML file: + +```yaml + links: + ExampleLink: + Description: "A link between two (podio generated) objects" + Author: "It could be you" + From: ExampleHit + To: TypeWithEnergy +``` + +This definition will yield the following typedefs +```cpp +using ExampleLinkCollection = podio::LinkCollection; + +using ExampleLink = typename ExampleLinkCollection::value_type; +// this is equivalent to +// using ExampleLink = podio::Link; + +using MutableExampleLink = typename ExampleLinkCollection::mutable_type; +// this is equivalent to +// using MutableExampleLink = podio::MutableLink; +``` + +additionally, this will generate the necessary code to enable I/O for this link +type. + ### Assigning to interface types Interface types support the same functionality as normal (immutable) datatypes. diff --git a/doc/index.rst b/doc/index.rst index 1be94b679..04dd90785 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -15,6 +15,7 @@ Welcome to PODIO's documentation! examples.md frame.md userdata.md + links.md storage_details.md cmake.md advanced_topics.md diff --git a/doc/links.md b/doc/links.md index c7973ca38..d2c9f874a 100644 --- a/doc/links.md +++ b/doc/links.md @@ -45,12 +45,16 @@ For a more detailed explanation of the internals and the actual implementation see [the implementation details](#implementation-details). ## How to use `Link`s -Using `Link`s is quite simple. In line with other datatypes that are -generated by podio all the functionality can be gained by including the -corresponding `Collection` header. After that it is generally recommended to -introduce a type alias for easier usage. **As a general rule `Links` need -to be declared with the default (immutable) types.** Trying to instantiate them -with `Mutable` types will result in a compilation error. +Using `Link`s is quite simple. The most straight forward way is to simply +declare them as part of the datamodel, [as described +here](datamodel_syntax.md#definition-of-links). That will result in code +generation that effectively does what is described below here. However, it's not +strictly necessary to do that in case non-generated code is preferred. In line +with other datatypes that are generated by podio all the functionality can be +gained by including the corresponding `Collection` header. After that it is +generally recommended to introduce a type alias for easier usage. **As a general +rule `Links` need to be declared with the default (immutable) types.** Trying to +instantiate them with `Mutable` types will result in a compilation error. ```cpp #include "podio/LinkCollection.h" diff --git a/doc/templates.md b/doc/templates.md index 79b6cbb63..6335932a9 100644 --- a/doc/templates.md +++ b/doc/templates.md @@ -26,19 +26,22 @@ Note that some of the information below will only apply to either of these gener Currently PODIO loads templates that are placed in [`/python/templates`](/python/templates). They are broadly split along the classes that are generated for each datatype or component from the EDM definition: -| template file(s) | content | generated file(s) | -|---------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------| -| `Component.h.jinja2` | Definition for each component | `[/].h` | -| `Data.h.jinja2` | POD struct of each datatype (living in the POD layer) | `[/]Data.h` | -| `Obj.{h,cc}.jinja2` | `Obj` class for each datatype (living in the object layer) and managing resources | `[/]Obj.h`, `src/Obj.cc` | -| `[Mutable]Object.{h,cc}.jinja2` | The user facing interfaces for each datatype (living in the user layer) | `[/][Mutable].h`, `src/[Mutable].cc` | -| `Collection.{h,cc}.jinja2` | The user facing collection interface (living in the user layer) | `[/]Collection.h`, `src/Collection.cc` | -| `CollectionData.{h,cc}.jinja2` | The classes managing the collection storage (not user facing!) | `[/]CollectionData.h`, `src/CollectionData.cc` | -| `datamodel.h.jinja2` | The *full datamodel header* that includes everything of a generated EDM (via including all generated `Collections`). | `[]/.h` | -| `selection.xml.jinja2` | The `selection.xml` file that is necessary for generating a root dictionary for the generated datamodel | `src/selection.xml` | -| `SIOBlock.{h,cc}.jinja2` | The SIO blocks that are necessary for the SIO backend | `[/]SIOBlock.h`, `src/SIOBlock.cc` | -| `MutableStruct.jl.jinja2` | The mutable struct definitions of components and datatypes for julia | `[/]Struct.jl`, `[/]Struct.jl` | -| `ParentModule.jl.jinja2` | The constructor and collection definitions of components and datatypes in the data model are contained within a single module named after the package-name | `[/].jl` | +| template file(s) | content | generated file(s) | +|------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------| +| `Component.h.jinja2` | Definition for each component | `[/].h` | +| `Data.h.jinja2` | POD struct of each datatype (living in the POD layer) | `[/]Data.h` | +| `Obj.{h,cc}.jinja2` | `Obj` class for each datatype (living in the object layer) and managing resources | `[/]Obj.h`, `src/Obj.cc` | +| `[Mutable]Object.{h,cc}.jinja2` | The user facing interfaces for each datatype (living in the user layer) | `[/][Mutable].h`, `src/[Mutable].cc` | +| `Collection.{h,cc}.jinja2` | The user facing collection interface (living in the user layer) | `[/]Collection.h`, `src/Collection.cc` | +| `CollectionData.{h,cc}.jinja2` | The classes managing the collection storage (not user facing!) | `[/]CollectionData.h`, `src/CollectionData.cc` | +| `datamodel.h.jinja2` | The *full datamodel header* that includes everything of a generated EDM (via including all generated `Collections`). | `[]/.h` | +| `selection.xml.jinja2` | The `selection.xml` file that is necessary for generating a root dictionary for the generated datamodel | `src/selection.xml` | +| `SIOBlock.{h,cc}.jinja2` | The SIO blocks that are necessary for the SIO backend | `[/]SIOBlock.h`, `src/SIOBlock.cc` | +| `LinkCollection.h.jinja2` | The header that is generated for each *Link* containing effectively typedefs only | | +| `DatamodelLinksSIOBlock.cc.jinja2` | The .cc file that is necessary for enabling SIO based I/O for *Link*s | | +| `DatamodelLinks.cc.jinja2` | The global .cc file that is necessary to enable I/O for all *Link*s | | +| `MutableStruct.jl.jinja2` | The mutable struct definitions of components and datatypes for julia | `[/]Struct.jl`, `[/]Struct.jl` | +| `ParentModule.jl.jinja2` | The constructor and collection definitions of components and datatypes in the data model are contained within a single module named after the package-name | `[/].jl` | The presence of a `[]` subdirectory for the header files is controlled by the `includeSubfolder` option in the yaml definition file. diff --git a/include/podio/LinkCollection.h b/include/podio/LinkCollection.h index ae2337d84..e06f4227b 100644 --- a/include/podio/LinkCollection.h +++ b/include/podio/LinkCollection.h @@ -8,30 +8,17 @@ #define PODIO_ENABLE_SIO 0 #endif -/// Macro for registering links at the CollectionBufferFactory by injecting the -/// corresponding buffer creation function. -#define PODIO_REGISTER_LINK_BUFFERFACTORY(FromT, ToT) \ +/// Main macro for declaring links. Takes care of registering the necessary +/// buffer creation functionality with the CollectionBufferFactory. +#define PODIO_DECLARE_LINK(FromT, ToT) \ const static auto PODIO_PP_CONCAT(REGISTERED_LINK_, __COUNTER__) = \ podio::detail::registerLinkCollection(podio::detail::linkCollTypeName()); -/// Macro for registering the necessary SIOBlock for a Link with the SIOBlock factory -#define PODIO_REGISTER_LINK_SIOFACTORY(FromT, ToT) \ - const static auto PODIO_PP_CONCAT(LINK_SIO_BLOCK_, __COUNTER__) = podio::LinkSIOBlock{}; - #if PODIO_ENABLE_SIO && __has_include("podio/detail/LinkSIOBlock.h") - #include "podio/detail/LinkSIOBlock.h" - /// Main macro for declaring links. Takes care of the following things: - /// - Registering the necessary buffer creation functionality with the - /// CollectionBufferFactory. - /// - Registering the necessary SIOBlock with the SIOBlock factory - #define PODIO_DECLARE_LINK(FromT, ToT) \ - PODIO_REGISTER_LINK_BUFFERFACTORY(FromT, ToT) \ - PODIO_REGISTER_LINK_SIOFACTORY(FromT, ToT) -#else - /// Main macro for declaring links. Takes care of the following things: - /// - Registering the necessary buffer creation functionality with the - /// CollectionBufferFactory. - #define PODIO_DECLARE_LINK(FromT, ToT) PODIO_REGISTER_LINK_BUFFERFACTORY(FromT, ToT) + #include + /// Macro for registering the necessary SIOBlock for a Link with the SIOBlock factory + #define PODIO_DECLARE_LINK_SIO(FromT, ToT) \ + const static auto PODIO_PP_CONCAT(LINK_SIO_BLOCK_, __COUNTER__) = podio::LinkSIOBlock{}; #endif #endif // PODIO_LINKCOLLECTION_H diff --git a/python/podio_gen/cpp_generator.py b/python/podio_gen/cpp_generator.py index b565f2a80..980e92383 100644 --- a/python/podio_gen/cpp_generator.py +++ b/python/podio_gen/cpp_generator.py @@ -91,7 +91,7 @@ def pre_process(self): return {} - def post_process(self, _): + def post_process(self, datamodel): """Do the cpp specific post processing""" self._write_edm_def_file() @@ -99,6 +99,8 @@ def post_process(self, _): self._prepare_iorules() self._create_selection_xml() + if the_links := datamodel["links"]: + self._write_links_registration_file(the_links) self._write_all_collections_header() self._write_cmake_lists_file() @@ -207,6 +209,23 @@ def do_process_interface(self, _, interface): self._fill_templates("Interface", interface) return interface + def do_process_link(self, _, link): + """Process a link definition and generate the necessary code""" + link["include_types"] = [] + for rel in ("From", "To"): + rel_type = link[rel] + include_header = f"{rel_type.bare_type}Collection" + if self._is_interface(rel_type.full_type): + # Interfaces do not have a Collection header + include_header = rel_type.bare_type + link["include_types"].append( + self._build_include_for_class( + include_header, self._needs_include(rel_type.full_type) + ) + ) + self._fill_templates("LinkCollection", link) + return link + def print_report(self): """Print a summary report about the generated code""" if not self.verbose: @@ -506,8 +525,10 @@ def _write_list(name, target_folder, files, comment): def _write_all_collections_header(self): """Write a header file that includes all collection headers""" - - collection_files = (x.split("::")[-1] + "Collection.h" for x in self.datamodel.datatypes) + collection_files = ( + x.split("::")[-1] + "Collection.h" + for x in list(self.datamodel.datatypes.keys()) + list(self.datamodel.links.keys()) + ) self._write_file( os.path.join(self.install_dir, self.package_name, f"{self.package_name}.h"), self._eval_template( @@ -520,6 +541,20 @@ def _write_all_collections_header(self): ), ) + def _write_links_registration_file(self, links): + """Write a .cc file that registers all the link collections that were + defined with this datamodel""" + link_data = {"links": links, "incfolder": self.incfolder} + self._write_file( + "DatamodelLinks.cc", + self._eval_template("DatamodelLinks.cc.jinja2", link_data), + ) + if "SIO" in self.io_handlers: + self._write_file( + "DatamodelLinkSIOBlock.cc", + self._eval_template("DatamodelLinksSIOBlock.cc.jinja2", link_data), + ) + def _write_edm_def_file(self): """Write the edm definition to a compile time string""" model_encoder = DataModelJSONEncoder() diff --git a/python/podio_gen/generator_base.py b/python/podio_gen/generator_base.py index 111467f6d..e23b0fbae 100644 --- a/python/podio_gen/generator_base.py +++ b/python/podio_gen/generator_base.py @@ -96,15 +96,21 @@ class ClassGeneratorBaseMixin: list. This function also has to take care of filling the necessary templates! + do_process_link(name: str, link: dict): do some language specific processing + for a link type, populating the link dictionary further. + When called only the "class" key will be populated. Return a + dictionary or None. If None, this will not be put into the + "links" list. This function also has to take care of filling + the necessary templates! + post_process(datamodel: dict): do some global post processing for which all components and datatypes need to have been processed already. Gets called with the dictionary that has been created in pre_process and filled during the processing. The process components and datatypes are accessible via the "components", - "datatypes" and "interfaces" keys respectively. + "datatypes", "interfaces" and "links" keys respectively. print_report(): prints a report summarizing what has been generated - """ def __init__( @@ -154,6 +160,7 @@ def process(self): datamodel["components"] = [] datamodel["datatypes"] = [] datamodel["interfaces"] = [] + datamodel["links"] = [] for name, component in self.datamodel.components.items(): comp = self._process_component(name, component) @@ -170,6 +177,11 @@ def process(self): if interf is not None: datamodel["interfaces"].append(interf) + for name, link in self.datamodel.links.items(): + proc_link = self._process_link(name, link) + if proc_link is not None: + datamodel["links"].append(proc_link) + self.post_process(datamodel) if self.verbose: self.print_report() @@ -200,6 +212,13 @@ def _process_interface(self, name, interface): return self.do_process_interface(name, interface) + def _process_link(self, name, link): + """Process a single link definition into a dictionary that can be used + in jinja2 templates and return that""" + link = deepcopy(link) + link["class"] = DataType(name) + return self.do_process_link(name, link) + @staticmethod def _get_filenames_templates(template_base, name): """Get the list of output filenames and corresponding template names""" @@ -217,6 +236,7 @@ def get_fn_format(tmpl): "Collection": "Collection", "CollectionData": "CollectionData", "MutableStruct": "Struct", + "LinkCollection": "Collection", } return f'{prefix.get(tmpl, "")}{{name}}{postfix.get(tmpl, "")}.{{end}}' @@ -227,6 +247,8 @@ def get_fn_format(tmpl): "Interface": ("h",), "MutableStruct": ("jl",), "ParentModule": ("jl",), + "LinkCollection": ("h",), + "DatamodelLinks": ("cc",), }.get(template_base, ("h", "cc")) fn_templates = [] diff --git a/python/podio_gen/generator_utils.py b/python/podio_gen/generator_utils.py index b3dd2ceb0..ea37acf9f 100644 --- a/python/podio_gen/generator_utils.py +++ b/python/podio_gen/generator_utils.py @@ -302,6 +302,7 @@ def __init__( datatypes=None, components=None, interfaces=None, + links=None, options=None, schema_version=None, ): @@ -318,6 +319,7 @@ def __init__( self.components = components or {} self.datatypes = datatypes or {} self.interfaces = interfaces or {} + self.links = links or {} def _to_json(self): """Return the dictionary, so that we can easily hook this into the pythons diff --git a/python/podio_gen/julia_generator.py b/python/podio_gen/julia_generator.py index 12b966188..689a5623c 100644 --- a/python/podio_gen/julia_generator.py +++ b/python/podio_gen/julia_generator.py @@ -74,6 +74,10 @@ def do_process_interface(self, _, __): """Julia does not support interface types yet, so this does nothing""" return None + def do_process_link(self, _, __): + """Julia does not support link types yet, so this does nothing""" + return None + def get_upstream_name(self): """Get the name of the upstream datamodel if any""" if self.upstream_edm: diff --git a/python/podio_gen/podio_config_reader.py b/python/podio_gen/podio_config_reader.py index 247ec3214..b01b1ea37 100644 --- a/python/podio_gen/podio_config_reader.py +++ b/python/podio_gen/podio_config_reader.py @@ -172,6 +172,8 @@ class ClassDefinitionValidator: # it applies and also which accessor functions to generate required_interface_keys = required_datatype_keys + ("Members", "Types") + required_link_keys = required_datatype_keys + ("From", "To") + valid_extra_code_keys = ( "declaration", "implementation", @@ -187,6 +189,7 @@ def validate(cls, datamodel, upstream_edm=None): expose_pod_members = datamodel.options["exposePODMembers"] cls._check_datatypes(datamodel, expose_pod_members, upstream_edm) cls._check_interfaces(datamodel, upstream_edm) + cls._check_links(datamodel, upstream_edm) @classmethod def _check_comp(cls, member, components, upstream_edm): @@ -256,7 +259,7 @@ def _check_interfaces(cls, datamodel, upstream_edm): f"'{name}' defines an interface type with the same name " "as an existing datatype" ) - cls._check_interface_fields(name, definition) + cls._check_required_fields(name, definition, cls.required_interface_keys, "interface") cls._check_interface_types(name, definition, all_types) @classmethod @@ -271,6 +274,23 @@ def _check_datatype(cls, classname, definition, expose_pod_members, datamodel, u ) cls._check_relations(classname, definition, datamodel, upstream_edm) + @classmethod + def _check_links(cls, datamodel, upstream_edm): + """Check the link definitions""" + all_types = list(datamodel.datatypes.keys()) + list(datamodel.interfaces.keys()) + ext_types = [] + if upstream_edm: + ext_types = list(upstream_edm.datatypes.keys()) + list(upstream_edm.interfaces.keys()) + all_types.extend(ext_types) + + for name, definition in datamodel.links.items(): + if name in all_types: + raise DefinitionError( + f"'{name}' defines a link type with the same name as an existing datatype" + ) + cls._check_required_fields(name, definition, cls.required_link_keys, "link") + cls._check_link_types(name, definition, all_types) + @classmethod def _check_members(cls, classname, members, expose_pod_members, datamodel, upstream_edm): """Check the members of a class for name clashes or undefined classes.""" @@ -377,17 +397,17 @@ def _check_keys(cls, classname, definition): ) @classmethod - def _check_interface_fields(cls, name, definition): - """Check whether the fields of an interface definition follow the required schema""" - for key in cls.required_interface_keys: + def _check_required_fields(cls, name, definition, required_keys, cat_name): + """Check whether the keys in definition are **exactly** the required ones""" + for key in required_keys: if key not in definition: raise DefinitionError( - f"interface '{name}' does not define '{key}' field which is required" + f"{cat_name} '{name}' does not define '{key}' field which is required" ) - invalid_keys = [k for k in definition.keys() if k not in cls.required_interface_keys] + invalid_keys = [k for k in definition.keys() if k not in required_keys] if invalid_keys: - raise DefinitionError(f"interface '{name}' defines invalid fields: {invalid_keys}") + raise DefinitionError(f"{cat_name} '{name}' defines invalid fields: {invalid_keys}") @classmethod def _check_interface_types(cls, name, definition, known_datatypes): @@ -398,6 +418,16 @@ def _check_interface_types(cls, name, definition, known_datatypes): f"interface '{name}' tries to use Type '{wrapped_type}' which is undefined" ) + @classmethod + def _check_link_types(cls, name, definition, known_datatypes): + """Check whether a link really only uses known datatypes""" + for link_dir in ("From", "To"): + if definition[link_dir].full_type not in known_datatypes: + raise DefinitionError( + f"link '{name}' tries to use undefined Type '{definition[link_dir]}' " + f"as its '{link_dir}' relation" + ) + @classmethod def _fill_defaults(cls, definition): """Fill some of the fields with empty defaults in order to make it easier to @@ -512,6 +542,18 @@ def _read_interface(cls, value): return interface + @classmethod + def _read_link(cls, value): + """Read a link definition and put it into a more easily digestible format""" + link = {} + for category, definition in value.items(): + if category in ("From", "To"): + link[category] = DataType(definition) + else: + link[category] = copy.deepcopy(definition) + + return link + @classmethod def parse_model(cls, model_dict, package_name, upstream_edm=None, parent_path=None): """Parse a model from the dictionary, e.g. read from a yaml file.""" @@ -546,6 +588,11 @@ def parse_model(cls, model_dict, package_name, upstream_edm=None, parent_path=No for klassname, value in model_dict["interfaces"].items(): interfaces[klassname] = cls._read_interface(value) + links = {} + if "links" in model_dict: + for klassname, value in model_dict["links"].items(): + links[klassname] = cls._read_link(value) + options = copy.deepcopy(cls.options) if "options" in model_dict: for option, value in model_dict["options"].items(): @@ -559,7 +606,7 @@ def parse_model(cls, model_dict, package_name, upstream_edm=None, parent_path=No # If this doesn't raise an exception everything should in principle work out validator = ClassDefinitionValidator() - datamodel = DataModel(datatypes, components, interfaces, options, schema_version) + datamodel = DataModel(datatypes, components, interfaces, links, options, schema_version) validator.validate(datamodel, upstream_edm) return datamodel diff --git a/python/podio_gen/test_ClassDefinitionValidator.py b/python/podio_gen/test_ClassDefinitionValidator.py index 8e7b2ba05..268c3c05b 100644 --- a/python/podio_gen/test_ClassDefinitionValidator.py +++ b/python/podio_gen/test_ClassDefinitionValidator.py @@ -16,10 +16,10 @@ from podio_gen.generator_utils import DataModel, DataType -def make_dm(components, datatypes, interfaces=None, options=None): +def make_dm(components, datatypes, interfaces=None, links=None, options=None): """Small helper function to turn things into a datamodel dict as expected by the validator""" - return DataModel(datatypes, components, interfaces, options) + return DataModel(datatypes, components, interfaces, links, options) class ClassDefinitionValidatorTest(unittest.TestCase): # pylint: disable=too-many-public-methods @@ -79,6 +79,15 @@ def setUp(self): } } + self.valid_link = { + "LinkType": { + "Author": "Princess Zelda", + "Description": "Because we know our lore", + "From": DataType("DataType"), + "To": DataType("DataType"), + } + } + # The default options that should be used for validation self.def_opts = {"exposePODMembers": False} @@ -465,7 +474,7 @@ def test_datatype_valid_upstream(self): DefinitionError, "{} should allow to use to use upstream datatypes and components", self.validate, - make_dm({}, datatype, {}, self.def_opts), + make_dm({}, datatype, {}, options=self.def_opts), upstream_dm, ) @@ -528,7 +537,7 @@ def test_interface_invalid_fields(self): interface = deepcopy(self.valid_interface) interface["InterfaceType"][inv_field] = ["An invalid field"] with self.assertRaises(DefinitionError): - self.validate(make_dm({}, self.valid_datatype, interface), False) + self.validate(make_dm({}, self.valid_datatype, interface)) def test_interface_missing_fields(self): """Make sure that interfaces have all the required types when they pass validation""" @@ -599,6 +608,91 @@ def test_interface_valid_upstream(self): upstream_dm, ) + def test_link_valid_def(self): + """Make sure that a valid link definition inside a valid datamodel + passes""" + self._assert_no_exception( + DefinitionError, + "{} should not raise for a valid link type", + self.validate, + make_dm({}, self.valid_datatype, links=self.valid_link), + ) + + links = deepcopy(self.valid_link) + links["LinkType"]["From"] = DataType("InterfaceType") + self._assert_no_exception( + DefinitionError, + "{} should not raise for a valid link type", + self.validate, + make_dm({}, self.valid_datatype, self.valid_interface, links), + ) + + def test_link_invalid_fields(self): + """Make sure that link definitions cannot contain any superfluous fields""" + for inv_field in ("Members", "OneToOneRelations", "OneToManyRelations", "VectorMembers"): + link = deepcopy(self.valid_link) + link["LinkType"][inv_field] = ["A value that does not matter"] + with self.assertRaises(DefinitionError): + self.validate(make_dm({}, self.valid_datatype, links=link)) + + def test_link_missing_fields(self): + """Make sure that links need to have all the required fields""" + for req in ClassDefinitionValidator.required_link_keys: + link_type = deepcopy(self.valid_link) + del link_type["LinkType"][req] + with self.assertRaises(DefinitionError): + self.validate(make_dm({}, self.valid_datatype, links=link_type)) + + def test_link_only_defined_datatypes(self): + """Make sure links can only use defined datatypes""" + link_type = deepcopy(self.valid_link) + link_type["LinkType"]["From"] = DataType("NonExistantType") + with self.assertRaises(DefinitionError): + self.validate(make_dm({}, self.valid_datatype, links=link_type)) + + link_type = deepcopy(self.valid_link) + link_type["LinkType"]["To"] = DataType("NonExistantType") + with self.assertRaises(DefinitionError): + self.validate(make_dm({}, self.valid_datatype, links=link_type)) + + def test_link_no_redefining_datatypes(self): + """Make sure that a link cannot shadow / redeclare an existing datatype""" + link_type = { + "DataType": { + "Author": "T.B. Lee", + "Description": "Redefining datatypes is bad", + "From": DataType("DataType"), + "To": DataType("DataType"), + } + } + with self.assertRaises(DefinitionError): + self.validate(make_dm({}, self.valid_datatype, links=link_type)) + + # We can also not redefine interface types + link_type["InterfaceType"] = link_type.pop("DataType") + with self.assertRaises(DefinitionError): + self.validate(make_dm({}, self.valid_datatype, self.valid_interface, link_type)) + + def test_link_valid_upstream(self): + """Check that links can use upstream datatypes""" + upstream_dm = make_dm({}, self.valid_datatype, self.valid_interface) + + link_type = { + "Century": { + "Author": "Link", + "Description": "Linking to other datamodels is possible!", + "From": DataType("DataType"), + "To": DataType("InterfaceType"), + } + } + self._assert_no_exception( + DefinitionError, + "{} links should be able to use upstream datatypes", + self.validate, + make_dm({}, {}, links=link_type), + upstream_dm, + ) + if __name__ == "__main__": unittest.main() diff --git a/python/templates/CMakeLists.txt b/python/templates/CMakeLists.txt index 014b4e0bd..e1584a4fe 100644 --- a/python/templates/CMakeLists.txt +++ b/python/templates/CMakeLists.txt @@ -17,6 +17,8 @@ set(PODIO_TEMPLATES ${CMAKE_CURRENT_LIST_DIR}/selection.xml.jinja2 ${CMAKE_CURRENT_LIST_DIR}/SIOBlock.cc.jinja2 ${CMAKE_CURRENT_LIST_DIR}/SIOBlock.h.jinja2 + ${CMAKE_CURRENT_LIST_DIR}/LinkCollection.h.jinja2 + ${CMAKE_CURRENT_LIST_DIR}/DatamodelLinks.cc.jinja2 ${CMAKE_CURRENT_LIST_DIR}/DatamodelDefinition.h.jinja2 ${CMAKE_CURRENT_LIST_DIR}/macros/collections.jinja2 ${CMAKE_CURRENT_LIST_DIR}/macros/declarations.jinja2 diff --git a/python/templates/DatamodelLinks.cc.jinja2 b/python/templates/DatamodelLinks.cc.jinja2 new file mode 100644 index 000000000..b2f69fe00 --- /dev/null +++ b/python/templates/DatamodelLinks.cc.jinja2 @@ -0,0 +1,10 @@ +// AUTOMATICALLY GENERATED FILE - DO NOT EDIT + +#include "podio/LinkCollection.h" +{% for link in links %} +#include "{{ incfolder }}{{ link['class'].bare_type }}Collection.h" +{% endfor %} + +{% for link in links %} +PODIO_DECLARE_LINK({{ link["From"].full_type }}, {{ link["To"].full_type }}) +{% endfor %} diff --git a/python/templates/DatamodelLinksSIOBlock.cc.jinja2 b/python/templates/DatamodelLinksSIOBlock.cc.jinja2 new file mode 100644 index 000000000..b49ebe923 --- /dev/null +++ b/python/templates/DatamodelLinksSIOBlock.cc.jinja2 @@ -0,0 +1,10 @@ +// AUTOMATICALLY GENERATED FILE - DO NOT EDIT + +#include "podio/LinkCollection.h" +{% for link in links %} +#include "{{ incfolder }}{{ link['class'].bare_type }}Collection.h" +{% endfor %} + +{% for link in links %} +PODIO_DECLARE_LINK_SIO({{ link["From"].full_type }}, {{ link["To"].full_type }}) +{% endfor %} diff --git a/python/templates/LinkCollection.h.jinja2 b/python/templates/LinkCollection.h.jinja2 new file mode 100644 index 000000000..7ebc9dea7 --- /dev/null +++ b/python/templates/LinkCollection.h.jinja2 @@ -0,0 +1,20 @@ +{% import "macros/utils.jinja2" as utils %} +// AUTOMATICALLY GENERATED FILE - DO NOT EDIT + +#ifndef {{ package_name.upper() }}_{{ class.bare_type }}Collection_H +#define {{ package_name.upper() }}_{{ class.bare_type }}Collection_H + +{% for include in include_types %} +{{ include }} +{% endfor %} + +#include + +{{ utils.namespace_open(class.namespace) }} +using {{ class.bare_type }}Collection = podio::LinkCollection<{{ From }}, {{ To }}>; +using {{ class.bare_type }} = typename {{ class.bare_type }}Collection::value_type; +using Mutable{{ class.bare_type }} = typename {{ class.bare_type }}Collection::mutable_type; + +{{ utils.namespace_close(class.namespace) }} + +#endif diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 65608cca7..1c0345447 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -19,7 +19,6 @@ PODIO_GENERATE_DATAMODEL(datamodel datalayout.yaml headers sources # Use the cmake building blocks to add the different parts (conditionally) PODIO_ADD_DATAMODEL_CORE_LIB(TestDataModel "${headers}" "${sources}") -target_sources(TestDataModel PRIVATE DatamodelLinks.cc) find_package(nlohmann_json 3.10) if (nlohmann_json_FOUND) message(STATUS "Found compatible version of JSON library, will add JSON support to test datamodel") @@ -29,9 +28,6 @@ endif() PODIO_ADD_ROOT_IO_DICT(TestDataModelDict TestDataModel "${headers}" src/selection.xml) PODIO_ADD_SIO_IO_BLOCKS(TestDataModel "${headers}" "${sources}") -if (TARGET TestDataModelSioBlocks) - target_sources(TestDataModelSioBlocks PRIVATE DatamodelLinks.cc) -endif() # Build the extension data model and link it against the upstream model PODIO_GENERATE_DATAMODEL(extension_model datalayout_extension.yaml ext_headers ext_sources diff --git a/tests/DatamodelLinks.cc b/tests/DatamodelLinks.cc deleted file mode 100644 index 8a1bb8a07..000000000 --- a/tests/DatamodelLinks.cc +++ /dev/null @@ -1,8 +0,0 @@ -#include "podio/LinkCollection.h" - -#include "datamodel/ExampleClusterCollection.h" -#include "datamodel/ExampleHitCollection.h" -#include "datamodel/TypeWithEnergy.h" - -PODIO_DECLARE_LINK(ExampleHit, ExampleCluster) -PODIO_DECLARE_LINK(ExampleCluster, TypeWithEnergy) diff --git a/tests/datalayout.yaml b/tests/datalayout.yaml index 06cd5ebee..80a98364a 100644 --- a/tests/datalayout.yaml +++ b/tests/datalayout.yaml @@ -272,3 +272,16 @@ interfaces: Members: - double energy // the energy - int PDG // a pdg + +links: + TestLink: + Description: "A link for testing" + Author: "Thomas Madlener" + From: ExampleHit + To: ExampleCluster + + TestInterfaceLink: + Description: "A link with an interface type for testing" + Author: "Thomas Madlener" + From: ExampleCluster + To: TypeWithEnergy diff --git a/tests/datamodel/LinkCollections.h b/tests/datamodel/LinkCollections.h deleted file mode 100644 index 53f32f078..000000000 --- a/tests/datamodel/LinkCollections.h +++ /dev/null @@ -1,15 +0,0 @@ -#ifndef PODIO_TESTS_LINKCOLLECTIONS_H -#define PODIO_TESTS_LINKCOLLECTIONS_H - -#include "podio/LinkCollection.h" - -#include "datamodel/ExampleClusterCollection.h" -#include "datamodel/ExampleHitCollection.h" -#include "datamodel/TypeWithEnergy.h" - -// Define an link that is used for the I/O tests -using TestLinkCollection = podio::LinkCollection; -// Define a link with an interface type that is used for I/O tests -using TestInterfaceLinkCollection = podio::LinkCollection; - -#endif // PODIO_TESTS_LINKCOLLECTIONS_H diff --git a/tests/read_python_frame.h b/tests/read_python_frame.h index 6d9d69840..4c0716002 100644 --- a/tests/read_python_frame.h +++ b/tests/read_python_frame.h @@ -3,7 +3,8 @@ #include "datamodel/ExampleClusterCollection.h" #include "datamodel/ExampleHitCollection.h" -#include "datamodel/LinkCollections.h" +#include "datamodel/TestInterfaceLinkCollection.h" +#include "datamodel/TestLinkCollection.h" #include "datamodel/TypeWithEnergy.h" #include "podio/Frame.h" diff --git a/tests/read_test.h b/tests/read_test.h index b27b3a353..0f0b5d349 100644 --- a/tests/read_test.h +++ b/tests/read_test.h @@ -12,7 +12,8 @@ #include "datamodel/ExampleWithNamespace.h" #include "datamodel/ExampleWithOneRelationCollection.h" #include "datamodel/ExampleWithVectorMemberCollection.h" -#include "datamodel/LinkCollections.h" +#include "datamodel/TestInterfaceLinkCollection.h" +#include "datamodel/TestLinkCollection.h" #include "datamodel/TypeWithEnergy.h" // podio specific includes diff --git a/tests/scripts/dumpModelRoundTrip.sh b/tests/scripts/dumpModelRoundTrip.sh index 00895c33c..4adfb4fa2 100755 --- a/tests/scripts/dumpModelRoundTrip.sh +++ b/tests/scripts/dumpModelRoundTrip.sh @@ -34,7 +34,7 @@ ${PODIO_BASE}/python/podio_class_generator.py \ # Compare to the originally generated code, that has been used to write the data # file. Need to diff subfolders explicitly here because $PODIO_BASE/tests contains # more stuff -DIFF_EXTRA_ARGS="--exclude=LinkCollections.h" +DIFF_EXTRA_ARGS="" if [ ${ENABLE_SIO} = "OFF" ]; then DIFF_EXTRA_ARGS="${DIFF_EXTRA_ARGS} --exclude='*SIO*'" fi diff --git a/tests/unittests/links.cpp b/tests/unittests/links.cpp index a296d06f6..aa8a9ef38 100644 --- a/tests/unittests/links.cpp +++ b/tests/unittests/links.cpp @@ -4,7 +4,7 @@ #include "datamodel/ExampleClusterCollection.h" #include "datamodel/ExampleHitCollection.h" -#include "datamodel/LinkCollections.h" +#include "datamodel/TestInterfaceLinkCollection.h" #include "datamodel/TypeWithEnergy.h" #ifdef PODIO_JSON_OUTPUT diff --git a/tests/write_frame.h b/tests/write_frame.h index 54a4b67d5..087f4ae50 100644 --- a/tests/write_frame.h +++ b/tests/write_frame.h @@ -15,7 +15,8 @@ #include "datamodel/ExampleWithNamespaceCollection.h" #include "datamodel/ExampleWithOneRelationCollection.h" #include "datamodel/ExampleWithVectorMemberCollection.h" -#include "datamodel/LinkCollections.h" +#include "datamodel/TestInterfaceLinkCollection.h" +#include "datamodel/TestLinkCollection.h" #include "datamodel/TypeWithEnergy.h" #include "extension_model/ContainedTypeCollection.h" diff --git a/tests/write_frame.py b/tests/write_frame.py index bd3068a13..c814fbd42 100644 --- a/tests/write_frame.py +++ b/tests/write_frame.py @@ -9,9 +9,6 @@ if ROOT.gSystem.Load("libTestDataModelDict") < 0: # noqa: E402 raise RuntimeError("Could not load TestDataModel dictionary") -if ROOT.gInterpreter.LoadFile("datamodel/LinkCollections.h"): - raise RuntimeError("Could not load LinkCollections.h header") - from ROOT import ( # pylint: disable=wrong-import-position ExampleHitCollection, ExampleClusterCollection,