From e62dc5ae52a8f4a835e96fad9a4b1976181b19e1 Mon Sep 17 00:00:00 2001 From: Rolf Krahl Date: Tue, 16 Jan 2024 12:36:27 +0100 Subject: [PATCH] - Review documentation Section "ICAT data XML files", adding more inline examples - Drop icatdump-simple-2.xml example, rename icatdump-simple-1.xml to icatdump-simple.xml --- doc/examples/icatdump-simple-2.xml | 108 ------------------ ...tdump-simple-1.xml => icatdump-simple.xml} | 0 doc/src/file-icatdata.rst | 108 +++++++++++++----- 3 files changed, 80 insertions(+), 136 deletions(-) delete mode 100644 doc/examples/icatdump-simple-2.xml rename doc/examples/{icatdump-simple-1.xml => icatdump-simple.xml} (100%) diff --git a/doc/examples/icatdump-simple-2.xml b/doc/examples/icatdump-simple-2.xml deleted file mode 100644 index 1c30960..0000000 --- a/doc/examples/icatdump-simple-2.xml +++ /dev/null @@ -1,108 +0,0 @@ - - - - 2024-01-03T13:27:37+00:00 - https://icat.example.com:8181/ICATService/ICAT?wsdl - 6.0.0 - icatdump (python-icat 1.2.0) - - - - Goethe University Frankfurt, Faculty of Philosophy and History - ahau@example.org - Hau - Arnold Hau - Arnold - db/ahau - 0000-0002-3263 - - - Goethe University Frankfurt, Faculty of Philosophy and History - ahau@example.org - Hau - Arnold Hau - Arnold - db/ahau - 0000-0002-3263 - - - Université Paul-Valéry Montpellier 3 - jbotu@example.org - Botul - Jean-Baptiste Botul - Jean-Baptiste - db/jbotu - 0000-0002-3264 - - - jdoe@example.org - Doe - John Doe - John - db/jdoe - - - University of Nancago - nbour@example.org - Bourbaki - Nicolas Bourbaki - Nicolas - db/nbour - 0000-0002-3266 - - - investigation_10100601-ST_owner - - - investigation_10100601-ST_reader - - - investigation_10100601-ST_writer - - - - - - - - - - - - - - - - - - - - - - - - - DOI:00.0815/inv-00601 - 2010-10-12T15:00:00+00:00 - 4 - 127125 - 10100601-ST - 2010-09-30T10:27:24+00:00 - Ni-Mn-Ga flat cone - 1.1-N - - - owner - - - - reader - - - - writer - - - - - diff --git a/doc/examples/icatdump-simple-1.xml b/doc/examples/icatdump-simple.xml similarity index 100% rename from doc/examples/icatdump-simple-1.xml rename to doc/examples/icatdump-simple.xml diff --git a/doc/src/file-icatdata.rst b/doc/src/file-icatdata.rst index b856e62..fa82f96 100644 --- a/doc/src/file-icatdata.rst +++ b/doc/src/file-icatdata.rst @@ -72,7 +72,7 @@ ICAT data XML files In this section we describe the ICAT data file format using the XML backend. Consider the following example: -.. literalinclude:: ../examples/icatdump-simple-1.xml +.. literalinclude:: ../examples/icatdump-simple.xml :language: xml The root element of ICAT data XML files is ``icatdata``. It may @@ -88,7 +88,8 @@ logical structure explained above. The present example contains two chunks. Each element within the ``data`` element corresponds to an ICAT object according to the ICAT schema. In the present example, the first chunk contains five User objects and three Grouping objects. -The second chunk only contains one Investigation. +The Groupings include related UserGroups. The second chunk only +contains one Investigation, including related investigationGroups. These object elements should have an ``id`` attribute that may be used to reference the object in relations later on. The ``id`` value has @@ -104,27 +105,87 @@ the related object's attribute values, using XML attributes of the same name. In the latter case, the attribute values must uniquely define the related object. +In the present example, consider the first grouping: + +.. code-block:: XML + + + investigation_10100601-ST_owner + + + + + +It includes a related userGroup object that in turn references a +related User. This User is referenced in the ``ref`` attribute using +a key defined in the User's ``id`` attribute earlier in the file. +Another example is how the Investigation references its Facility: + +.. code-block:: XML + + + + + + + +The Facility is not defined in the data file. It is assumed to exist +in ICAT before ingesting the file. In this case, it must be +referenced by the unique key that could have been obtained by calling +``facility.getUniqueKey()``. Alternatively, the Facility could have +been referenced by attribute as in: + +.. code-block:: XML + + + + + + + + The object elements may include one-to-many relations. In this case, the related objects will be created along with the parent in one -single cascading call. Alternatively, these related objects may be -added separately as subelements of the ``data`` element later in the -file. In the present example, the Grouping object include their -related UserGroup objects. Note that these UserGroups include their -relation to the User. The User object is referenced by their -respective id in the ``ref`` attribute. But the UserGroups do not -include their relation with Grouping. That relationship is implied by -the parent relation of the object in the file. - -In a similar way, the Investigation in the second chunk includes +single cascading call. In the present example, the Grouping objects +include their related UserGroup objects. Note that these UserGroups +include their relation to the User, but not their relation with +Grouping. The latter relationship is implied by the parent relation +of the object in the file. + +As an alternative, the Usergroups could have been added to the file as +separate objects as direct subelements of ``data`` as in: + +.. code-block:: XML + + + + Goethe University Frankfurt, Faculty of Philosophy and History + ahau@example.org + Hau + Arnold Hau + Arnold + db/ahau + 0000-0002-3263 + + + investigation_10100601-ST_owner + + + + + + + +The Investigation in the second chunk in the present example includes related InvestigationGroups that will be created along with the Investigation. The InvestigationGroup objects include a reference to the corresponding Grouping. Note that these references go across chunk boundaries. The index that caches the object ids to resolve object relations from the first chunk that did contain the ids of the -Groupings will already have been discarded from memeory when the -second chunk is read. But the references use the key that can be -passed to :meth:`icat.client.Client.searchUniqueKey` to search these -Groupings from ICAT. +Groupings will already have been discarded from memory when the second +chunk is read. But the references use the key that can be passed to +:meth:`icat.client.Client.searchUniqueKey` to search these Groupings +from ICAT. Finally note the the file format also depends on the ICAT schema version: the present example can only be ingested into ICAT server 5.0 @@ -132,21 +193,12 @@ or newer, because the attributes fileCount and fileSize have been added to Investigation in this version. With older ICAT versions, it will fail because the attributes are not defined. -Consider a second example, it defines a subset of the same content -as the previous example: - -.. literalinclude:: ../examples/icatdump-simple-2.xml - :language: xml - :lines: 1-9,28-52,56-58,70-82,108 - -The difference is that we now add the Usergroup objects separately in -direct subelements of ``data`` instead of including them in the -related Grouping objects. - You will find more extensive examples in the source distribution of python-icat. The distribution also provides XML Schema Definition files for the ICAT data XML file format corresponding to various ICAT -schema versions. +schema versions. Note the these XML Schema Definition +files are provided for reference only. The :ref:`icatingest` script +does not validate its input. ICAT data YAML files ~~~~~~~~~~~~~~~~~~~~