Skip to content

Commit

Permalink
Deployed 2ab4674 with MkDocs version: 1.6.1
Browse files Browse the repository at this point in the history
  • Loading branch information
github-actions[bot] committed Dec 5, 2024
1 parent 92f889e commit ed19bcf
Show file tree
Hide file tree
Showing 10 changed files with 28 additions and 30 deletions.
2 changes: 1 addition & 1 deletion how-tos/build-a-plugin.html
Original file line number Diff line number Diff line change
Expand Up @@ -1190,7 +1190,7 @@


<h1 id="build-your-own-pynxtools-plugin">Build your own pynxtools plugin<a class="headerlink" href="#build-your-own-pynxtools-plugin" title="Permanent link">&para;</a></h1>
<p>The pynxtools <a href="https://github.com/FAIRmat-NFDI/pynxtools/tree/master/src/pynxtools/dataconverter">dataconverter</a> is used to convert experimental data to NeXus/HDF5 files based on any provided <a href="https://manual.nexusformat.org/nxdl.html#index-1">NXDL schemas</a>. The converter allows extending support to other data formats by allowing extensions called <code>readers</code>. There exist a set of <a href="https://github.com/FAIRmat-NFDI/pynxtools/tree/master/src/pynxtools/dataconverter/readers">built-in pynxtools readers</a> as well as <a href="../reference/plugins.html">pynxtools plugins</a> to convert supported data files for some experimental techniques into compliant NeXus files.</p>
<p>The pynxtools <a href="https://github.com/FAIRmat-NFDI/pynxtools/tree/master/src/pynxtools/dataconverter">dataconverter</a> is used to convert experimental data to NeXus/HDF5 files based on any provided <a href="https://manual.nexusformat.org/nxdl.html">NXDL schemas</a>. The converter allows extending support to other data formats by allowing extensions called <code>readers</code>. There exist a set of <a href="../reference/built-in-readers.html">built-in pynxtools readers</a> as well as <a href="../reference/plugins.html">pynxtools reader plugins</a> to convert supported data files for some experimental techniques into NeXus-compliant files.</p>
<p>Your current data is not supported yet by the built-in pynxtools readers or the officially supported pynxtools plugins?</p>
<p>Don't worry, the following how-to will guide you through the steps of writing a reader for your own data.</p>
<h2 id="getting-started">Getting started<a class="headerlink" href="#getting-started" title="Permanent link">&para;</a></h2>
Expand Down
8 changes: 4 additions & 4 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -1132,11 +1132,11 @@ <h1 id="fairmat-nexus-documentation">FAIRmat NeXus documentation<a class="header
<p><code>pynxtools</code> offers scientists a convenient way to use the NeXus format and solves the challenge of unstructured and non-standardized data in experimental materials science. We consider this package useful for meeting the following FAIR principle as defined in <a href="https://direct.mit.edu/dint/article/2/1-2/10/10017/FAIR-Principles-Interpretations-and-Implementation">FAIR Principles: Interpretations and Implementation Considerations</a>: F2-4, I2-I3, and R1.</p>
<!-- Finally, a paragraph that describes whom the product is useful for. -->
<p>FAIRmat's contribution to the existing NeXus standard, together with the tools provided through <code>pynxtools</code>, enable scientists and research groups working with data, as well as helping communities implement standardized FAIR research data.</p>
<p>Additionally, the software is used as a plugin in the research data management system <a href="https://nomad-lab.eu/nomad-lab/">NOMAD</a> for making experimental data searchable and publishable. NOMAD is developed by the FAIRMAT consortium, as a part of the German National Research Data Infrastructure (NFDI).</p>
<p>Additionally, the software is used as a plugin in the research data management system <a href="https://nomad-lab.eu/nomad-lab/">NOMAD</a> for making experimental data searchable and publishable. NOMAD is developed by the FAIRmat consortium, as a part of the German National Research Data Infrastructure (NFDI).</p>
<div class="home-grid">
<div>
<h3 id="tutorial">Tutorial<a class="headerlink" href="#tutorial" title="Permanent link">&para;</a></h3>
<p>A series of tutorials giving you an overview on how to store or convert your data to NeXus compliant files.</p>
<p>A series of tutorials giving you an overview on how to store or convert your data to NeXus-compliant files.</p>
<ul>
<li><a href="tutorial/converting-data-to-nexus.html">Converting your data to NeXus</a></li>
<li><a href="tutorial/nexus-to-nomad.html">Uploading NeXus data to NOMAD</a></li>
Expand All @@ -1152,7 +1152,7 @@ <h3 id="how-to-guides">How-to guides<a class="headerlink" href="#how-to-guides"
<li><a href="how-tos/create-nexus-files-by-python.html">Creation of NeXus files in python via hard-coding</a></li>
<li><a href="how-tos/using-pynxtools-test-framework.html">Using pynxtools test framework for plugins</a></li>
</ul>
<p><strong>The following How-To guides are still under development:</strong></p>
<p><strong>The following How-to guides are still under development:</strong></p>
<ul>
<li><a href="how-tos/writing-an-appdef.html">Writing an application definition</a></li>
<li><a href="how-tos/using-multiple-appdefs.html">Storing data in multiple application definitions</a></li>
Expand Down Expand Up @@ -1187,7 +1187,7 @@ <h4 id="pynxtools_1">pynxtools<a class="headerlink" href="#pynxtools_1" title="P
<p>Within FAIRmat, we maintain a number of generic built-in pynxtools readers, together with reader plugins for different experimental techniques. Here you can find more information:</p>
<ul>
<li><a href="reference/built-in-readers.html">Built-in pynxtools readers</a></li>
<li><a href="reference/plugins.html">FAIRMat-supported pynxtools plugins</a></li>
<li><a href="reference/plugins.html">FAIRmat-supported pynxtools plugins</a></li>
</ul>
</div>
</div>
Expand Down
14 changes: 5 additions & 9 deletions learn/nexus-primer.html
Original file line number Diff line number Diff line change
Expand Up @@ -980,22 +980,18 @@ <h1 id="a-primer-on-nexus">A primer on NeXus<a class="headerlink" href="#a-prime
<p class="admonition-title">Work in progress</p>
<p>This part of the documentation is still being written and it might be confusing or incomplete.</p>
</div>
<p>NeXus is is a description of a common data exchange format initially developed for neutron, X-ray, and muon experiments. Within FAIRmat we extensively extended the format to cover a range of experiments with major support for APM, ARPES, XPS, and optical spectroscpy but we also give advice and guidance for developing standards for other formats as well.</p>
<p>NeXus is is a description of a common data exchange format initially developed for neutron, X-ray, and muon experiments. Within FAIRmat we extensively extended the format to cover a range of experiments with major support for APM, ARPES, XPS, and optical spectroscopy, but we also give advice and guidance for developing standards for other formats as well.</p>
<div class="admonition info">
<p class="admonition-title">NeXus as a tool for FAIR data</p>
<p>NeXus is supported be the research data management platform NOMAD.
Experimental data following an NeXus application definition can easily be uploaded and is recognized by NOMAD's search system.
If you want to learn more about uploading NeXus data to NOMAD, please refer to the <a href="../tutorial/nexus-to-nomad.html">NeXus to nomad</a> tutorial
of this documentation.
Accordingly, if you want to build data according to the FAIR principles, you can think of NeXus fulfilling the interoperability and
reproducibility part and a research data management platform like NOMAD the findable and accessible part.</p>
Experimental data following an NeXus application definition can easily be uploaded and is recognized by NOMAD's search system. If you want to learn more about uploading NeXus data to NOMAD, please refer to the <a href="../tutorial/nexus-to-nomad.html">NeXus to NOMAD</a> tutorial of this documentation. Accordingly, if you want to build data according to the FAIR principles, you can think of NeXus fulfilling the interoperability and reproducibility part and a research data management platform like NOMAD the findable and accessible part.</p>
</div>
<h2 id="what-is-nexus">What is NeXus?<a class="headerlink" href="#what-is-nexus" title="Permanent link">&para;</a></h2>
<p>Sometimes, NeXus is seen as writing data to some form of file in hdf5 format.
While this is partly true, NeXus is independent of the actual storage format but is typically written into an hdf5 file.</p>
<p>Sometimes, NeXus is seen as writing data to some form of file in HDF5 format.
While this is partly true, NeXus is independent of the actual storage format, but is typically written into an HDF5 file.</p>
<p>But what is NeXus then? It is the conceptual layer above the file structure.
It is a contract on which data has to be present and how to name them in a given dataset.
Hence, using NeXus participates in making data FAIR.
Hence, the use of NeXus helps to make data FAIR.
It especially covers the interoperability and reproducibility part of research data.</p>
<div class="admonition info">
<p class="admonition-title">NeXus path notations</p>
Expand Down
2 changes: 1 addition & 1 deletion learn/nexus-rules.html
Original file line number Diff line number Diff line change
Expand Up @@ -1011,7 +1011,7 @@ <h2 id="namefitting">Namefitting<a class="headerlink" href="#namefitting" title=
<p>Note that this explicitly also means that it is not allowed to have whitespace (including " ") in NeXus names.</p>
<p>In NeXus base classes and application definitions, there are two options for defining a concept name. If the group or field in the definition is lowercase, that means that any instance must have the exact same (<strong>fixed</strong>) name. As an example, if there is a field called <code>my_field</code> in an application definition, the only allowed name in a file would be <code>my_field</code>.</p>
<p>Aside from this lower case notation, there is also the option to allow for <strong>selectable</strong> names. This is achieved by uppercase notation. As an example, if a field in an application definition is called <code>FIELD</code>, the name can be any name as long as it maches the regular expression above. For example, <code>field</code>, <code>field0</code>, <code>any_other_name</code> would be allowed names, while <code>any other name</code> would not be allowed.</p>
<p>There is also the possibility of mixed lowercase and uppercase notation in base classes and application definitions. For example, there might be a <code>userID(NXuser)</code> group. In this case, allowed names include any name that start with <code>user</code>, e.g., <code>user0</code>, <code>user_abcde</code>, as long as the part that replaces the docstring is still valid according to the regex above. Note that here it is also <strong>not</strong> allowed to write <code>user</code> without replacing the uppercase part of the name.</p>
<p>There is also the possibility of mixed lowercase and uppercase notation in base classes and application definitions. For example, there might be a <code>userID(NXuser)</code> group. In this case, allowed names include any name that start with <code>user</code>, e.g., <code>user0</code>, <code>user_abcde</code>, as long as the part that replaces the docstring is still valid according to the regex above. Note that here it is also allowed to write <code>user</code> without replacing the uppercase part of the name.</p>
<p>The validation of names is performed by <strong>namefitting</strong>, i.e., fitting the name that is used by the data provider to the name given in the base class / application definitions.</p>
<p>A python implementation of this process can be found in <a href="https://github.com/FAIRmat-NFDI/nexus_definitions/blob/34aed4a74b8d2a682eb0b9292055dc00e5e0220e/dev_tools/utils/nxdl_utils.py#L112">this function</a>. This function returns twice the length for an exact match, otherwise the number of matching characters (case insensitive) or zero, if <code>name_any</code> is set to True, is returned. This is also the function that is used in the <a href="https://github.com/FAIRmat-NFDI/pynxtools/blob/master/src/pynxtools/dataconverter/validation.py">validation</a> implemented in pynxtools.</p>
<h2 id="special-rules-for-names-and-namefitting">Special rules for names and namefitting<a class="headerlink" href="#special-rules-for-names-and-namefitting" title="Permanent link">&para;</a></h2>
Expand Down
2 changes: 1 addition & 1 deletion learn/nexus-validation.html
Original file line number Diff line number Diff line change
Expand Up @@ -1090,7 +1090,7 @@ <h3 id="a-note-to-windows-users">A note to Windows users<a class="headerlink" hr
</code></pre></div>
<p>This workaround was tested with Windows 11, but should very likely also work with Windows 10 and lower.</p>
<h2 id="other-approaches-not-part-of-pynxtools">Other approaches (not part of pynxtools)<a class="headerlink" href="#other-approaches-not-part-of-pynxtools" title="Permanent link">&para;</a></h2>
<p>Aside from the tools we developed within FAIRmat, the <a href="https://manual.nexusformat.org/validation.htm">official NeXus website</a> lists two more programs for the validation of NeXus files:</p>
<p>Aside from the tools we developed within FAIRmat, the <a href="https://manual.nexusformat.org/validation.html">official NeXus website</a> lists two more programs for the validation of NeXus files:</p>
<ol>
<li><a href="https://github.com/nexusformat/cnxvalidate">cnxvalidate</a></li>
<li><a href="https://github.com/prjemian/punx">punx</a></li>
Expand Down
8 changes: 5 additions & 3 deletions reference/built-in-readers.html
Original file line number Diff line number Diff line change
Expand Up @@ -1161,9 +1161,11 @@ <h2 id="the-jsonmapreader">The <a href="https://github.com/FAIRmat-NFDI/pynxtool
<h3 id="the-mappingjson-file">The mapping.json file<a class="headerlink" href="#the-mappingjson-file" title="Permanent link">&para;</a></h3>
<p>This file is designed to let you fill in the requirements of a NeXus Application Definition without writing any code. If you already have data in the formats listed above, you just need to use this mapping file to help the dataconverter pick your data correctly.</p>
<p>The mapping files will always be based on the template the dataconverter generates. See above on how to generate a mapping file. The right hand side values of the template keys are what you can modify. These keys are called NeXus template paths, because they combine the actual path that will be used in the HDF5 hierarchy with additional NeXus datatype hints to guide the dataconverter to add NX_class annotations.</p>
<p>Here are the three different ways you can fill the right hand side of the template keys:
* Write the nested path in your datafile. This is indicated by a leading <code>/</code> before the word <code>entry</code> to make <code>/entry/data/current_295C</code> below.
Example:</p>
<p>Here are the three different ways you can fill the right hand side of the template keys:</p>
<ul>
<li>Write the nested path in your datafile. This is indicated by a leading <code>/</code> before the word <code>entry</code> to make <code>/entry/data/current_295C</code> below.
Example:</li>
</ul>
<p><div class="highlight"><pre><span></span><code><a id="__codelineno-1-1" name="__codelineno-1-1" href="#__codelineno-1-1"></a><span class="w"> </span><span class="nt">&quot;/ENTRY[entry]/DATA[data]/current_295C&quot;</span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;/entry/data/current_295C&quot;</span><span class="p">,</span>
<a id="__codelineno-1-2" name="__codelineno-1-2" href="#__codelineno-1-2"></a><span class="w"> </span><span class="nt">&quot;/ENTRY[entry]/NXODD_name/posint_value&quot;</span><span class="p">:</span><span class="w"> </span><span class="s2">&quot;/a_level_down/another_level_down/posint_value&quot;</span><span class="p">,</span>
</code></pre></div>
Expand Down
2 changes: 1 addition & 1 deletion reference/definitions.html
Original file line number Diff line number Diff line change
Expand Up @@ -924,7 +924,7 @@ <h1 id="nexus-definitions">NeXus definitions<a class="headerlink" href="#nexus-d
<li><a href="https://manual.nexusformat.org/classes/index.html">Official NIAC definitions</a></li>
<li><a href="https://fairmat-nfdi.github.io/nexus_definitions/">Latest FAIRmat definitions</a></li>
</ul>
<p>The FAIRmat definitions are regularly contributed to NIAC (around every 6 months) but generally reflect
<p>The FAIRmat definitions are regularly contributed to NIAC (around every 6 months), but generally reflect
a state which is still under development and may contain new or improved application definitions or base classes.
Consider it as the public review stage of these application definitions.
However, there might be some parts which are still under discussion and will be subject to change.</p>
Expand Down
2 changes: 1 addition & 1 deletion search/search_index.json

Large diffs are not rendered by default.

Binary file modified sitemap.xml.gz
Binary file not shown.
Loading

0 comments on commit ed19bcf

Please sign in to comment.