Bring in NOMAD examples

.github/workflows/pytest.yml

@@ -26,9 +26,15 @@ jobs:
         uses: actions/setup-python@v5
         with:
           python-version: ${{ matrix.python_version }}
-      - name: Install package
+      - name: Install uv
         run: |
           curl -LsSf https://astral.sh/uv/install.sh | sh
+      - name: Install nomad
+        if: "${{ matrix.python_version != '3.8'}"
+        run: |
+          uv pip install nomad-lab[infrastructure]@git+https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR.git
+      - name: Install package
+        run: |
           uv pip install ".[dev]"
       - name: Test with pytest
         run: |

.pre-commit-config.yaml

@@ -1,7 +1,7 @@
 repos:
   - repo: https://github.com/astral-sh/ruff-pre-commit
     # Ruff version.
-    rev: v0.7.0
+    rev: v0.8.0
     hooks:
       # Run the linter.
       - id: ruff

MANIFEST.in

@@ -1,4 +1,5 @@
 prune *
 exclude *
 recursive-include src/pynxtools_mpes *.py
-include pyproject.toml README.md dev-requirements.txt
+include pyproject.toml README.md dev-requirements.txt
+graft src/pynxtools_mpes/nomad/examples

docs/tutorial/standalone.md

@@ -33,7 +33,7 @@ $<eln-file path> \
 
 ### Examples
 
-You can find exhaustive examples how to use `pynxtools-mpes` for your ARPES research data pipeline [here](https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-remote-tools-hub/-/tree/develop/docker/mpes).
+You can find exhaustive examples how to use `pynxtools-mpes` for your ARPES research data pipeline in [`src/pynxtools-mpes/nomad/examples`](../../src/pynxtools_mpes/nomad/examples/). These are designed for working with [`NOMAD`](https://nomad-lab.eu/) and its [`NOMAD Remote Tools Hub (NORTH)`](https://nomad-lab.eu/prod/v1/gui/analyze/north).
 
 There are also small example files for using the `pynxtools` dataconverter with the `mpes` reader and the `NXmpes` application definition in [`tests/data`](https://github.com/FAIRmat-NFDI/pynxtools.mpes/tree/main/tests/data).
 

pyproject.toml

@@ -55,6 +55,9 @@ docs = [
 [project.entry-points."pynxtools.reader"]
 mpes = "pynxtools_mpes.reader:MPESReader"
 
+[project.entry-points.'nomad.plugin']
+mpes_example = "pynxtools_mpes.nomad.entrypoints:mpes_example"
+
 [tool.setuptools.packages.find]
 where = [
     "src",

src/pynxtools_mpes/nomad/entrypoints.py

@@ -0,0 +1,41 @@
+#
+# Copyright The NOMAD Authors.
+#
+# This file is part of NOMAD. See https://nomad-lab.eu for further info.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+"""Entry points for mpes examples."""
+
+try:
+    from nomad.config.models.plugins import ExampleUploadEntryPoint
+except ImportError as exc:
+    raise ImportError(
+        "Could not import nomad package. Please install the package 'nomad-lab'."
+    ) from exc
+
+mpes_example = ExampleUploadEntryPoint(
+    title="Multidimensional photoemission spectroscopy (MPES)",
+    category="FAIRmat examples",
+    description="""
+        This example presents the capabilities of the NOMAD platform to store and standardize multidimensional photoemission spectroscopy (MPES) experimental data. It contains four major examples:
+
+      - Taking a pre-binned file, here stored in a h5 file, and converting it into the standardized MPES NeXus format.
+        There exists a [NeXus application definition for MPES](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes) which details the internal structure of such a file.
+      - Binning of raw data (see [here](https://www.nature.com/articles/s41597-020-00769-8) for additional resources) into a h5 file and consecutively generating a NeXus file from it.
+      - An analysis example using data in the NeXus format and employing the [pyARPES](https://github.com/chstan/arpes) analysis tool to reproduce the main findings of [this paper](https://arxiv.org/pdf/2107.07158.pdf).
+      - Importing angle-resolved data stored in NXmpes_arpes, and convert these into momentum space coordinates using tools in pyARPES.
+    """,
+    plugin_package="pynxtools_mpes",
+    resources=["nomad/examples/*"],
+)

src/pynxtools_mpes/nomad/examples/E1 Convert to NeXus.ipynb

@@ -0,0 +1,161 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Converting Multidimensional Photoemission Spectroscopy (MPES) data into the NeXus format"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This example shows how to convert x-array based h5 measurement files as generated by the [preprocessing output](https://www.nature.com/articles/s41597-020-00769-8) of the software used at the Fritz-Haber Institute into the [MPES NeXus format](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes).\n",
+    "For an example on how to generate such a h5 measurement file please refer to the [postprocessing example](./E2%20ARPES%20postprocessing.ipynb)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Download the data\n",
+    "Since the provided datafile is comparably large (~200MB) it is not directly provided with the example.\n",
+    "You can [download](https://zenodo.org/record/7573825/files/MoTe2.h5?download=1) it from zenodo. Place the file in the directory of this notebook afterwards. Under Linux, macOS and in a NORTH container you can directly use the cell below to download the file with curl."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "tags": [
+     "skip-execution"
+    ]
+   },
+   "outputs": [],
+   "source": [
+    "! curl -o MoTe2.h5 \"https://zenodo.org/records/7573825/files/MoTe2.h5?download=1\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Convert the file to NeXus\n",
+    "To convert the available files to the NeXus format we use the convert function readily supplied by pynxtools.\n",
+    "It uses the downloaded measurement file, a json config file and optionally an electronic lab notebook (ELN) yaml file that is used in the [binning of WSe2 example](./E2%20Binning%20of%20WSe2.ipynb).\n",
+    "The json config file maps specific metadata from the h5 measurement file to the nxs file, i.e. a pressure reading which automatically gets collected during measurement.\n",
+    "The ELN is a file which supplies additional metadata which is written into the NeXus file.\n",
+    "This is data which is not collected automatically, such as the person conducting the experiment.\n",
+    "It can be written manually or generated, e.g. by the NOMAD ELN functionality.\n",
+    "\n",
+    "The convert command may also be executed in the command line with the command `dataconverter`:\n",
+    "```\n",
+    "dataconverter \\\n",
+    "    --reader mpes \\\n",
+    "    --nxdl NXmpes \\\n",
+    "    --output MoTe2.mpes.nxs \\\n",
+    "    MoTe2.h5 \\\n",
+    "    config_file.json\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pynxtools.dataconverter.convert import convert"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The input parameters are defined as follows:\n",
+    "\n",
+    "**reader**: The specific reader which gets called inside pynxtools. This is supplied in the pynxtools python code. If you create a specific reader for your measurement file it gets selecetd here. If you use the binning procedure from FHI to generate a xarray h5 file you should use the reader called `mpes`.\n",
+    "\n",
+    "**nxdl**: The specific nxdl file to be used. For MPES this should always be `NXmpes` or one of its subdefinitions of the form `NXmpes_<name>`.\n",
+    "    \n",
+    "**output**: The output filename of the NeXus file."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "convert(input_file=[\"MoTe2.h5\", \"config_file.json\"],\n",
+    "        reader='mpes',\n",
+    "        nxdl='NXmpes',\n",
+    "        output='MoTe2.mpes.nxs')"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "tags": []
+   },
+   "source": [
+    "## View the data with H5Web\n",
+    "H5Web is a tool for visualizing any data in the h5 data format. Since the NeXus format builds opon h5 it can be used to view this data as well. We just import the package and call H5Web with the output filename from the convert command above. For an analysis on NeXus data files please refer to [analysis example](./E3%20pyARPES%20analysis.ipynb).\n",
+    "\n",
+    "You can also view this data with the H5Viewer or other tools from your local filesystem."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from jupyterlab_h5web import H5Web"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "H5Web('MoTe2.mpes.nxs')"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.12"
+  },
+  "vscode": {
+   "interpreter": {
+    "hash": "982c6a11128e6710069266786b6cc2519a098e817123fe996210f6bb9a67eb40"
+   }
+  },
+  "widgets": {
+   "application/vnd.jupyter.widget-state+json": {
+    "state": {},
+    "version_major": 2,
+    "version_minor": 0
+   }
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}