diff --git a/src/pynxtools_mpes/nomad/examples/E1 Convert to NeXus.ipynb b/src/pynxtools_mpes/nomad/examples/E1 Convert to NeXus.ipynb index dbcc2b2..921df2e 100644 --- a/src/pynxtools_mpes/nomad/examples/E1 Convert to NeXus.ipynb +++ b/src/pynxtools_mpes/nomad/examples/E1 Convert to NeXus.ipynb @@ -1,7 +1,6 @@ { "cells": [ { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -9,16 +8,14 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "This example shows how to convert x-array based h5 measurement files as generated by the [preprocessing output](https://www.nature.com/articles/s41597-020-00769-8) of the software used at the Fritz-Haber Institute into the [MPES NeXus format](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes).\n", - "For an example on how to generate such a h5 measurement file please refer to the [binning example](./E2%20Binning%20of%20WSe2.ipynb)." + "For an example on how to generate such a h5 measurement file please refer to the [postprocessing example](./E2%20ARPES%20postprocessing.ipynb)." ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -41,7 +38,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -58,15 +54,15 @@ "dataconverter \\\n", " --reader mpes \\\n", " --nxdl NXmpes \\\n", - " --input-file MoTe2.h5 \\\n", - " --input-file config_file.json \\\n", - " --output MoTe2.mpes.nxs\n", + " --output MoTe2.mpes.nxs \\\n", + " MoTe2.h5 \\\n", + " config_file.json\n", "```" ] }, { "cell_type": "code", - "execution_count": null, + "execution_count": 1, "metadata": {}, "outputs": [], "source": [ @@ -74,7 +70,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ @@ -100,7 +95,6 @@ ] }, { - "attachments": {}, "cell_type": "markdown", "metadata": { "tags": [] @@ -133,7 +127,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -147,7 +141,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.13" + "version": "3.8.12" }, "vscode": { "interpreter": { diff --git a/src/pynxtools_mpes/nomad/examples/E2 ARPES postprocessing.ipynb b/src/pynxtools_mpes/nomad/examples/E2 ARPES postprocessing.ipynb new file mode 100644 index 0000000..5382ef7 --- /dev/null +++ b/src/pynxtools_mpes/nomad/examples/E2 ARPES postprocessing.ipynb @@ -0,0 +1,874 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "8ad4167a-e4e7-498d-909a-c04da9f177ed", + "metadata": { + "tags": [] + }, + "source": [ + "# Postprocessing Multidimensional Photoemission Spectroscopy (MPES) data and converting it into the NeXus format\n", + "\n", + "In this example, we pull data from a time-resolved ARPES on WSe2 measurement from Zenodo, and load it into the sed package using functions of the mpes package. Then, we run a conversion pipeline on it, containing steps for visualizing the channels, correcting image distortions, calibrating the momentum space, correcting for energy distortions and calibrating the energy axis. Finally, the data are binned in calibrated axes and stored in the standardised [MPES NExus format](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html#nxmpes).\n", + "For performance reasons, best store the data on a locally attached storage (no network drive). This can also be achieved transparently using the included MirrorUtil class.\n", + "\n", + "This example works on a rather large dataset (~6GB) and hence it takes some time and resources to execute. If you just want to learn how to convert already processed data in an xarray based h5 file into the NeXus format you may have a look at the simpler [Convert to NeXus example](./E1%20Convert%20to%20NeXus.ipynb), which is runs quicker and has lower requirements.\n", + "\n", + "Further information on the postprocessing pipeline can be found in the [documentation of sed](https://opencompes.github.io/sed/) or in [R.P. Xian et al., Sci Data 7, 442 (2020)](https://www.nature.com/articles/s41597-020-00769-8)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fb045e17-fa89-4c11-9d51-7f06e80d96d5", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "import os\n", + "\n", + "import sed\n", + "\n", + "%matplotlib widget" + ] + }, + { + "cell_type": "markdown", + "id": "5d8f092b-df92-4abb-b00c-444f0351efd3", + "metadata": {}, + "source": [ + "## Download RAW data (trARPES data of WSe2)\n", + "\n", + "Here, we just set the main file folder for holding the measurement data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e14a45c3-b8d0-4aa0-8469-f2c641fdb236", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "FDIR = f'{os.getcwd()}/data/Scan049_1'\n", + "ECAL = f'{os.getcwd()}/data/energycal_2019_01_08'" + ] + }, + { + "cell_type": "markdown", + "id": "fbbd67d7-2a2b-42ec-ba9e-28c1e64d6dce", + "metadata": {}, + "source": [ + "Since the provided measurement files are rather large (~6GB), they are not directly provided with the example.\n", + "You can [download](https://zenodo.org/record/6369728/files/WSe2.zip) it from zenodo. This may take some time. Place the file in the directory of this notebook afterwards. Under Linux, macOS and in a NORTH container you can directly use the cell below to download the file with curl." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "334632d0-84d1-46fe-a4c0-3966feb1e69a", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "! curl -o WSe2.zip \"https://zenodo.org/records/6369728/files/WSe2.zip\"" + ] + }, + { + "cell_type": "markdown", + "id": "c12a6c18-a6c6-417f-b016-1c7e11fe9116", + "metadata": {}, + "source": [ + "Now we extract the measurement files." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f79c9e03-7f8a-4917-97f9-bf94fc36270e", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "! unzip WSe2.zip -d data" + ] + }, + { + "cell_type": "markdown", + "id": "42a6afaa-17dd-4637-ba75-a28c4ead1adf", + "metadata": {}, + "source": [ + "## Load Data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "906abf04-6232-4b72-90f1-3843d51b5108", + "metadata": { + "jupyter": { + "source_hidden": true + }, + "tags": [] + }, + "outputs": [], + "source": [ + "metadata = {}\n", + "# manual Meta data. These should ideally come from an Electronic Lab Notebook.\n", + "#General\n", + "metadata['experiment_summary'] = 'WSe2 XUV NIR pump probe data.'\n", + "metadata['entry_title'] = 'Valence Band Dynamics - 800 nm linear s-polarized pump, 0.6 mJ/cm2 absorbed fluence'\n", + "metadata['experiment_title'] = 'Valence band dynamics of 2H-WSe2'\n", + "\n", + "#User\n", + "# Fill general parameters of NXuser\n", + "# TODO: discuss how to deal with multiple users?\n", + "metadata['user0'] = {}\n", + "metadata['user0']['name'] = 'Julian Maklar'\n", + "metadata['user0']['role'] = 'Principal Investigator'\n", + "metadata['user0']['affiliation'] = 'Fritz Haber Institute of the Max Planck Society'\n", + "metadata['user0']['address'] = 'Faradayweg 4-6, 14195 Berlin'\n", + "metadata['user0']['email'] = 'maklar@fhi-berlin.mpg.de'\n", + "\n", + "#NXinstrument\n", + "metadata['instrument'] = {}\n", + "metadata['instrument']['energy_resolution'] = 140.\n", + "#analyzer\n", + "metadata['instrument']['analyzer']={}\n", + "metadata['instrument']['analyzer']['slow_axes'] = \"delay\" # the scanned axes\n", + "metadata['instrument']['analyzer']['spatial_resolution'] = 10.\n", + "metadata['instrument']['analyzer']['energy_resolution'] = 110.\n", + "metadata['instrument']['analyzer']['momentum_resolution'] = 0.08\n", + "metadata['instrument']['analyzer']['working_distance'] = 4.\n", + "metadata['instrument']['analyzer']['lens_mode'] = \"6kV_kmodem4.0_30VTOF.sav\"\n", + "\n", + "#probe beam\n", + "metadata['instrument']['beam']={}\n", + "metadata['instrument']['beam']['probe']={}\n", + "metadata['instrument']['beam']['probe']['incident_energy'] = 21.7\n", + "metadata['instrument']['beam']['probe']['incident_energy_spread'] = 0.11\n", + "metadata['instrument']['beam']['probe']['pulse_duration'] = 20.\n", + "metadata['instrument']['beam']['probe']['frequency'] = 500.\n", + "metadata['instrument']['beam']['probe']['incident_polarization'] = [1, 1, 0, 0] # p pol Stokes vector\n", + "metadata['instrument']['beam']['probe']['extent'] = [80., 80.]\n", + "#pump beam\n", + "metadata['instrument']['beam']['pump']={}\n", + "metadata['instrument']['beam']['pump']['incident_energy'] = 1.55\n", + "metadata['instrument']['beam']['pump']['incident_energy_spread'] = 0.08\n", + "metadata['instrument']['beam']['pump']['pulse_duration'] = 35.\n", + "metadata['instrument']['beam']['pump']['frequency'] = 500.\n", + "metadata['instrument']['beam']['pump']['incident_polarization'] = [1, -1, 0, 0] # s pol Stokes vector\n", + "metadata['instrument']['beam']['pump']['incident_wavelength'] = 800.\n", + "metadata['instrument']['beam']['pump']['average_power'] = 300.\n", + "metadata['instrument']['beam']['pump']['pulse_energy'] = metadata['instrument']['beam']['pump']['average_power']/metadata['instrument']['beam']['pump']['frequency']#µJ\n", + "metadata['instrument']['beam']['pump']['extent'] = [230., 265.]\n", + "metadata['instrument']['beam']['pump']['fluence'] = 0.15\n", + "\n", + "#sample\n", + "metadata['sample']={}\n", + "metadata['sample']['preparation_date'] = '2019-01-13T10:00:00+00:00'\n", + "metadata['sample']['preparation_description'] = 'Cleaved'\n", + "metadata['sample']['sample_history'] = 'Cleaved'\n", + "metadata['sample']['chemical_formula'] = 'WSe2'\n", + "metadata['sample']['description'] = 'Sample'\n", + "metadata['sample']['name'] = 'WSe2 Single Crystal'\n", + "\n", + "metadata['file'] = {}\n", + "metadata['file'][\"trARPES:Carving:TEMP_RBV\"] = 300.\n", + "metadata['file'][\"trARPES:XGS600:PressureAC:P_RD\"] = 5.e-11\n", + "metadata['file'][\"KTOF:Lens:Extr:I\"] = -0.12877\n", + "metadata['file'][\"KTOF:Lens:UDLD:V\"] = 399.99905\n", + "metadata['file'][\"KTOF:Lens:Sample:V\"] = 17.19976\n", + "metadata['file'][\"KTOF:Apertures:m1.RBV\"] = 3.729931\n", + "metadata['file'][\"KTOF:Apertures:m2.RBV\"] = -5.200078\n", + "metadata['file'][\"KTOF:Apertures:m3.RBV\"] = -11.000425\n", + "\n", + "# Sample motor positions\n", + "metadata['file']['trARPES:Carving:TRX.RBV'] = 7.1900000000000004\n", + "metadata['file']['trARPES:Carving:TRY.RBV'] = -6.1700200225439552\n", + "metadata['file']['trARPES:Carving:TRZ.RBV'] = 33.4501953125\n", + "metadata['file']['trARPES:Carving:THT.RBV'] = 423.30500940561586\n", + "metadata['file']['trARPES:Carving:PHI.RBV'] = 0.99931647456264949\n", + "metadata['file']['trARPES:Carving:OMG.RBV'] = 11.002500171914066" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "77c9e4a2-e6fd-4ab0-bd55-417cc3995e13", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "base_path = '.'\n", + "data_path = f'{base_path}/data'" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f1f82054", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# The Scan directory\n", + "fdir = data_path + '/Scan049_1'\n", + "# create sed processor using the config file:\n", + "sp = sed.SedProcessor(\n", + " folder=fdir,\n", + " config=f\"{base_path}/sed_config/mpes_example_config.yaml\",\n", + " metadata=metadata,\n", + " collect_metadata=True,\n", + " verbose=True,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "50d0a3b3", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Apply jittering to X, Y, t, ADC columns.\n", + "# Columns are defined in the config, or can be provided as list.\n", + "sp.add_jitter()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0a0d336b", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Plot of the count rate through the scan\n", + "rate, secs = sp.loader.get_count_rate(range(100))\n", + "plt.plot(secs, rate)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "dfb42777", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# The time elapsed in the scan\n", + "sp.loader.get_elapsed_time()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fb074f29", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "# Inspect data in dataframe Columns:\n", + "# axes = ['X', 'Y', 't', 'ADC']\n", + "# bins = [100, 100, 100, 100]\n", + "# ranges = [(0, 1800), (0, 1800), (130000, 140000), (0, 9000)]\n", + "# sp.view_event_histogram(dfpid=1, axes=axes, bins=bins, ranges=ranges)\n", + "sp.view_event_histogram(dfpid=2)" + ] + }, + { + "cell_type": "markdown", + "id": "70aa4343", + "metadata": {}, + "source": [ + "## Distortion correction and Momentum Calibration workflow\n", + "### Distortion correction\n", + "#### 1. step: \n", + "Bin and load part of the dataframe in detector coordinates, and choose energy plane where high-symmetry points can well be identified. Either use the interactive tool, or pre-select the range:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "76bf8aad", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "#sp.bin_and_load_momentum_calibration(df_partitions=20, plane=170)\n", + "sp.bin_and_load_momentum_calibration(df_partitions=100, plane=33, width=10, apply=True)" + ] + }, + { + "cell_type": "markdown", + "id": "fee3ca76", + "metadata": {}, + "source": [ + "#### 2. Step:\n", + "Next, we select a number of features corresponding to the rotational symmetry of the material, plus the center. These can either be auto-detected (for well-isolated points), or provided as a list (these can be read-off the graph in the cell above).\n", + "These are then symmetrized according to the rotational symmetry, and a spline-warping correction for the x/y coordinates is calculated, which corrects for any geometric distortions from the perfect n-fold rotational symmetry." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fd9666c5", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "#features = np.array([[203.2, 341.96], [299.16, 345.32], [350.25, 243.70], [304.38, 149.88], [199.52, 152.48], [154.28, 242.27], [248.29, 248.62]])\n", + "#sp.define_features(features=features, rotation_symmetry=6, include_center=True, apply=True)\n", + "# Manual selection: Use a GUI tool to select peaks:\n", + "#sp.define_features(rotation_symmetry=6, include_center=True)\n", + "#sp.generate_splinewarp(rotation_symmetry=6, include_center=True, fwhm=10, sigma=12, sigma_radius=4)\n", + "# Autodetect: Uses the DAOStarFinder routine to locate maxima.\n", + "# Parameters are:\n", + "# fwhm: Full-width at half maximum of peaks.\n", + "# sigma: Number of standard deviations above the mean value of the image peaks must have.\n", + "# sigma_radius: number of standard deviations around a peak that peaks are fitted\n", + "sp.define_features(rotation_symmetry=6, auto_detect=True, include_center=True, fwhm=10, sigma=12, sigma_radius=4, apply=True)" + ] + }, + { + "cell_type": "markdown", + "id": "f7519ff8", + "metadata": {}, + "source": [ + "#### 3. Step: \n", + "Generate nonlinear correction using splinewarp algorithm. If no landmarks have been defined in previous step, default parameters from the config are used" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d27cd7a4", + "metadata": {}, + "outputs": [], + "source": [ + "# Option whether a central point shall be fixed in the determiantion fo the correction\n", + "sp.generate_splinewarp(include_center=True)" + ] + }, + { + "cell_type": "markdown", + "id": "4211ac21", + "metadata": {}, + "source": [ + "#### Optional (Step 3a): \n", + "Save distortion correction parameters to configuration file in current data folder: " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7f32988f", + "metadata": {}, + "outputs": [], + "source": [ + "# Save generated distortion correction parameters for later reuse\n", + "sp.save_splinewarp()" + ] + }, + { + "cell_type": "markdown", + "id": "b5e69ffa", + "metadata": {}, + "source": [ + "#### 4. Step:\n", + "To adjust scaling, position and orientation of the corrected momentum space image, you can apply further affine transformations to the distortion correction field. Here, first a postential scaling is applied, next a translation, and finally a rotation around the center of the image (defined via the config). One can either use an interactive tool, or provide the adjusted values and apply them directly." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "62abfa41", + "metadata": {}, + "outputs": [], + "source": [ + "#sp.pose_adjustment(xtrans=14, ytrans=18, angle=2)\n", + "sp.pose_adjustment(xtrans=8, ytrans=7, angle=-4, apply=True)" + ] + }, + { + "cell_type": "markdown", + "id": "a78a68e9", + "metadata": {}, + "source": [ + "#### 5. Step:\n", + "Finally, the momentum correction is applied to the dataframe, and corresponding meta data are stored" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "845f002d", + "metadata": {}, + "outputs": [], + "source": [ + "sp.apply_momentum_correction()" + ] + }, + { + "cell_type": "markdown", + "id": "d9810488", + "metadata": {}, + "source": [ + "### Momentum calibration workflow\n", + "#### 1. Step:\n", + "First, the momentum scaling needs to be calibtrated. Either, one can provide the coordinates of one point outside the center, and provide its distane to the Brillouin zone center (which is assumed to be located in the center of the image), one can specify two points on the image and their distance (where the 2nd point marks the BZ center),or one can provide absolute k-coordinates of two distinct momentum points.\n", + "\n", + "If no points are provided, an interactive tool is created. Here, left mouse click selectes the off-center point (brillouin_zone_cetnered=True) or toggle-selects the off-center and center point." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a358f07d", + "metadata": {}, + "outputs": [], + "source": [ + "k_distance = 2/np.sqrt(3)*np.pi/3.28 # k-distance of the K-point in a hexagonal Brilloiun zone\n", + "#sp.calibrate_momentum_axes(k_distance = k_distance)\n", + "point_a = [308, 345]\n", + "sp.calibrate_momentum_axes(point_a=point_a, k_distance = k_distance, apply=True)\n", + "#point_b = [247, 249]\n", + "#sp.calibrate_momentum_axes(point_a=point_a, point_b = point_b, k_coord_a = [.5, 1.1], k_coord_b = [1.3, 0], equiscale=False" + ] + }, + { + "cell_type": "markdown", + "id": "1a3697b1", + "metadata": {}, + "source": [ + "##### Optional (Step 1a): \n", + "Save momentum calibration parameters to configuration file in current data folder: " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "86bedfa7", + "metadata": {}, + "outputs": [], + "source": [ + "# Save generated momentum calibration parameters for later reuse\n", + "sp.save_momentum_calibration()" + ] + }, + { + "cell_type": "markdown", + "id": "c2f8a513", + "metadata": {}, + "source": [ + "#### 2. Step:\n", + "Now, the distortion correction and momentum calibration needs to be applied to the dataframe." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f9ae5066", + "metadata": {}, + "outputs": [], + "source": [ + "sp.apply_momentum_calibration()" + ] + }, + { + "cell_type": "markdown", + "id": "0bce2388", + "metadata": {}, + "source": [ + "## Energy Correction (optional)\n", + "The purpose of the energy correction is to correct for any momentum-dependent distortion of the energy axis, e.g. from geometric effects in the flight tube, or from space charge" + ] + }, + { + "cell_type": "markdown", + "id": "5289de59", + "metadata": {}, + "source": [ + "#### 1st step:\n", + "Here, one can select the functional form to be used, and adjust its parameters. The binned data used for the momentum calibration is plotted around the Fermi energy (defined by tof_fermi), and the correction function is plotted ontop. Possible correction functions are: \"sperical\" (parameter: diameter), \"Lorentzian\" (parameter: gamma), \"Gaussian\" (parameter: sigma), and \"Lorentzian_asymmetric\" (parameters: gamma, amplitude2, gamma2).\n", + "\n", + "One can either use an interactive alignment tool, or provide parameters directly." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "72c6c8f9", + "metadata": {}, + "outputs": [], + "source": [ + "#sp.adjust_energy_correction(amplitude=2.5, center=(730, 730), gamma=920, tof_fermi = 66200)\n", + "sp.adjust_energy_correction(amplitude=2.5, center=(730, 730), gamma=920, tof_fermi = 66200, apply=True)" + ] + }, + { + "cell_type": "markdown", + "id": "e43fbf33", + "metadata": {}, + "source": [ + "##### Optional (Step 1a): \n", + "Save energy correction parameters to configuration file in current data folder: " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7699e690", + "metadata": {}, + "outputs": [], + "source": [ + "# Save generated energy correction parameters for later reuse\n", + "sp.save_energy_correction()" + ] + }, + { + "cell_type": "markdown", + "id": "41a6a3e6", + "metadata": {}, + "source": [ + "#### 2. Step\n", + "After adjustment, the energy correction is directly applied to the TOF axis." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eb1e2bee", + "metadata": {}, + "outputs": [], + "source": [ + "sp.apply_energy_correction()" + ] + }, + { + "cell_type": "markdown", + "id": "8b571b4c", + "metadata": {}, + "source": [ + "## 3. Energy calibration\n", + "For calibrating the energy axis, a set of data taken at different bias voltages around the value where the measurement was taken is required." + ] + }, + { + "cell_type": "markdown", + "id": "6bc28642", + "metadata": {}, + "source": [ + "#### 1. Step:\n", + "In a first step, the data are loaded, binned along the TOF dimension, and normalized. The used bias voltages can be either provided, or read from attributes in the source files if present." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9f44a586", + "metadata": {}, + "outputs": [], + "source": [ + "# Load energy calibration EDCs\n", + "energycalfolder = data_path + \"/energycal_2019_01_08/\"\n", + "scans = np.arange(1,12)\n", + "voltages = np.arange(12,23,1)\n", + "files = [energycalfolder + r'Scan' + str(num).zfill(3) + '_' + str(num+11) + '.h5' for num in scans]\n", + "sp.load_bias_series(data_files=files, normalize=True, biases=voltages, ranges=[(64000, 75000)])" + ] + }, + { + "cell_type": "markdown", + "id": "314a79c8", + "metadata": {}, + "source": [ + "#### 2. Step:\n", + "Next, the same peak or feature needs to be selected in each curve. For this, one needs to define \"ranges\" for each curve, within which the peak of interest is located. One can either provide these ranges manually, or provide one range for a \"reference\" curve, and infer the ranges for the other curves using a dynamic time warping algorithm." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2f843244", + "metadata": {}, + "outputs": [], + "source": [ + "# Option 1 = specify the ranges containing a common feature (e.g an equivalent peak) for all bias scans\n", + "# rg = [(129031.03103103103, 129621.62162162163), (129541.54154154155, 130142.14214214214), (130062.06206206206, 130662.66266266267), (130612.61261261262, 131213.21321321322), (131203.20320320321, 131803.8038038038), (131793.7937937938, 132384.38438438438), (132434.43443443443, 133045.04504504506), (133105.10510510512, 133715.71571571572), (133805.8058058058, 134436.43643643643), (134546.54654654654, 135197.1971971972)]\n", + "# sp.find_bias_peaks(ranges=rg, infer_others=False)\n", + "# Option 2 = specify the range for one curve and infer the others\n", + "# This will open an interactive tool to select the correct ranges for the curves.\n", + "# IMPORTANT: Don't choose the range too narrow about a peak, and choose a refid\n", + "# somewhere in the middle or towards larger biases!\n", + "rg = (66100, 67000)\n", + "sp.find_bias_peaks(ranges=rg, ref_id=5, infer_others=True, apply=True)" + ] + }, + { + "cell_type": "markdown", + "id": "b2638818", + "metadata": {}, + "source": [ + "#### 3. Step:\n", + "Next, the detected peak positions and bias voltages are used to determine the calibration function. This can be either done by fitting the functional form d^2/(t-t0)^2 via lmfit (\"lmfit\"), or using a polynomial approxiamtion (\"lstsq\" or \"lsqr\"). Here, one can also define a reference id, and a reference energy. Those define the absolute energy position of the feature used for calibration in the \"reference\" trace, at the bias voltage where the final measurement has been performed. The energy scale can be either \"kientic\" (decreasing energy with increasing TOF), or \"binding\" (increasing energy with increasing TOF).\n", + "\n", + "After calculating the calibration, all traces corrected with the calibration are plotted ontop of each other, the calibration function together with the extracted features is plotted." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "22e15f5d", + "metadata": {}, + "outputs": [], + "source": [ + "# use the refid of the bias that the measurement was taken at\n", + "# Eref can be used to set the absolute energy (kinetic energy, E-EF) of the feature used for energy calibration (if known)\n", + "refid=4\n", + "Eref=-0.5\n", + "# the lmfit method uses a fit of (d/(t-t0))**2 to determine the energy calibration\n", + "sp.calibrate_energy_axis(ref_energy=Eref, ref_id=refid, energy_scale=\"kinetic\", method=\"lmfit\")" + ] + }, + { + "cell_type": "markdown", + "id": "df63c6c7", + "metadata": {}, + "source": [ + "##### Optional (Step 3a): \n", + "Save energy calibration parameters to configuration file in current data folder: " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b870293c", + "metadata": {}, + "outputs": [], + "source": [ + "# Save generated energy calibration parameters for later reuse\n", + "sp.save_energy_calibration()" + ] + }, + { + "cell_type": "markdown", + "id": "563709c7", + "metadata": {}, + "source": [ + "#### 4. Step:\n", + "Finally, the the energy axis is added to the dataframe." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c470ffd9", + "metadata": {}, + "outputs": [], + "source": [ + "sp.append_energy_axis()" + ] + }, + { + "cell_type": "markdown", + "id": "b2d8cdf9", + "metadata": {}, + "source": [ + "## 4. Delay calibration:\n", + "The delay axis is calculated from the ADC input column based on the provided delay range. ALternatively, the delay scan range can also be extracted from attributes inside a source file, if present." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0943d349", + "metadata": {}, + "outputs": [], + "source": [ + "#from pathlib import Path\n", + "#datafile = \"file.h5\"\n", + "#print(datafile)\n", + "#sp.calibrate_delay_axis(datafile=datafile)\n", + "delay_range = (-500, 1500)\n", + "sp.calibrate_delay_axis(delay_range=delay_range, preview=True)" + ] + }, + { + "cell_type": "markdown", + "id": "d9d0b018", + "metadata": {}, + "source": [ + "## 5. Visualization of calibrated histograms\n", + "With all calibrated axes present in the dataframe, we can visualize the corresponding histograms, and determine the respective binning ranges" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c330da64", + "metadata": {}, + "outputs": [], + "source": [ + "axes = ['kx', 'ky', 'energy', 'delay']\n", + "ranges = [[-3, 3], [-3, 3], [-6, 2], [-600, 1600]]\n", + "sp.view_event_histogram(dfpid=1, axes=axes, ranges=ranges)" + ] + }, + { + "cell_type": "markdown", + "id": "6902fd56-1456-4da6-83a4-0f3f6b831eb6", + "metadata": {}, + "source": [ + "## Define the binning ranges and compute calibrated data volume" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a7601cd7-cd51-40a9-8fc7-8b7d32ff15d0", + "metadata": {}, + "outputs": [], + "source": [ + "axes = ['kx', 'ky', 'energy', 'delay']\n", + "bins = [100, 100, 200, 50]\n", + "ranges = [[-2, 2], [-2, 2], [-4, 2], [-600, 1600]]\n", + "res = sp.compute(bins=bins, axes=axes, ranges=ranges, normalize_to_acquisition_time=\"delay\")" + ] + }, + { + "cell_type": "markdown", + "id": "523794dc", + "metadata": {}, + "source": [ + "## Some visualization:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "99d7d136-b677-4c16-bc8f-31ba8216579c", + "metadata": {}, + "outputs": [], + "source": [ + "fig, axs = plt.subplots(4, 1, figsize=(6, 18), constrained_layout=True)\n", + "res.loc[{'energy':slice(-.1, 0)}].sum(axis=(2,3)).T.plot(ax=axs[0])\n", + "res.loc[{'kx':slice(-.8, -.5)}].sum(axis=(0,3)).T.plot(ax=axs[1])\n", + "res.loc[{'ky':slice(-.2, .2)}].sum(axis=(1,3)).T.plot(ax=axs[2])\n", + "res.loc[{'kx':slice(-.8, -.5), 'energy':slice(.5, 2)}].sum(axis=(0,1)).plot(ax=axs[3])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "596a3217", + "metadata": {}, + "outputs": [], + "source": [ + "# save to NXmpes NeXus (including standardized metadata)\n", + "sp.save(\"WSe2.nxs\")" + ] + }, + { + "cell_type": "markdown", + "id": "185bb6c9-cfea-4987-8ac1-8d2884dd11f3", + "metadata": {}, + "source": [ + "## View the data with H5Web\n", + "H5Web is a tool for visualizing any data in the h5 data format. Since the NeXus format builds opon h5 it can be used to view this data as well. We just import the package and call H5Web with the output filename from the convert command above. For an analysis on NeXus data files please refer to [analysis example](./E3%20pyARPES%20analysis.ipynb).\n", + "\n", + "You can also view this data with the H5Viewer or other tools from your local filesystem." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "35934a0f-b7bd-4fd4-bcda-054cc2f034f3", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "from jupyterlab_h5web import H5Web" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ccc16e59-4c61-4530-87b7-786e60c990b4", + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "H5Web('WSe2.nxs')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e0940aee-f887-4b60-b70d-23f231bede20", + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "interpreter": { + "hash": "728003ee06929e5fa5ff815d1b96bf487266025e4b7440930c6bf4536d02d243" + }, + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.12" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "state": { + "undefined": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "2.0.0", + "model_name": "VBoxModel", + "state": { + "_view_name": "ErrorWidgetView", + "error": {}, + "msg": "Model class 'VBoxModel' from module '@jupyter-widgets/controls' is loaded but can not be instantiated" + } + } + }, + "version_major": 2, + "version_minor": 0 + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/src/pynxtools_mpes/nomad/examples/E2 Binning of WSe2.ipynb b/src/pynxtools_mpes/nomad/examples/E2 Binning of WSe2.ipynb deleted file mode 100644 index eed9baf..0000000 --- a/src/pynxtools_mpes/nomad/examples/E2 Binning of WSe2.ipynb +++ /dev/null @@ -1,711 +0,0 @@ -{ - "cells": [ - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Binning raw Multidimensional Photoemission Spectroscopy (MPES) data and converting it into the NeXus format" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This example shows how to generate xarray based h5 files from WSe2 trARPES measurement data as detailed in this [paper](https://www.nature.com/articles/s41597-020-00769-8) and how to generate a file in the standardised [MPES NeXus format](https://manual.nexusformat.org/classes/contributed_definitions/NXmpes.html#nxmpes) from it.\n", - "Due to the size of the example file (~6GB) you need at least 40 GB of memory on your computer you're executing this example on. If you just want to have a look on how to convert a pre-binned xarray based h5 file into the NeXus format you may have a look at the simpler [Convert to NeXus example](./E1%20Convert%20to%20NeXus.ipynb), which has lower hardware requirements." - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Download RAW data (trARPES data of WSe2)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "Here, we just set the main file folder for holding the measurement data." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import os\n", - "FDIR = f'{os.getcwd()}/Scan049_1'\n", - "ECAL = f'{os.getcwd()}/energycal_2019_01_08'" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Since the provided measurement files are rather large (~6GB), they are not directly provided with the example.\n", - "You can [download](https://zenodo.org/record/6369728/files/WSe2.zip) it from zenodo. This may take some time. Place the file in the directory of this notebook afterwards. Under Linux, macOS and in a NORTH container you can directly use the cell below to download the file with curl." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "! curl -o WSe2.zip \"https://zenodo.org/records/6369728/files/WSe2.zip\"" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we extract the measurement files." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "! unzip WSe2.zip" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Binning of measurement data" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "First we import the necessary packages. For a manual on how to install this dependencies refer to the provided [INSTALL.md](./INSTALL.md) file. If you're running a pre-built docker container or working with the NORTH tools, these dependencies are already available for you." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from mpes import base as base, fprocessing as fp, analysis as aly\n", - "import matplotlib.pyplot as plt\n", - "import numpy as np\n", - "import os\n", - "from dask import compute\n", - "import datetime as dt\n", - "import h5py" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Initial data binning for distortion correction" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "parp = fp.parallelHDF5Processor(folder=FDIR)\n", - "parp.gather(identifier=r'/*.h5', file_sorting=True)\n", - "len(parp.files)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Bin a small range of of files to create a momentum map for distortion correction\n", - "parp.files = parp.files[0:50]\n", - "axes = ['X', 'Y', 't']\n", - "# Important to keep the whole detector area for the initial binning!\n", - "bins = [512, 512, 300]\n", - "ranges = [(0, 2048), (0, 2048), (64000, 68000)]\n", - "parp.parallelBinning(axes=axes, nbins=bins, ranges=ranges, scheduler='threads', ret=False)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Determine correction landmarks" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Select an energy slice at the valence band maximum, containing the 6 K-points and the Gamma point as distinct features\n", - "mc = aly.MomentumCorrector(parp.combinedresult['binned'])\n", - "mc.selectSlice2D(slice(165, 175), 2)\n", - "# Extract these high-symmetry points \n", - "mc.featureExtract(mc.slice, sigma=5, fwhm=10, sigma_radius=3)\n", - "mc.view(points=mc.features, annotated=True)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Calculate thin plate spline symmetry correction" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Calculate a non-linear coordinate transformation based on thin plate splines that restores 6-fold symmetry\n", - "mc.splineWarpEstimate(image=mc.slice, landmarks=mc.pouter_ord, include_center=True,\n", - " iterative=False, interp_order=2, update=True)\n", - "mc.view(image=mc.slice_transformed, annotated=True, points={'feats':mc.ptargs}, backend='bokeh', crosshair=True, radii=[75,110,150], crosshair_thickness=0.2)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Image registration" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Apply a coordinate translation to move the image into the center of the detector\n", - "mc.coordinateTransform(type='translation', xtrans=70., ytrans=70., keep=True)\n", - "plt.imshow(mc.slice_transformed, origin='lower', cmap='terrain_r')\n", - "plt.axvline(x=256)\n", - "plt.axhline(y=256)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Rotate the image into a high-symmetry direction\n", - "mc.coordinateTransform( type='rotation', angle=-5, center=(256., 256.), keep=True)\n", - "plt.imshow(mc.slice_transformed, origin='lower', cmap='terrain_r')\n", - "plt.axvline(x=256)\n", - "plt.axhline(y=256)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Display the final deformation field\n", - "subs = 20\n", - "plt.scatter(mc.cdeform_field[::subs,::subs].ravel(), mc.rdeform_field[::subs,::subs].ravel(), c='b')" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Momentum calibration" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Pick one high-symmetry point\n", - "point_b = [252.,255.]\n", - "# Pick the BZ center\n", - "point_a = [308.,346.]\n", - "# give the distance of the two in inverse Angstrom\n", - "distance = np.pi*4/3/3.297\n", - "# Momentum calibration assuming equal scaling along both x and y directions (equiscale=True)\n", - "# Requirements : pixel coordinates of and the momentum space distance between two symmetry points, \n", - "# plus the momentum coordinates\n", - "# of one of the two points \n", - "ext = mc.calibrate(mc.slice_transformed,\n", - " point_from=point_a,\n", - " point_to=point_b,\n", - " dist=distance,\n", - " equiscale=True,\n", - " ret=['extent'])" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Display corrected image in calibrated coordinates\n", - "mc.view(image=mc.slice_transformed, imkwds=ext)\n", - "plt.xlabel('$k_x$', fontsize=15)\n", - "plt.ylabel('$k_y$', fontsize=15)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Energy calibration" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Bin traces for energy calibration\n", - "axes = ['t']\n", - "bins = [1000]\n", - "ranges = [(63000, 80000)]\n", - "traces, tof = fp.extractEDC(folder=ECAL,\n", - " axes=axes, bins=bins, ranges=ranges)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Applied bias voltages (negated, in order to achieve negative binding energies, E-E_F)\n", - "voltages = np.arange(-12.2, -23.2, -1)\n", - "ec = aly.EnergyCalibrator(biases=voltages, traces=traces, tof=tof)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Normalize traces to maximum\n", - "ec.normalize(smooth=True, span=7, order=1)\n", - "ec.view(traces=ec.traces_normed, xaxis=ec.tof, backend='bokeh')" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Define a TOF feature range, and translate it for each of the traces according to their shift in bias voltage\n", - "rg = [(65000, 65200)]\n", - "ec.addFeatures(traces=ec.traces_normed, refid=0, ranges=rg[0], infer_others=True, mode='append')\n", - "ec.featranges" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# Extract the first peak from each feature range\n", - "ec.featureExtract(traces=ec.traces_normed, ranges=ec.featranges)\n", - "ec.view(traces=ec.traces_normed, peaks=ec.peaks, backend='bokeh')" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Calculate energy calibration" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# calculate the energy calibration (3rd order polynom). Eref corresponds to the binding energy (E-E_F) of the selected feature in the refid trace.\n", - "refid=5\n", - "Eref=-1.3\n", - "axs = ec.calibrate(ret='all', Eref=Eref, t=ec.tof, refid=refid)\n", - "ec.view(traces=ec.traces_normed, xaxis=ec.calibration['axis'], backend='bokeh')" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Quality of calibration" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# inspect the quality of the energy calibration\n", - "for i in range(0,len(voltages)):\n", - " plt.plot(ec.calibration['axis']-(voltages[i]-voltages[refid]), ec.traces_normed[i])\n", - "plt.xlim([-15,5])" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Inspect calibration function" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# energy calibration function vs. TOF\n", - "ec.view(traces=ec.calibration['axis'][None,:], xaxis=ec.tof, backend='matplotlib', show_legend=False)\n", - "plt.scatter(ec.peaks[:,0], ec.biases-ec.biases[refid]+Eref, s=50, c='k')\n", - "plt.xlabel('Time-of-flight', fontsize=15)\n", - "plt.ylabel('Energy (eV)', fontsize=15)\n", - "plt.ylim([-8,6])\n", - "plt.xlim([63400,69800])" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Dataframe processor" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# create the dask data frame processor\n", - "dfp = fp.dataframeProcessor(datafolder=FDIR)\n", - "dfp.read(source='folder', ftype='h5', timeStamps=True)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Apply energy calibration" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# apply the energy calibration\n", - "dfp.appendEAxis(E0=ec.calibration['E0'], a=ec.calibration['coeffs'])\n", - "dfp.edf.head(8)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Apply distortion correction" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# apply the distortion correction\n", - "dfp.applyKCorrection(type='tps_matrix',\n", - " rdeform_field = mc.rdeform_field,\n", - " cdeform_field = mc.cdeform_field,\n", - " X='X', Y='Y', newX='Xm', newY='Ym')\n", - "dfp.edf.head(8)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Apply momentum calibration" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# apply the momentum calibration\n", - "dfp.appendKAxis(point_b[0], point_b[1], X='Xm', Y='Ym', rstart=parp.binranges[0][0],\n", - " cstart=parp.binranges[1][0],\n", - " rstep=parp.binsteps[0],\n", - " cstep=parp.binsteps[1],\n", - " fc=mc.calibration['coeffs'][0],\n", - " fr=mc.calibration['coeffs'][1])\n", - "dfp.edf.head(8)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Apply pump-probe delay axis conversion" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# calculate the pump-probe delay from the ADC coordinates\n", - "ADCRange = (650, 6900)\n", - "timeRange = (-100, 200)\n", - "dfp.edf['delay'] = timeRange[0] + (dfp.edf['ADC']-ADCRange[0]) *\\\n", - " (timeRange[1] - timeRange[0])/(ADCRange[1]-ADCRange[0])\n", - "dfp.edf.head(8)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Bin 4D data in transformed grid" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "# process the 4-dimensional binning\n", - "axes = ['kx', 'ky', 'E', 'delay']\n", - "bins = [50, 50, 100, 21]\n", - "ranges = [(-2, 2), (-2, 2), (-3, 2), (-110, 190)]\n", - "# jittering of energy and ADC should best be done on the bin size of the hardware, \n", - "# not the rebinned bin size. This requires reverse-calculating the jitter amplitudes\n", - "# from the bin sizes.\n", - "TOFrange=[64500,67000]\n", - "e_t_conversion = (base.tof2evpoly(ec.calibration['coeffs'],\n", - " ec.calibration['E0'], \n", - " TOFrange[0])\n", - " - base.tof2evpoly(ec.calibration['coeffs'],\n", - " ec.calibration['E0'], TOFrange[1])\n", - " ) / (TOFrange[1] - TOFrange[0])\n", - "d_adc_conversion = (timeRange[1] - timeRange[0]) / (ADCRange[1] - ADCRange[0])\n", - "jitter_amplitude = [0.5,\n", - " 0.5,\n", - " 1*bins[2]/abs(ranges[2][1]-ranges[2][0])*e_t_conversion,\n", - " 1*bins[3]/abs(ranges[3][1]-ranges[3][0])*d_adc_conversion]\n", - "dfp.distributedBinning(axes=axes,\n", - " nbins=bins,\n", - " ranges=ranges,\n", - " scheduler='threads',\n", - " ret=False,\n", - " jittered=True,\n", - " jitter_amplitude=jitter_amplitude)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Run the following cell to store metadata from EPICS archive only if outside the FHI network\n", - "This adds additional metadata to the xarray. This data may also be provided through additional ELN entries through a NOMAD instance or with a handwritten file directly to the mpes parser." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "metadata = {\"file\": {}}\n", - "metadata['file'][\"KTOF:Lens:Extr:I\"] = -0.12877\n", - "metadata['file'][\"KTOF:Lens:UDLD:V\"] = 399.99905\n", - "metadata['file'][\"KTOF:Lens:Sample:V\"] = 17.19976\n", - "metadata['file'][\"KTOF:Apertures:m1.RBV\"] = 3.729931\n", - "metadata['file'][\"KTOF:Apertures:m2.RBV\"] = -5.200078\n", - "metadata['file'][\"KTOF:Apertures:m3.RBV\"] = -11.000425\n", - "\n", - "# Sample motor positions\n", - "metadata['file']['trARPES:Carving:TRX.RBV'] = 7.1900000000000004\n", - "metadata['file']['trARPES:Carving:TRY.RBV'] = -6.1700200225439552\n", - "metadata['file']['trARPES:Carving:TRZ.RBV'] = 33.4501953125\n", - "metadata['file']['trARPES:Carving:THT.RBV'] = 423.30500940561586\n", - "metadata['file']['trARPES:Carving:PHI.RBV'] = 0.99931647456264949\n", - "metadata['file']['trARPES:Carving:OMG.RBV'] = 11.002500171914066" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Generate xarray\n", - "Remember to remove the optional argument, metadata_dict, from the gather_metadata() function if the previous cell was not run.\n", - "The missing archive metadata warnings are not critical for this example and can thus be ignored." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import copy\n", - "res_xarray = dfp.gather_metadata(metadata_dict=copy.deepcopy(metadata), ec=ec, mc=mc)" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## Create a NeXus file from a xarray\n", - "This conversion basically follows the same procedure as in the [convert to NeXus example](./E1%20Convert%20to%20Nexus.ipynb). Please refer to this notebook for details on the convert function. Here, we are using the objects keywords of `convert` to pass the generated xarray directly, instead of loading a h5 datafile." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from pynxtools.dataconverter.convert import convert" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "convert(input_file=[\"config_file.json\", \"WSe2_eln.yaml\"],\n", - " objects=res_xarray,\n", - " reader='mpes',\n", - " nxdl='NXmpes',\n", - " output='WSe2.mpes.nxs')" - ] - }, - { - "attachments": {}, - "cell_type": "markdown", - "metadata": { - "tags": [] - }, - "source": [ - "## View the data with H5Web\n", - "H5Web is a tool for visualizing any data in the h5 data format. Since the NeXus format builds opon h5 it can be used to view this data as well. We just import the package and call H5Web with the output filename from the convert command above. For an analysis on NeXus data files please refer to [analysis example](./E3%20pyARPES%20analysis.ipynb).\n", - "\n", - "You can also view this data with the H5Viewer or other tools from your local filesystem." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from jupyterlab_h5web import H5Web" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "H5Web('WSe2.mpes.nxs')" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.4" - }, - "vscode": { - "interpreter": { - "hash": "c2ecc4d45d4efcd07af778d75fd26bf86d0642a6646ea5c57f06d5857684e419" - } - }, - "widgets": { - "application/vnd.jupyter.widget-state+json": { - "state": {}, - "version_major": 2, - "version_minor": 0 - } - } - }, - "nbformat": 4, - "nbformat_minor": 4 -} diff --git a/src/pynxtools_mpes/nomad/examples/E3 pyARPES analysis.ipynb b/src/pynxtools_mpes/nomad/examples/E3 pyARPES analysis.ipynb index 28bd5b6..8ef1889 100644 --- a/src/pynxtools_mpes/nomad/examples/E3 pyARPES analysis.ipynb +++ b/src/pynxtools_mpes/nomad/examples/E3 pyARPES analysis.ipynb @@ -6,7 +6,7 @@ "metadata": {}, "source": [ "# Analysing Multidimensional Photoemission Spectroscopy (MPES) data with pyARPES\n", - "This example shows how to analyse data in the NeXus format with the [pyARPES](https://github.com/chstan/arpes) python package. You'll find details on how to generate such NeXus files in the [convert](./E1%20Convert%20to%20NeXus.ipynb) or [binning](./E2%20Binning%20of%20WSe2.ipynb) example." + "This example shows how to analyse data in the NeXus format with the [pyARPES](https://github.com/chstan/arpes) python package. You'll find details on how to generate such NeXus files in the [convert](./E1%20Convert%20to%20NeXus.ipynb) or [postprocessing](./E2%20ARPES%20postprocessing.ipynb) example." ] }, { @@ -17,12 +17,8 @@ "outputs": [], "source": [ "from arpes.plotting.qt_tool import qt_tool\n", - "from arpes.plotting.basic_tools import path_tool\n", - "from arpes.io import example_data, load_data\n", - "from arpes.endstations.plugin.nexus import NeXusEndstation\n", "import xarray as xr\n", "import numpy as np\n", - "from pathlib import Path\n", "import h5py as h5" ] }, @@ -37,7 +33,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 3, "id": "e9658e8f-21e3-445e-b04f-c8657f707fb6", "metadata": {}, "outputs": [], @@ -77,7 +73,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 4, "id": "3b259695-1f73-4bd1-a1bc-4b5abd0432a6", "metadata": {}, "outputs": [], @@ -186,7 +182,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 6, "id": "dda4bc9a-9729-4bd3-9456-a174427e7a3c", "metadata": {}, "outputs": [], @@ -222,7 +218,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -236,12 +232,19 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.10.4" + "version": "3.8.12" }, "vscode": { "interpreter": { "hash": "c2ecc4d45d4efcd07af778d75fd26bf86d0642a6646ea5c57f06d5857684e419" } + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "state": {}, + "version_major": 2, + "version_minor": 0 + } } }, "nbformat": 4, diff --git a/src/pynxtools_mpes/nomad/examples/E4 Convert to k-space.ipynb b/src/pynxtools_mpes/nomad/examples/E4 Convert to k-space.ipynb new file mode 100644 index 0000000..7c607c0 --- /dev/null +++ b/src/pynxtools_mpes/nomad/examples/E4 Convert to k-space.ipynb @@ -0,0 +1,278 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Convert measurement angles to k-space\n", + "\n", + "In this example, we will use the NeXus format to automatically detect and read the angles from an ARPES measurement and do a k-space conversion with pyArpes." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "import warnings\n", + "warnings.filterwarnings(\"ignore\")\n", + "import numpy as np\n", + "\n", + "from arpes.io import load_data\n", + "from arpes.endstations.plugin.nexus import NeXusEndstation\n", + "from arpes.utilities.conversion import convert_to_kspace\n", + "\n", + "\n", + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Download data\n", + "\n", + "First we need to automatically download the data from [Nomad](https://nomad-lab.eu/prod/v1/api/v1/entries/xV9yMspIyXMfia6nNw3NIbnlZ91H/raw/Scan1496.nxs)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "! curl -o Scan1496.nxs https://nomad-lab.eu/prod/v1/api/v1/entries/xV9yMspIyXMfia6nNw3NIbnlZ91H/raw/Scan1496.nxs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Load data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "Scan1496 = load_data(\"Scan1496.nxs\", location=NeXusEndstation)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Apply rotational offset from experimental geometry" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "Scan1496.S.apply_offsets({\n", + " \"chi\": 19/180*np.pi,\n", + "})" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Subtract photon energy" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "Scan1496_shifted = Scan1496\n", + "Scan1496_shifted['eV'] = Scan1496_shifted['eV'] - Scan1496.attrs[\"hv\"].magnitude" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We read all metadata from the nexus file into xarray attributes for easy programmatic access." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "Scan1496_shifted.attrs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can easily readout interesting values including their units, e.g., the incident photon energy of the beam:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "Scan1496_shifted.attrs['instrument/beam_probe/incident_energy']" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Generate Fermi surface" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "fsmap = Scan1496_shifted.S.generic_fermi_surface(0)\n", + "fsmap.S.plot()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Convert into k-space" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "fsmap_converted = convert_to_kspace(\n", + " fsmap,\n", + " kx=np.linspace(-.6, .6, 400),\n", + " ky=np.linspace(-1.2, 1.2, 400),\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "fsmap_converted.T.plot()" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "Scan1496_converted = convert_to_kspace(\n", + " Scan1496,\n", + " kx=np.linspace(-.6, .6, 400),\n", + " ky=np.linspace(-1.2, 1.2, 400),\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "Scan1496_converted.S.generic_fermi_surface(0).T.plot()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "Scan1496.loc[{\"eV\":slice(1, 1.2)}].sum(dim=\"eV\").S.plot()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "tags": [] + }, + "outputs": [], + "source": [ + "Scan1496_converted.loc[{\"eV\":slice(1, 1.2)}].sum(dim=\"eV\").S.plot()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.8.12" + }, + "vscode": { + "interpreter": { + "hash": "a164666994e9db75450cd7016dd7e51d42ea6e7c1e5e8017af1f8068ca906367" + } + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "state": {}, + "version_major": 2, + "version_minor": 0 + } + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/src/pynxtools_mpes/nomad/examples/config_file.json b/src/pynxtools_mpes/nomad/examples/config_file.json old mode 100644 new mode 100755 index 94a8a4c..226c89d --- a/src/pynxtools_mpes/nomad/examples/config_file.json +++ b/src/pynxtools_mpes/nomad/examples/config_file.json @@ -1,31 +1,31 @@ { "/@default": "entry", - "/ENTRY[entry]/@default": "data", - "/ENTRY[entry]/definition": "NXmpes", - "/ENTRY[entry]/definition/@version": "None", - "/ENTRY[entry]/title": "@attrs:metadata/entry_title", - "/ENTRY[entry]/start_time": "@attrs:metadata/timing/acquisition_start", - "/ENTRY[entry]/experiment_institution": "Fritz Haber Institute - Max Planck Society", - "/ENTRY[entry]/experiment_facility": "Time Resolved ARPES", - "/ENTRY[entry]/experiment_laboratory": "Clean Room 4", - "/ENTRY[entry]/entry_identifier": "@attrs:metadata/entry_identifier", - "/ENTRY[entry]/end_time": "@attrs:metadata/timing/acquisition_stop", - "/ENTRY[entry]/duration": "@attrs:metadata/timing/acquisition_duration", - "/ENTRY[entry]/duration/@units": "s", - "/ENTRY[entry]/collection_time": "@attrs:metadata/timing/collection_time", - "/ENTRY[entry]/collection_time/@units": "s", - "/ENTRY[entry]/USER[user]": { - "name": "@attrs:metadata/user0/name", - "role": "@attrs:metadata/user0/role", - "affiliation": "@attrs:metadata/user0/affiliation", - "address": "@attrs:metadata/user0/address", - "email": "@attrs:metadata/user0/email" + "/ENTRY/@default": "data", + "/ENTRY/title": "['@eln:/ENTRY/title', '@attrs:metadata/entry_title']", + "/ENTRY/start_time": "@attrs:metadata/timing/acquisition_start", + "/ENTRY/experiment_institution": "Fritz Haber Institute - Max Planck Society", + "/ENTRY/experiment_facility": "Time Resolved ARPES", + "/ENTRY/experiment_laboratory": "Clean Room 4", + "/ENTRY/entry_identifier": { + "identifier":"@attrs:metadata/entry_identifier" }, - "/ENTRY[entry]/INSTRUMENT[instrument]": { + "/ENTRY/end_time": "@attrs:metadata/timing/acquisition_stop", + "/ENTRY/duration": "@attrs:metadata/timing/acquisition_duration", + "/ENTRY/duration/@units": "s", + "/ENTRY/collection_time": "@attrs:metadata/timing/collection_time", + "/ENTRY/collection_time/@units": "s", + "/ENTRY/USER[user]": { + "name": "!['@eln:/ENTRY/User/name', '@attrs:metadata/user0/name']", + "role": "['@eln:/ENTRY/User/role', '@attrs:metadata/user0/role']", + "affiliation": "!['@eln:/ENTRY/User/affiliation', '@attrs:metadata/user0/affiliation']", + "address": "['@eln:/ENTRY/User/address', '@attrs:metadata/user0/address']", + "email": "['@eln:/ENTRY/User/email', '@attrs:metadata/user0/email']" + }, + "/ENTRY/INSTRUMENT[instrument]": { "name": "Time-of-flight momentum microscope equipped delay line detector, at the endstation of the high rep-rate HHG source at FHI", "name/@short_name": "TR-ARPES @ FHI", "energy_resolution": { - "resolution": 140.0, + "resolution": "!['@eln:/ENTRY/Instrument/Analyzer/energy_resolution', '@attrs:metadata/instrument/energy_resolution', '140']", "resolution/@units": "meV", "physical_quantity": "energy", "type": "estimated" @@ -36,16 +36,11 @@ "physical_quantity": "time", "type": "estimated" }, - "RESOLUTION[momentum_resolution]": { - "resolution": "@attrs:metadata/instrument/analyzer/momentum_resolution", - "resolution/@units": "1/angstrom", - "physical_quantity": "momentum", - "type": "estimated" - }, + "RESOLUTION[momentum_resolution]": "@link:/entry/instrument/electronanalyser/momentum_resolution", "pressure_gauge": { "name": "sample_chamber_pressure", "measurement": "pressure", - "value": "@attrs:metadata/file/trARPES:XGS600:PressureAC:P_RD", + "value": "!['@eln:/ENTRY/Sample/gas_pressure', '@attrs:metadata/file/trARPES:XGS600:PressureAC:P_RD']", "value/@units": "mbar" }, "ELECTRONANALYSER[electronanalyser]": { @@ -54,26 +49,22 @@ "vendor": "SPECS GmbH", "model": "Metis 1000 Momentum Microscope" }, - "fast_axes": [ - "kx", - "ky", - "E" - ], + "fast_axes": ["kx", "ky", "E"], "slow_axes": "@attrs:metadata/instrument/analyzer/slow_axes", "energy_resolution": { - "resolution": 110.0, + "resolution": "!['@eln:/ENTRY/Instrument/Analyzer/energy_resolution', '@attrs:metadata/instrument/analyzer/energy_resolution']", "resolution/@units": "meV", "physical_quantity": "energy", "type": "estimated" }, "momentum_resolution": { - "resolution": "@attrs:metadata/instrument/analyzer/momentum_resolution", + "resolution": "!['@eln:/ENTRY/Instrument/Analyzer/momentum_resolution', '@attrs:metadata/instrument/analyzer/momentum_resolution']", "resolution/@units": "1/angstrom", "physical_quantity": "momentum", "type": "estimated" }, "spatial_resolution": { - "resolution": "@attrs:metadata/instrument/analyzer/spatial_resolution", + "resolution": "!['@eln:/ENTRY/Instrument/Analyzer/spatial_resolution', '@attrs:metadata/instrument/analyzer/spatial_resolution']", "resolution/@units": "µm", "physical_quantity": "length", "type": "estimated" @@ -84,197 +75,175 @@ "AXISNAME[trans_z]/@depends_on": "rot_y", "AXISNAME[trans_z]/@transformation_type": "translation", "AXISNAME[trans_z]/@units": "mm", - "AXISNAME[trans_z]/@vector": [ - 0, - 0, - 1 - ], + "AXISNAME[trans_z]/@vector": [0, 0, 1], "AXISNAME[rot_y]": -115.0, "AXISNAME[rot_y]/@depends_on": ".", "AXISNAME[rot_y]/@transformation_type": "rotation", "AXISNAME[rot_y]/@units": "degrees", - "AXISNAME[rot_y]/@vector": [ - 0, - 1, - 0 - ] - } - } - }, - "/ENTRY[entry]/INSTRUMENT[instrument]/ELECTRONANALYSER[electronanalyser]/COLLECTIONCOLUMN[collectioncolumn]": { - "projection": "@attrs:metadata/instrument/analyzer/projection", - "scheme": "momentum dispersive", - "lens_mode": "@attrs:metadata/instrument/analyzer/lens_mode", - "extractor_voltage": "@attrs:metadata/file/KTOF:Lens:Extr:V", - "extractor_voltage/@units": "V", - "extractor_current": "@attrs:metadata/file/KTOF:Lens:Extr:I", - "extractor_current/@units": "µA", - "working_distance": 4.0, - "working_distance/@units": "mm", - "LENS_EM[lens_*{A,B,C,D,E,F,G,H,I,UCA,UFA,Foc}]": { - "name": "*", - "voltage": "@attrs:metadata/file/KTOF:Lens:*:V", - "voltage/@units": "V" - }, - "field_aperture": { - "shape": "@attrs:metadata/instrument/analyzer/fa_shape", - "size": "@attrs:metadata/instrument/analyzer/fa_size", - "size/@units": "µm", - "POSITIONER[fa_m1]": { - "value": "@attrs:metadata/file/KTOF:Apertures:m1.RBV", - "value/@units": "mm" + "AXISNAME[rot_y]/@vector": [0, 1, 0] + }, + "COLLECTIONCOLUMN[collectioncolumn]": { + "projection": "@attrs:metadata/instrument/analyzer/projection", + "scheme": "['@attrs:metadata/instrument/analyzer/scheme', 'momentum dispersive']", + "lens_mode": "@attrs:metadata/instrument/analyzer/lens_mode", + "extractor_voltage": "@attrs:metadata/file/KTOF:Lens:Extr:V", + "extractor_voltage/@units": "V", + "extractor_current": "@attrs:metadata/file/KTOF:Lens:Extr:I", + "extractor_current/@units": "µA", + "working_distance": 4.0, + "working_distance/@units": "mm", + "LENS_EM[lens_*{A,B,C,D,E,F,G,H,I,UCA,UFA,Foc}]": { + "name": "*", + "voltage": "@attrs:metadata/file/KTOF:Lens:*:V", + "voltage/@units": "V" + }, + "field_aperture": { + "shape": "@attrs:metadata/instrument/analyzer/fa_shape", + "size": "@attrs:metadata/instrument/analyzer/fa_size", + "size/@units": "µm", + "POSITIONER[fa_m1]": { + "value": "@attrs:metadata/file/KTOF:Apertures:m1.RBV", + "value/@units": "mm" + }, + "POSITIONER[fa_m2]": { + "value": "@attrs:metadata/file/KTOF:Apertures:m2.RBV", + "value/@units": "mm" + } + }, + "contrast_aperture": { + "shape": "@attrs:metadata/instrument/analyzer/ca_shape", + "size": "@attrs:metadata/instrument/analyzer/ca_size", + "size/@units": "µm", + "POSITIONER[ca_m3]": { + "value": "@attrs:metadata/file/KTOF:Apertures:m3.RBV", + "value/@units": "mm" + } + } }, - "POSITIONER[fa_m2]": { - "value": "@attrs:metadata/file/KTOF:Apertures:m2.RBV", - "value/@units": "mm" + "ENERGYDISPERSION[energydispersion]": { + "pass_energy": "@attrs:metadata/file/KTOF:Lens:TOF:V", + "pass_energy/@units": "eV", + "scheme": "tof", + "tof_distance": 0.9, + "tof_distance/@units": "m" + }, + "DETECTOR[detector]": { + "amplifier_type": "MCP", + "detector_type": "DLD", + "sensor_pixels": [1800, 1800], + "sensor_pixels/@units": "", + "amplifier_bias": "@attrs:metadata/file/KTOF:Lens:MCPfront:V", + "amplifier_bias/@units": "V", + "amplifier_voltage": "@attrs:metadata/file/KTOF:Lens:MCPback:V", + "amplifier_voltage/@units": "V", + "detector_voltage": "@attrs:metadata/file/KTOF:Lens:UDLD:V", + "detector_voltage/@units": "V" } }, - "contrast_aperture": { - "shape": "@attrs:metadata/instrument/analyzer/ca_shape", - "size": "@attrs:metadata/instrument/analyzer/ca_size", - "size/@units": "µm", - "POSITIONER[ca_m3]": { - "value": "@attrs:metadata/file/KTOF:Apertures:m3.RBV", - "value/@units": "mm" - } - } - }, - "/ENTRY[entry]/INSTRUMENT[instrument]/ELECTRONANALYSER[electronanalyser]/ENERGYDISPERSION[energydispersion]": { - "pass_energy": "@attrs:metadata/file/KTOF:Lens:TOF:V", - "pass_energy/@units": "eV", - "scheme": "tof", - "tof_distance": 0.9, - "tof_distance/@units": "m" - }, - "/ENTRY[entry]/INSTRUMENT[instrument]/ELECTRONANALYSER[electronanalyser]/DETECTOR[detector]": { - "amplifier_type": "MCP", - "detector_type": "DLD", - "sensor_pixels": [ - 1800, - 1800 - ], - "amplifier_bias": "@attrs:metadata/file/KTOF:Lens:MCPfront:V", - "amplifier_bias/@units": "V", - "amplifier_voltage": "@attrs:metadata/file/KTOF:Lens:MCPback:V", - "amplifier_voltage/@units": "V", - "detector_voltage": "@attrs:metadata/file/KTOF:Lens:UDLD:V", - "detector_voltage/@units": "V" - }, - "/ENTRY[entry]/INSTRUMENT[instrument]/source_TYPE[source_probe]": { - "name": "HHG @ TR-ARPES @ FHI", - "probe": "photon", - "type": "HHG laser", - "mode": "Single Bunch", - "frequency": "@attrs:metadata/instrument/beam/probe/frequency", - "frequency/@units": "kHz", - "associated_beam": "@link:/entry/instrument/beam_probe" - }, - "/ENTRY[entry]/INSTRUMENT[instrument]/beam_TYPE[beam_probe]": { - "distance": 0.0, - "distance/@units": "mm", - "incident_energy": "@attrs:metadata/instrument/beam/probe/incident_energy", - "incident_energy/@units": "eV", - "incident_energy_spread": "@attrs:metadata/instrument/beam/probe/incident_energy_spread", - "incident_energy_spread/@units": "eV", - "pulse_duration": "@attrs:metadata/instrument/beam/probe/pulse_duration", - "pulse_duration/@units": "fs", - "incident_polarization": "@attrs:metadata/instrument/beam/probe/incident_polarization", - "incident_polarization/@units": "V^2/mm^2", - "extent": "@attrs:metadata/instrument/beam/probe/extent", - "extent/@units": "µm", - "associated_source": "@link:/entry/instrument/source_probe" - }, - "/ENTRY[entry]/INSTRUMENT[instrument]/source_TYPE[source_pump]": { - "name": "OPCPA @ TR-ARPES @ FHI", - "probe": "visible light", - "type": "Optical Laser", - "mode": "Single Bunch", - "frequency": "@attrs:metadata/instrument/beam/pump/frequency", - "frequency/@units": "kHz", - "associated_beam": "@link:/entry/instrument/beam_pump" - }, - "/ENTRY[entry]/INSTRUMENT[instrument]/beam_TYPE[beam_pump]": { - "distance": 0.0, - "distance/@units": "mm", - "incident_energy": "@attrs:metadata/instrument/beam/pump/incident_energy", - "incident_energy/@units": "eV", - "incident_energy_spread": "@attrs:metadata/instrument/beam/pump/incident_energy_spread", - "incident_energy_spread/@units": "eV", - "incident_wavelength": "@attrs:metadata/instrument/beam/pump/incident_wavelength", - "incident_wavelength/@units": "nm", - "pulse_duration": "@attrs:metadata/instrument/beam/pump/pulse_duration", - "pulse_duration/@units": "fs", - "incident_polarization": "@attrs:metadata/instrument/beam/pump/incident_polarization", - "incident_polarization/@units": "V^2/mm^2", - "pulse_energy": "@attrs:metadata/instrument/beam/pump/pulse_energy", - "pulse_energy/@units": "µJ", - "average_power": "@attrs:metadata/instrument/beam/pump/average_power", - "average_power/@units": "mW", - "extent": "@attrs:metadata/instrument/beam/pump/extent", - "extent/@units": "µm", - "fluence": "@attrs:metadata/instrument/beam/pump/fluence", - "fluence/@units": "mJ/cm^2", - "associated_source": "@link:/entry/instrument/source_pump" - }, - "/ENTRY[entry]/INSTRUMENT[instrument]/MANIPULATOR[manipulator]": { - "temperature_sensor": { - "name": "sample_temperature", - "measurement": "temperature", - "value": "@attrs:metadata/file/trARPES:Carving:TEMP_RBV", - "value/@units": "K" + "sourceTYPE[source_probe]": { + "name": "HHG @ TR-ARPES @ FHI", + "probe": "photon", + "type": "HHG laser", + "mode": "Single Bunch", + "frequency": "['@eln:/ENTRY/Instrument/Source/Probe/frequency', '@attrs:metadata/instrument/beam/probe/frequency']", + "frequency/@units": "kHz", + "associated_beam": "/entry/instrument/beam_probe" }, - "sample_bias_voltmeter": { - "name": "sample_bias", - "measurement": "voltage", - "value": "@attrs:metadata/file/KTOF:Lens:Sample:V", - "value/@units": "V" + "beamTYPE[beam_probe]": { + "distance": 0.0, + "distance/@units": "mm", + "incident_energy": "!['@eln:/ENTRY/Instrument/Beam/Probe/incident_energy', '@attrs:metadata/instrument/beam/probe/incident_energy']", + "incident_energy/@units": "eV", + "incident_energy_spread": "['@eln:/ENTRY/Instrument/Beam/Probe/incident_energy_spread', '@attrs:metadata/instrument/beam/probe/incident_energy_spread']", + "incident_energy_spread/@units": "eV", + "pulse_duration": "['@eln:/ENTRY/Instrument/Beam/Probe/pulse_duration', '@attrs:metadata/instrument/beam/probe/pulse_duration']", + "pulse_duration/@units": "fs", + "incident_polarization": "['@eln:/ENTRY/Instrument/Beam/Probe/incident_polarization', '@attrs:metadata/instrument/beam/probe/incident_polarization']", + "incident_polarization/@units": "V^2/mm^2", + "extent": "['@eln:/ENTRY/Instrument/Beam/Probe/extent', '@attrs:metadata/instrument/beam/probe/extent']", + "extent/@units": "µm", + "associated_source": "/entry/instrument/source_probe" }, - "depends_on": "/entry/instrument/manipulator/transformations/trans_z", - "TRANSFORMATIONS[transformations]": { - "AXISNAME[trans_z]": -0.32, - "AXISNAME[trans_z]/@depends_on": "rot_z", - "AXISNAME[trans_z]/@transformation_type": "translation", - "AXISNAME[trans_z]/@units": "m", - "AXISNAME[trans_z]/@vector": [ - 0, - 0, - 1 - ], - "AXISNAME[rot_z]/@depends_on": "rot_x", - "AXISNAME[rot_z]": -25.0, - "AXISNAME[rot_z]/@transformation_type": "rotation", - "AXISNAME[rot_z]/@units": "degrees", - "AXISNAME[rot_z]/@vector": [ - 0, - 0, - 1 - ], - "AXISNAME[rot_x]/@depends_on": ".", - "AXISNAME[rot_x]": -90.0, - "AXISNAME[rot_x]/@transformation_type": "rotation", - "AXISNAME[rot_x]/@units": "degrees", - "AXISNAME[rot_x]/@vector": [ - 1, - 0, - 0 - ] + "sourceTYPE[source_pump]": { + "name": "OPCPA @ TR-ARPES @ FHI", + "probe": "visible light", + "type": "Optical Laser", + "mode": "Single Bunch", + "frequency": "['@eln:/ENTRY/Instrument/Source/Pump/frequency', '@attrs:metadata/instrument/beam/pump/frequency']", + "frequency/@units": "kHz", + "associated_beam": "/entry/instrument/beam_pump" + }, + "beamTYPE[beam_pump]": { + "distance": 0.0, + "distance/@units": "mm", + "incident_energy": "!['@eln:/ENTRY/Instrument/Beam/Pump/incident_energy', '@attrs:metadata/instrument/beam/pump/incident_energy']", + "incident_energy/@units": "eV", + "incident_energy_spread": "['@eln:/ENTRY/Instrument/Beam/Pump/incident_energy_spread', '@attrs:metadata/instrument/beam/pump/incident_energy_spread']", + "incident_energy_spread/@units": "eV", + "incident_wavelength": "['@eln:/ENTRY/Instrument/Beam/Pump/incident_wavelength', '@attrs:metadata/instrument/beam/pump/incident_wavelength']", + "incident_wavelength/@units": "nm", + "pulse_duration": "['@eln:/ENTRY/Instrument/Beam/Pump/pulse_duration', '@attrs:metadata/instrument/beam/pump/pulse_duration']", + "pulse_duration/@units": "fs", + "incident_polarization": "['@eln:/ENTRY/Instrument/Beam/Pump/incident_polarization', '@attrs:metadata/instrument/beam/pump/incident_polarization']", + "incident_polarization/@units": "V^2/mm^2", + "pulse_energy": "['@eln:/ENTRY/Instrument/Beam/Pump/pulse_energy', '@attrs:metadata/instrument/beam/pump/pulse_energy']", + "pulse_energy/@units": "µJ", + "average_power": "['@eln:/ENTRY/Instrument/Beam/Pump/average_power', '@attrs:metadata/instrument/beam/pump/average_power']", + "average_power/@units": "mW", + "extent": "['@eln:/ENTRY/Instrument/Beam/Pump/extent', '@attrs:metadata/instrument/beam/pump/extent']", + "extent/@units": "µm", + "fluence": "['@eln:/ENTRY/Instrument/Beam/Pump/fluence', '@attrs:metadata/instrument/beam/pump/fluence']", + "fluence/@units": "mJ/cm^2", + "associated_source": "/entry/instrument/source_pump" + }, + "MANIPULATOR[manipulator]": { + "temperature_sensor": { + "name": "sample_temperature", + "measurement": "temperature", + "value": "!['@eln:/ENTRY/Instrument/Manipulator/sample_temperature', '@attrs:metadata/file/trARPES:Carving:TEMP_RBV']", + "value/@units": "K" + }, + "sample_bias_voltmeter": { + "name": "sample_bias", + "measurement": "voltage", + "value": "!@attrs:metadata/file/KTOF:Lens:Sample:V", + "value/@units": "V" + }, + "depends_on": "/entry/instrument/manipulator/transformations/trans_z", + "TRANSFORMATIONS[transformations]": { + "AXISNAME[trans_z]": -0.32, + "AXISNAME[trans_z]/@depends_on": "rot_z", + "AXISNAME[trans_z]/@transformation_type": "translation", + "AXISNAME[trans_z]/@units": "m", + "AXISNAME[trans_z]/@vector": [0, 0, 1], + "AXISNAME[rot_z]/@depends_on": "rot_x", + "AXISNAME[rot_z]": -25.0, + "AXISNAME[rot_z]/@transformation_type": "rotation", + "AXISNAME[rot_z]/@units": "degrees", + "AXISNAME[rot_z]/@vector": [0, 0, 1], + "AXISNAME[rot_x]/@depends_on": ".", + "AXISNAME[rot_x]": -90.0, + "AXISNAME[rot_x]/@transformation_type": "rotation", + "AXISNAME[rot_x]/@units": "degrees", + "AXISNAME[rot_x]/@vector": [1, 0, 0] + } } }, - "/ENTRY[entry]/SAMPLE[sample]": { - "preparation_date": "@attrs:metadata/sample/preparation_date", - "sample_history/notes/type": "text/plain", - "sample_history/notes/description": "@attrs:metadata/sample/sample_history", - "description": "@attrs:metadata/sample/chemical_formula", - "name": "@attrs:metadata/sample/chemical_formula", + "/ENTRY/SAMPLE[sample]": { + "preparation_date": "['@eln:/ENTRY/Sample/preparation_date', '@attrs:metadata/sample/preparation_date']", + "history/notes/description": "['@eln:/ENTRY/Sample/sample_history', '@attrs:metadata/sample/sample_history']", + "history/notes/type": "text/plain", + "description": "['@eln:/ENTRY/Sample/description', '@attrs:metadata/sample/chemical_formula']", + "name": "['@eln:/ENTRY/Sample/name', '@attrs:metadata/sample/name']", "situation": "vacuum", - "SUBSTANCE[substance]/molecular_formula_hill": "@attrs:metadata/sample/chemical_formula", - "temperature": { + "SUBSTANCE[substance]/molecular_formula_hill": "['@eln:/ENTRY/Sample/chemical_formula', '@attrs:metadata/sample/chemical_formula']", + "temperature_env": { "temperature_sensor": "@link:/entry/instrument/manipulator/temperature_sensor" }, - "gas_pressure": { + "gas_pressure_env": { "pressure_gauge": "@link:/entry/instrument/pressure_gauge" }, - "bias": { + "bias_env": { "voltmeter": "@link:/entry/instrument/manipulator/sample_bias_voltmeter" }, "depends_on": "/entry/sample/transformations/corrected_phi", @@ -283,119 +252,106 @@ "AXISNAME[corrected_phi]": 90.0, "AXISNAME[corrected_phi]/@units": "degrees", "AXISNAME[corrected_phi]/@transformation_type": "rotation", - "AXISNAME[corrected_phi]/@vector": [ - 0, - 1, - 0 - ], + "AXISNAME[corrected_phi]/@vector": [0, 1, 0], "AXISNAME[rot_omg]/@depends_on": "rot_phi", "AXISNAME[rot_omg]": "@attrs:metadata/file/trARPES:Carving:OMG.RBV", "AXISNAME[rot_omg]/@units": "degrees", "AXISNAME[rot_omg]/@transformation_type": "rotation", - "AXISNAME[rot_omg]/@vector": [ - 1, - 0, - 0 - ], + "AXISNAME[rot_omg]/@vector": [1, 0, 0], "AXISNAME[rot_phi]/@depends_on": "rot_tht", "AXISNAME[rot_phi]": "@attrs:metadata/file/trARPES:Carving:PHI.RBV", "AXISNAME[rot_phi]/@units": "degrees", "AXISNAME[rot_phi]/@transformation_type": "rotation", - "AXISNAME[rot_phi]/@vector": [ - 0, - 1, - 0 - ], + "AXISNAME[rot_phi]/@vector": [0, 1, 0], "AXISNAME[rot_tht]/@depends_on": "trans_z", "AXISNAME[rot_tht]": "@attrs:metadata/file/trARPES:Carving:THT.RBV", "AXISNAME[rot_tht]/@units": "degrees", "AXISNAME[rot_tht]/@transformation_type": "rotation", - "AXISNAME[rot_tht]/@vector": [ - 0, - 0, - 1 - ], + "AXISNAME[rot_tht]/@vector": [0, 0, 1], "AXISNAME[trans_z]/@depends_on": "trans_y", "AXISNAME[trans_z]": "@attrs:metadata/file/trARPES:Carving:TRZ.RBV", "AXISNAME[trans_z]/@units": "mm", "AXISNAME[trans_z]/@transformation_type": "translation", - "AXISNAME[trans_z]/@vector": [ - 0, - 0, - 1 - ], + "AXISNAME[trans_z]/@vector": [0, 0, 1], "AXISNAME[trans_y]/@depends_on": "trans_x", "AXISNAME[trans_y]": "@attrs:metadata/file/trARPES:Carving:TRY.RBV", "AXISNAME[trans_y]/@units": "mm", "AXISNAME[trans_y]/@transformation_type": "translation", - "AXISNAME[trans_y]/@vector": [ - 0, - 1, - 0 - ], + "AXISNAME[trans_y]/@vector": [0, 1, 0], "AXISNAME[trans_x]/@depends_on": "/entry/instrument/manipulator/transformations/trans_z", "AXISNAME[trans_x]": "@attrs:metadata/file/trARPES:Carving:TRX.RBV", "AXISNAME[trans_x]/@units": "mm", "AXISNAME[trans_x]/@transformation_type": "translation", - "AXISNAME[trans_x]/@vector": [ - 1, - 0, - 0 - ] + "AXISNAME[trans_x]/@vector": [1, 0, 0] } }, - "/ENTRY[entry]/PROCESS[process]/DISTORTION[distortion]": { - "symmetry": "@attrs:metadata/momentum_correction/rotsym", - "original_centre": "@attrs:metadata/momentum_correction/pcent", - "original_points": "@attrs:metadata/momentum_correction/pouter", - "cdeform_field": "@attrs:metadata/momentum_correction/cdeform_field", - "rdeform_field": "@attrs:metadata/momentum_correction/rdeform_field" + "/ENTRY/PROCESS_MPES[process]/DISTORTION[distortion]": { + "symmetry": "!['@attrs:metadata/momentum_correction/correction/rotation_symmetry', '@attrs:metadata/momentum_correction/rotsym']", + "symmetry/@units": "", + "original_centre": "['@attrs:metadata/momentum_correction/correction/center_point', '@attrs:metadata/momentum_correction/pcent']", + "original_centre/@units": "", + "original_points": "['@attrs:metadata/momentum_correction/correction/outer_points', '@attrs:metadata/momentum_correction/pouter']", + "original_points/@units": "", + "cdeform_field": "['@attrs:metadata/momentum_correction/correction/cdeform_field', '@attrs:metadata/momentum_correction/cdeform_field']", + "cdeform_field/@units": "", + "rdeform_field": "['@attrs:metadata/momentum_correction/correction/rdeform_field', '@attrs:metadata/momentum_correction/rdeform_field']", + "rdeform_field/@units": "" }, - "/ENTRY[entry]/PROCESS[process]/REGISTRATION[registration]": { - "depends_on": "/entry/process/registration/tranformations/rot_z", - "TRANSFORMATIONS[tranformations]": { - "AXISNAME[trans_x]": "@attrs:metadata/momentum_correction/adjust_params/xtrans", + "/ENTRY/PROCESS_MPES[process]/REGISTRATION[registration]": { + "depends_on": "/entry/process/registration/transformations/rot_z", + "TRANSFORMATIONS[transformations]": { + "AXISNAME[trans_x]": "['@attrs:metadata/momentum_correction/registration/trans_x/value', '@attrs:metadata/momentum_correction/adjust_params/xtrans']", "AXISNAME[trans_x]/@transformation_type": "translation", "AXISNAME[trans_x]/@units": "pixels", - "AXISNAME[trans_x]/@vector": "@attrs:metadata/momentum_correction/adjust_params/x_vector", - "AXISNAME[trans_x]/@depends_on": ".", - "AXISNAME[trans_y]": "@attrs:metadata/momentum_correction/adjust_params/ytrans", + "AXISNAME[trans_x]/@vector": "['@attrs:metadata/momentum_correction/registration/trans_x/vector', '@attrs:metadata/momentum_correction/adjust_params/x_vector']", + "AXISNAME[trans_x]/@depends_on": "['@attrs:metadata/momentum_correction/registration/trans_x/depends_on', '.']", + "AXISNAME[trans_y]": "['@attrs:metadata/momentum_correction/registration/trans_y/value', '@attrs:metadata/momentum_correction/adjust_params/ytrans']", "AXISNAME[trans_y]/@units": "pixels", "AXISNAME[trans_y]/@transformation_type": "translation", - "AXISNAME[trans_y]/@vector": "@attrs:metadata/momentum_correction/adjust_params/y_vector", - "AXISNAME[trans_y]/@depends_on": "trans_x", - "AXISNAME[rot_z]": "@attrs:metadata/momentum_correction/adjust_params/angle", + "AXISNAME[trans_y]/@vector": "['@attrs:metadata/momentum_correction/registration/trans_y/vector', '@attrs:metadata/momentum_correction/adjust_params/y_vector']", + "AXISNAME[trans_y]/@depends_on": "['@attrs:metadata/momentum_correction/registration/trans_y/depends_on', 'trans_x']", + "AXISNAME[rot_z]": "['@attrs:metadata/momentum_correction/registration/rot_z/value', '@attrs:metadata/momentum_correction/adjust_params/angle']", "AXISNAME[rot_z]/@units": "degrees", "AXISNAME[rot_z]/@transformation_type": "rotation", - "AXISNAME[rot_z]/@offset": "@attrs:metadata/momentum_correction/adjust_params/offset", - "AXISNAME[rot_z]/@vector": "@attrs:metadata/momentum_correction/adjust_params/rotation_vector", - "AXISNAME[rot_z]/@depends_on": "trans_y" + "AXISNAME[rot_z]/@offset": "['@attrs:metadata/momentum_correction/registration/rot_z/offset', '@attrs:metadata/momentum_correction/adjust_params/offset']", + "AXISNAME[rot_z]/@vector": "['@attrs:metadata/momentum_correction/registration/rot_z/vector', '@attrs:metadata/momentum_correction/adjust_params/rotation_vector']", + "AXISNAME[rot_z]/@depends_on": "['@attrs:metadata/momentum_correction/registration/rot_z/depends_on', 'trans_y']" } }, - "/ENTRY[entry]/PROCESS[process]/energy_calibration":{ - "coefficients": "@attrs:metadata/energy_correction/calibration/coefficients", - "fit_function": "@attrs:metadata/energy_correction/calibration/fit_function", - "original_axis": "@attrs:metadata/energy_correction/tof", - "calibrated_axis": "@attrs:metadata/energy_correction/calibration/axis" + "/ENTRY/PROCESS_MPES[process]/energy_calibration":{ + "coefficients": "['@attrs:metadata/energy_calibration/calibration/coefficients', '@attrs:metadata/energy_correction/calibration/coefficients']", + "coefficients/@units": "", + "fit_function": "['@attrs:metadata/energy_calibration/calibration/fit_function', '@attrs:metadata/energy_correction/calibration/fit_function']", + "original_axis": "['@attrs:metadata/energy_calibration/tof', '@attrs:metadata/energy_correction/tof']", + "original_axis/@units": "", + "calibrated_axis": "['@attrs:metadata/energy_calibration/calibration/axis', '@attrs:metadata/energy_correction/calibration/axis']", + "calibrated_axis/@units": "eV", + "physical_quantity": "energy" }, - "/ENTRY[entry]/PROCESS[process]/CALIBRATION[kx_calibration]": { - "scaling": "@attrs:metadata/momentum_correction/calibration/scale_kx", - "offset": "@attrs:metadata/momentum_correction/offset_kx", - "calibrated_axis": "@attrs:metadata/momentum_correction/calibration/axis_kx" + "/ENTRY/PROCESS_MPES[process]/CALIBRATION[kx_calibration]": { + "scaling": "['@attrs:metadata/momentum_calibration/calibration/kx_scale', '@attrs:metadata/momentum_correction/calibration/scale_kx']", + "scaling/@units": "", + "offset": "['@attrs:metadata/momentum_calibration/calibration/x_center', '@attrs:metadata/momentum_correction/offset_kx']", + "offset/@units": "", + "calibrated_axis": "['@attrs:metadata/momentum_calibration/calibration/kx_axis', '@attrs:metadata/momentum_correction/calibration/axis_kx']", + "calibrated_axis/@units": "1/angstrom", + "physical_quantity": "momentum" }, - "/ENTRY[entry]/PROCESS[process]/CALIBRATION[ky_calibration]": { - "scaling": "@attrs:metadata/momentum_correction/calibration/scale_ky", - "offset": "@attrs:metadata/momentum_correction/offset_ky", - "calibrated_axis": "@attrs:metadata/momentum_correction/calibration/axis_ky" + "/ENTRY/PROCESS_MPES[process]/CALIBRATION[ky_calibration]": { + "scaling": "['@attrs:metadata/momentum_calibration/calibration/ky_scale', '@attrs:metadata/momentum_correction/calibration/scale_ky']", + "offset": "['@attrs:metadata/momentum_calibration/calibration/y_center', '@attrs:metadata/momentum_correction/offset_ky']", + "calibrated_axis": "['@attrs:metadata/momentum_calibration/calibration/ky_axis', '@attrs:metadata/momentum_correction/calibration/axis_ky']", + "calibrated_axis/@units": "1/angstrom", + "physical_quantity": "momentum" }, - "/ENTRY[entry]/data": { + "/ENTRY/data": { "@axes": "@data:dims", - "AXISNAME_indices[@*_indices]": "@data:*.index", + "@*_indices": "@data:*.index", "@signal": "data", "data": "@data:data", "data/@units": "counts", - "AXISNAME[*]": "@data:*.data", - "AXISNAME[*]/@units": "@data:*.unit", - "energy/@type": "binding" + "*": "@data:*.data", + "*/@units": "@data:*.unit", + "energy/@type": "['@attrs:metadata/energy_calibration/calibration/energy_scale', 'kinetic']" } -} \ No newline at end of file +} diff --git a/src/pynxtools_mpes/nomad/examples/sed_config/mpes_example_config.yaml b/src/pynxtools_mpes/nomad/examples/sed_config/mpes_example_config.yaml new file mode 100644 index 0000000..90a6624 --- /dev/null +++ b/src/pynxtools_mpes/nomad/examples/sed_config/mpes_example_config.yaml @@ -0,0 +1,314 @@ +core: + # Set verbosity of sed + verbose: True + # The loader to use. The mpes loader allows for loading hdf5 files from the METIS momentum microscope. + loader: mpes + # Option to use the copy tool to mirror data to a local storage location before processing. + use_copy_tool: False + # path to the root of the source data directory + copy_tool_source: "/path/to/data/" + # path to the root or the local data storage + copy_tool_dest: "/path/to/localDataStore/" + # optional keyworkds for the copy tool: + copy_tool_kwds: + # number of parallel copy jobs + ntasks: 20 + # group id to set for copied files and folders + gid: 1001 + +dataframe: + # hdf5 group names to read from the h5 files (for mpes reader) + hdf5_groupnames: ["Stream_0", "Stream_1", "Stream_2", "Stream_4"] + # aliases to assign to the dataframe columns for the corresponding hdf5 streams + hdf5_aliases: + Stream_0: "X" + Stream_1: "Y" + Stream_2: "t" + Stream_4: "ADC" + # dataframe column name for the time stamp column + time_stamp_alias: "timeStamps" + # hdf5 group name containing eventIDs occuring at every millisecond (used to calculate timestamps) + ms_markers_group: "msMarkers" + # hdf5 attribute containing the timestamp of the first event in a file + first_event_time_stamp_key: "FirstEventTimeStamp" + # Time stepping in seconds of the succesive events in the timed dataframe + timed_dataframe_unit_time: 0.001 + # list of columns to apply jitter to + jitter_cols: ["X", "Y", "t", "ADC"] + # dataframe column containing x coordinates + x_column: "X" + # dataframe column containing y coordinates + y_column: "Y" + # dataframe column containing time-of-flight data + tof_column: "t" + # dataframe column containing analog-to-digital data + adc_column: "ADC" + # dataframe column containing corrected x coordinates + corrected_x_column: "Xm" + # dataframe column containing corrected y coordinates + corrected_y_column: "Ym" + # dataframe column containing corrected time-of-flight data + corrected_tof_column: "tm" + # dataframe column containing kx coordinates + kx_column: "kx" + # dataframe column containing ky coordinates + ky_column: "ky" + # dataframe column containing energy data + energy_column: "energy" + # dataframe column containing delay data + delay_column: "delay" + # time length of a base time-of-flight bin in ns + tof_binwidth: 4.125e-12 + # Binning factor of the tof_column-data compared to tof_binwidth (2^(tof_binning-1)) + tof_binning: 2 + # binning factor used for the adc coordinate (2^(adc_binning-1)) + adc_binning: 3 + # Default units for dataframe entries + units: + X: 'step' + Y: 'step' + t: 'step' + tof_voltage: 'V' + extractor_voltage: 'V' + extractor_current: 'A' + cryo_temperature: 'K' + sample_temperature: 'K' + dld_time: 'ns' + delay: 'ps' + timeStamp: 's' + energy: 'eV' + E: 'eV' + kx: '1/A' + ky: '1/A' + +energy: + # Number of bins to use for energy calibration traces + bins: 1000 + # Bin ranges to use for energy calibration curves (for tof_binning=0) + ranges: [128000, 138000] + # hdf5 path to attribute storing bias information for a given file + bias_key: "@KTOF:Lens:Sample:V" + # Option to normalize energy calibration traces + normalize: True + # Pixel range for smoothing + normalize_span: 7 + # Spline order for smoothing + normalize_order: 1 + # Radius parameter for fastdtw algorithm to find path correspondence + fastdtw_radius: 2 + # Window around a peak to make sure that no other peaks are present + peak_window: 7 + # Mehtod to use for energy calibration + calibration_method: "lmfit" + # Energy scale to use for energy calibration + energy_scale: "kinetic" + # Approximate position of the high-energy-cutoff in tof_column bins, + # used for displaying a graph to choose the energy correction function parameters. + tof_fermi: 132250 + # TOF range to visualize for the correction tool around tof_fermi + tof_width: [-600, 1000] + # x-intergration range for the correction tool around the center pixel + x_width: [-20, 20] + # y-intergration range for the correction tool around the center pixel + y_width: [-20, 20] + # High intensity cutoff for the visulaization tool + color_clip: 300 + correction: + # Correction type + correction_type: "Lorentzian" + # Correction amplitude + amplitude: 2.5 + # center coordinates for the correction (in detector coordinates) + center: [730.0, 730.0] + # gamma value for the Lorentzian correction (same for x and y) + gamma: 920.0 + # sigma value for the gaussian correction (same for x and y) + sigma: 700.0 + # diameter value for the radial correction (same for x and y) + diameter: 3000.0 + # Default energy calibration + calibration: + # time-of-flight distance (in m) + d: 1.058206295066418 + # time offset (in ns) + t0: 7.684410678887588e-07 + # energy offset (in eV) + E0: -30.440035779171833 + # energy scale of calibration + energy_scale: "kinetic" + +momentum: + # binning axes to use for momentum correction/calibration. + # Axes names starting with "@" refer to keys in the "dataframe" section + axes: ["@x_column", "@y_column", "@tof_column"] + # Bin numbers used for the respective axes + bins: [512, 512, 300] + # bin ranges to use (in unbinned detector coordinates) + ranges: [[-256, 1792], [-256, 1792], [132000, 136000]] + # The x/y pixel ranges of the detector + detector_ranges: [[0, 2048], [0, 2048]] + # The center pixel of the detector in the binned x/y coordinates + center_pixel: [256, 256] + # Sigma parameter for feature selection (intensity above background) + sigma: 5 + # FWHM parameter for feature selection (width of features to extract) + fwhm: 8 + # Sigma_radius parameter for feature selection (variation of radius size) + sigma_radius: 1 + # default momentum calibration + calibration: + # x momentum scaleing factor + kx_scale: 0.010729535670610963 + # y momentum scaleing factor + ky_scale: 0.010729535670610963 + # x BZ center pixel + x_center: 256.0 + # y BZ center pixel + y_center: 256.0 + # x start value of the calibration dataset + rstart: -256. + # y start value of the calibration dataset + cstart: -256. + # x direction pixel stepping of the calibration dataset + rstep: 4.0 + # y direction pixel stepping of the calibration dataset + cstep: 4.0 + correction: + # default feature points used for calculating the distortion correction. + feature_points: [[203.2, 341.96], [299.16, 345.32], [350.25, 243.70], [304.38, 149.88], [199.52, 152.48], [154.28, 242.27], [248.29, 248.62]] + # rotational symmetry of the structure used for correction. Should be an even number + rotation_symmetry: 6 + # Option whether the center of the structure is included in the feature points. + include_center: True + # Option whether the center should be included in the correction algorithm + use_center: True + +delay: + # value ranges of the analog-to-digital converter axes used for encoding the delay stage position + # (in unbinned coordinates) + adc_range: [1900, 25600] + # hdf5 attribute containing the starting point of the delay stage + p1_key: "@trARPES:DelayStage:p1" + # hdf5 attribute containing the end point of the delay stage + p2_key: "@trARPES:DelayStage:p2" + # hdf5 attribute containing the t0 value of the delay stage + t0_key: "@trARPES:DelayStage:t0" + +binning: + # Histogram computation mode to use. + hist_mode: "numba" + # Mode for hostogram recombination to use + mode: "fast" + # Whether to display a progress bar + pbar: True + # Number of parallel binning threads to use + num_cores: 20 + # Number of multithreading threads per worker thread + threads_per_worker: 4 + # API for numpy multithreading + threadpool_API: "blas" + +histogram: + # number of bins used for histogram visualization + bins: [80, 80, 80, 80] + # default axes to use for histgram visualization. + # Axes names starting with "@" refer to keys in the "dataframe" section + axes: ["@x_column", "@y_column", "@tof_column", "@adc_column"] + # default ranges to use for histogram visualization (in unbinned detector coordinates) + ranges: [[0, 1800], [0, 1800], [128000, 138000], [0, 32000]] + +metadata: + # URL of the epics archiver request engine + archiver_url: "http://aa0.fhi-berlin.mpg.de:17668/retrieval/data/getData.json?pv=" + # EPICS channels to collect from EPICS archiver + epics_pvs: ["KTOF:Lens:Extr:I", "trARPES:Carving:TEMP_RBV", "trARPES:XGS600:PressureAC:P_RD", "KTOF:Lens:UDLD:V", "KTOF:Lens:Sample:V", "KTOF:Apertures:m1.RBV", "KTOF:Apertures:m2.RBV", "KTOF:Apertures:m3.RBV", "trARPES:Carving:TRX.RBV", "trARPES:Carving:TRY.RBV", "trARPES:Carving:TRZ.RBV", "trARPES:Carving:THT.RBV", "trARPES:Carving:PHI.RBV", "trARPES:Carving:OMG.RBV"] + # hdf5 attribute containing the field aperture "in" motor position + fa_in_channel: 'KTOF:Apertures:m1.RBV' + # hdf5 attribute containing the field aperture "hor" motor position + fa_hor_channel: 'KTOF:Apertures:m2.RBV' + # hdf5 attribute containing the contrast aperture "in" motor position + ca_in_channel: 'KTOF:Apertures:m3.RBV' + # dictionary containing contrast and field aperture motor positions and sizes + aperture_config: + "2018-01-23T19:35:15": + fa_size: + '750': [[-3.0, -1.4], [-5.4, -4.6]] + grid: [[-3.0, -1.4], [0.15, 1.75]] + '1500': [[-3.0, -1.4], [6.25, 7.75]] + '200': [[3.3, 4.4], [-5.4, -4.6]] + '500': [[3.3, 4.4], [0.15, 1.75]] + '1000': [[3.3, 4.4], [6.25, 7.75]] + '20': [[9.6, 10.1], [-5.4, -4.6]] + '50': [[9.6, 10.1], [0.15, 1.75]] + '100': [[9.6, 10.1], [6.25, 7.75]] + open: [[-15, -9.0], [-15, -8.9]] + ca_size: + '50': [8.0, 8.4] + '200': [-0.5, -0.9] + '100': [3.4, 3.8] + grid: [-5.3, -5.9] + open: [-12.0, -8] + "2020-01-23T19:35:15": + fa_size: + '750': [[-6.2, -4.8], [5.0, 6.0]] + grid: [[-6.2, -4.8], [-0.7, -0.3]] + '500': [[-6.2, -4.8], [-7.0, -6.0]] + '200': [[0.5, 0.9], [-0.7, -0.3]] + '100': [[0.5, 0.9], [-7.0, -6.0]] + '300': [[0.5, 0.9], [5.0, 6.0]] + '10': [[6.5, 6.9], [-7.0, -6.0]] + '20': [[6.5, 6.9], [-0.7, -0.3]] + '50': [[6.5, 6.9], [5.0, 6.0]] + open: [[-15, -8.5], [-15, -8.9]] + ca_size: + '50': [9.0, 11.0] + '300': [-0.1, 0.1] + '200': [0.7, 1.5] + '100': [5.1, 5.9] + grid: [-5.5, -5.2] + open: [-15, -8.5] + # dictionary containing lens mode configurations + lens_mode_config: + "6kV_kmodem4.0_20VTOF_v3.sav": + Extr: 6000.0 + UCA: 1200 + UFA: 600.0 + Z1: 2452.9 + Z2: 1489.9 + A: 420.07 + B: 2494.8 + C: 489.2 + D: 228.05 + E: 113.82 + F: 54.232 + G: 20.0 + H: 25.5 + I: 36.0 + TOF: 20.0 + MCPfront: 20.0 + "6kV_kmodem4.0_30VTOF_453ns_focus.sav": + Extr: 6000.0 + UCA: 1200 + UFA: 600.0 + Z1: 2452.9 + Z2: 1489.9 + A: 403.07 + B: 2500 + C: 422.25 + D: 208.88 + E: 199.49 + F: 68.735 + G: 30.0 + H: 30.0 + I: 44.5 + TOF: 30.0 + MCPfront: 30.0 + +nexus: + # pynxtools reader to use for saving NXmpes files + reader: "mpes" + # NeXus application definition to use for saving + definition: "NXmpes" + # List conatining additional input files to be handed to the pynxtools converter tool, + # e.g. containing a configuration file, and additional metadata. + input_files: ["config_file.json"]