From 64b7c4531627cb61f234b4f4edc64571a596e084 Mon Sep 17 00:00:00 2001 From: Sherjeel Shabih Date: Tue, 21 Nov 2023 15:07:54 +0100 Subject: [PATCH] Update README.md --- .../dataconverter/readers/json_map/README.md | 53 ++++++++++++++++--- 1 file changed, 46 insertions(+), 7 deletions(-) diff --git a/pynxtools/dataconverter/readers/json_map/README.md b/pynxtools/dataconverter/readers/json_map/README.md index 4b4820c49..b81aec969 100644 --- a/pynxtools/dataconverter/readers/json_map/README.md +++ b/pynxtools/dataconverter/readers/json_map/README.md @@ -1,24 +1,63 @@ # JSON Map Reader -This reader allows you to convert either data from a .json file or an xarray exported as a .pickle using a flat .mapping.json file. +## What is this reader? + +This reader is designed to allow users of pynxtools to convert their existing data with the help of a map file. The map file tells the reader what to pick from your data files and convert them to FAIR NeXus files. The following formats are supported as input files: +* HDF5 (any extension works i.e. h5, hdf5, nxs, etc) +* JSON +* Python Dict Objects Pickled with [pickle](https://docs.python.org/3/library/pickle.html). These can contain [xarray.DataArray](https://docs.xarray.dev/en/stable/generated/xarray.DataArray.html) objects as well as regular Python types and Numpy types. It accepts any NXDL file that you like as long as your mapping file contains all the fields. Please use the --generate-template function of the dataconverter to create a .mapping.json file. ```console -user@box:~$ python convert.py --nxdl NXmynxdl --generate-template > mynxdl.mapping.json +user@box:~$ dataconverter --nxdl NXmynxdl --generate-template > mynxdl.mapping.json ``` There are some example files you can use: +[data.mapping.json](/tests/data/dataconverter/readers/json_map/data.mapping.json) -[data.mapping.json](/tests/data/tools/dataconverter/readers/json_map/data.mapping.json) - -[data.json](/tests/data/tools/dataconverter/readers/json_map/data.json) +[data.json](/tests/data/dataconverter/readers/json_map/data.json) ```console -user@box:~$ python convert.py --nxdl NXtest --input-file data.json --input-file data.mapping.json --reader json_map +user@box:~$ dataconverter --nxdl NXtest --input-file data.json --mapping data.mapping.json +``` + +##### [Example](/examples/json_map/) with HDF5 files. + +## The mapping.json file + +This file is designed to let you fill in the requirements of a NeXus Application Definition without writing any code. If you already have data in the formats listed above, you just need to use this mapping file to help the dataconverter pick your data correctly. + +The mapping files will always be based on the Template the dataconverter generates. See above on how to generate a mapping file. +The right hand side values of the Template keys are what you can modify. + +Here are the three different ways you can fill the right hand side of the Template keys: +* Write the nested path in your datafile. This is indicated by a leading `/` before the word `entry` to make `/entry/data/current_295C` below. +Example: + +```json + "/ENTRY[entry]/DATA[data]/current_295C": "/entry/data/current_295C", + "/ENTRY[entry]/NXODD_name/posint_value": "/a_level_down/another_level_down/posint_value", +``` + +* Write the values directly in the mapping file for missing data from your data file. + +```json + + "/ENTRY[entry]/PROCESS[process]/program": "Bluesky", + "/ENTRY[entry]/PROCESS[process]/program/@version": "1.6.7" +``` + +* Write JSON objects with a link key. This follows the same link mechanism that the dataconverter implements. In the context of this reader, you can only use external links to your data files. In the example below, `current.nxs` is an already existing HDF5 file that we link to in our new NeXus file without copying over the data. The format is as follows: +`"link": ":"` +Note: This only works for HDF5 files currently. + +```json + "/ENTRY[entry]/DATA[data]/current_295C": {"link": "current.nxs:/entry/data/current_295C"}, + "/ENTRY[entry]/DATA[data]/current_300C": {"link": "current.nxs:/entry/data/current_300C"}, ``` ## Contact person in FAIRmat for this reader -Sherjeel Shabih \ No newline at end of file +Sherjeel Shabih