Merge branch 'main' into 3-xps-reader-support-adding-slh-metadata-fil…

…es-for-sle-reader

.github/workflows/publish.yml

@@ -0,0 +1,38 @@
+# This workflow will upload a Python Package using Twine when a release is created
+# For more information see: https://help.github.com/en/actions/language-and-framework-guides/using-python-with-github-actions#publishing-to-package-registries
+
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+name: Upload Python Package
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+          submodules: recursive
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.x"
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install build
+      - name: Build package
+        run: python -m build
+      - name: Publish package
+        uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
+        with:
+          user: __token__
+          password: ${{ secrets.PYPI_API_TOKEN }}

.github/workflows/pytest.yml

@@ -5,9 +5,9 @@ name: pytest
 
 on:
   push:
-    branches: [master]
+    branches: [main]
   pull_request:
-    branches: [master]
+    branches: [main]
 
 jobs:
   pytest:

README.md

@@ -1,6 +1,10 @@
 [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
-![](https://github.com/FAIRmat-NFDI/pynxtools_xps/actions/workflows/pytest.yml/badge.svg)
-![](https://github.com/FAIRmat-NFDI/pynxtools_xps/actions/workflows/pylint.yml/badge.svg)
+![](https://github.com/FAIRmat-NFDI/pynxtools-xps/actions/workflows/pytest.yml/badge.svg)
+![](https://github.com/FAIRmat-NFDI/pynxtools-xps/actions/workflows/pylint.yml/badge.svg)
+![](https://github.com/FAIRmat-NFDI/pynxtools-xps/actions/workflows/publish.yml/badge.svg)
+![](https://img.shields.io/pypi/pyversions/pynxtools-xps)
+![](https://img.shields.io/pypi/l/pynxtools-xps)
+![](https://img.shields.io/pypi/v/pynxtools-xps)
 ![](https://coveralls.io/repos/github/FAIRmat-NFDI/pynxtools_xps/badge.svg?branch=master)
 
 # A reader for XPS data
@@ -14,20 +18,20 @@ Learn how to manage [python versions](https://github.com/pyenv/pyenv) and
 Install this package with
 
 ```shell
-pip install git+https://github.com/FAIRmat-NFDI/pynxtools_xps.git
+pip install git+https://github.com/FAIRmat-NFDI/pynxtools-xps.git
 ```
 
 for the latest development version.
 
 
 # Purpose
-This pynxtools reader plugin is used to translate diverse file formats from the scientific community and technology partners
+This reader plugin for [pynxtools](https://github.com/FAIRmat-NFDI/pynxtools) is used to translate diverse file formats from the scientific community and technology partners
 within the field of X-ray photoelectron spectroscopy into a standardized representation using the
 [NeXus](https://www.nexusformat.org/) application definition [NXmpes](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html#nxmpes).
 
 ## Supported file formats
 The reader decides which parser to use based on the file extension of the files provided. For the main XPS files, the following file extensions are supported:
-- .sle: [SpecsLabProdigy](https://www.specs-group.com/nc/specs/products/detail/prodigy/) files, propietary format of SPECS GmbH (v1.6)
+- .sle: [SpecsLabProdigy](https://www.specs-group.com/nc/specs/products/detail/prodigy/) files, propietary format of SPECS GmbH (1 and v4)
 - .xml: SpecsLab 2files, XML format from SPECS GmbH (v1.6)
 - .vms: VAMAS files, ISO standard data transfer format ([ISO 14976](https://www.iso.org/standard/24269.html)), both in regular and irregular format
 - .xy: SpecsLabProdigy export format in XY format (including all export settings)
@@ -45,10 +49,9 @@ An example script to run the XPS reader in pynxtools:
 --nxdl NXmpes \
 --input-file $<xps-file path> \
 --input-file $<eln-file path> \
---output <output-file path>.test.nxs
+--output <output-file path>.nxs
 ```
-Note that none of the supported file format have data/values for all required and recommended fields and attributes in NXmpes. In order for the validation step of the **XPS** reader to pass,
-you need to provide an ELN file that contains the missing values. An example can be found in  [*pynxtools_xps/examples*](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples).
+Note that none of the supported file format have data/values for all required and recommended fields and attributes in NXmpes. In order for the validation step of the XPS reader to pass, you need to provide an ELN file that contains the missing values. Example raw and converted data can be found in  [*pynxtools_xps/examples*](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/examples).
 
 
 # Contributing
@@ -58,7 +61,7 @@ you need to provide an ELN file that contains the missing values. An example can
 Install the package with its dependencies:
 
 ```shell
-git clone https://github.com/FAIRmat-NFDI/pynxtools_xps.git \\
+git clone https://github.com/FAIRmat-NFDI/pynxtools-xps.git \\
     --branch master \\
     --recursive pynxtools_xps
 cd pynxtools_xps
@@ -75,6 +78,12 @@ pre-commit install
 ```
 from the root of this repository.
 
+## Development Notes
+The development process is modular so that new parsers can be added. The design logic is the following:
+1. First, [`XpsDataFileParser`](https://github.com/FAIRmat-NFDI/pynxtools-xps/blob/main/pynxtools_xps/file_parser.py#L36) selects the proper parser based on the file extensions of the provided files. It then calls a sub-parser that can read files with such extensions and calls the `parse_file` function of that reader. In addition, it selects a proper config file from
+the `config` subfolder.
+2. Afterwards, the NXmpes nxdl template is filled with the data in `XpsDataFileParser` using the [`config`](https://github.com/FAIRmat-NFDI/pynxtools-xps/tree/main/pynxtools_xps/config) file. Data that is not in the given main files can be added through the ELN file (and must be added for required fields in NXmpes).
+
 ## Test this software
 
 Especially relevant for developers, there exists a basic test framework written in
@@ -84,15 +93,5 @@ Especially relevant for developers, there exists a basic test framework written
 python -m pytest -sv tests
 ```
 
-
-
-## Development Notes
-The development process is modular so that new parsers can be added. The read logic is the following.
-1. First, [*XpsDataFileParser*]([https://github.com/FAIRmat-NFDI/pynxtools_xps/blob/main/pynxtools_xps/dataconverter/readers/xps/file_parser.py#L39]) selects the proper parser based on the file extensions
-of the provided files. It then calls a sub-parser that can read files with such extensions and calls the *parse_file* function of that reader. In addition, it selects a proper config file from
-the *config* subfolder.
-2. Afterwards, the NXmpes nxdl template is filled with the data in *XpsDataFileParser* using the *config* file. Data that is not in the given main files can be added through the ELN file (and must
-be added for required fields in NXmpes).
-
 ## Contact person in FAIRmat for this reader
 Lukas Pielsticker

dev-requirements.txt

@@ -1,5 +1,5 @@
 #
-# This file is autogenerated by pip-compile with Python 3.8
+# This file is autogenerated by pip-compile with Python 3.10
 # by the following command:
 #
 #    pip-compile --extra=dev --output-file=dev-requirements.txt pyproject.toml
@@ -20,10 +20,6 @@ attrs==23.1.0
     #   requests-cache
 backcall==0.2.0
     # via ipython
-backports-zoneinfo==0.2.1
-    # via
-    #   pytz-deprecation-shim
-    #   tzlocal
 blosc2==2.0.0
     # via tables
 build==1.0.3
@@ -55,16 +51,13 @@ comm==0.2.0
 contourpy==1.1.1
     # via matplotlib
 coverage[toml]==7.3.2
-    # via
-    #   coverage
-    #   pytest-cov
+    # via pytest-cov
 cycler==0.12.1
     # via matplotlib
 cython==3.0.6
     # via tables
 dask[array]==2023.5.0
     # via
-    #   dask
     #   hyperspy
     #   kikuchipy
     #   orix
@@ -129,6 +122,7 @@ h5py==3.10.0
     #   orix
     #   pyfai
     #   pynxtools
+    #   pynxtools-xps (pyproject.toml)
     #   pyxem
     #   silx
 hdf5plugin==4.3.0
@@ -154,16 +148,9 @@ imageio==2.27.0
     #   scikit-image
 importlib-metadata==6.8.0
     # via
-    #   build
     #   dask
     #   hyperspy
-    #   jupyter-client
-    #   numba
     #   pynxtools
-importlib-resources==6.1.1
-    # via
-    #   matplotlib
-    #   radioactivedecay
 iniconfig==2.0.0
     # via pytest
 ipykernel==6.27.1
@@ -231,7 +218,7 @@ mpmath==1.3.0
 msgpack==1.0.7
     # via blosc2
 mypy==1.7.1
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 mypy-extensions==1.0.0
     # via mypy
 natsort==8.4.0
@@ -356,7 +343,7 @@ pillow==10.0.1
 pint==0.21.1
     # via hyperspy
 pip-tools==7.3.0
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 platformdirs==4.0.0
     # via
     #   jupyter-core
@@ -372,7 +359,7 @@ pooch==1.8.0
     #   kikuchipy
     #   orix
 pre-commit==3.5.0
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 prettytable==3.9.0
     # via hyperspy
 prompt-toolkit==3.0.41
@@ -394,20 +381,20 @@ pyfai==2023.9.0
 pygments==2.17.2
     # via ipython
 pynxtools==0.0.10
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 pyparsing==3.1.1
     # via matplotlib
 pyproject-hooks==1.0.0
     # via build
 pytest==7.4.3
     # via
-    #   pynxtools_xps (pyproject.toml)
+    #   pynxtools-xps (pyproject.toml)
     #   pytest-cov
     #   pytest-timeout
 pytest-cov==4.1.0
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 pytest-timeout==2.2.0
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 python-dateutil==2.8.2
     # via
     #   hyperspy
@@ -424,8 +411,6 @@ pytz-deprecation-shim==0.1.0.post0
     # via tzlocal
 pywavelets==1.4.1
     # via scikit-image
-pywin32==306
-    # via jupyter-core
 pyxem==0.16.0
     # via pynxtools
 pyyaml==6.0.1
@@ -451,7 +436,7 @@ requests==2.31.0
 requests-cache==1.1.1
     # via pynxtools
 ruff==0.1.7
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 scikit-image==0.20.0
     # via
     #   hyperspy
@@ -493,7 +478,7 @@ sparse==0.14.0
 stack-data==0.6.3
     # via ipython
 structlog==23.2.0
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 sympy==1.12
     # via
     #   hyperspy
@@ -550,15 +535,14 @@ transforms3d==0.4.1
     #   diffsims
     #   pyxem
 types-pytz==2023.3.1.1
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 types-pyyaml==6.0.12.12
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 types-requests==2.31.0.10
-    # via pynxtools_xps (pyproject.toml)
+    # via pynxtools-xps (pyproject.toml)
 typing-extensions==4.8.0
     # via
     #   cattrs
-    #   ipython
     #   mypy
 tzdata==2023.3
     # via
@@ -587,15 +571,15 @@ wcwidth==0.2.12
 wheel==0.42.0
     # via pip-tools
 xarray==2023.1.0
-    # via pynxtools
+    # via
+    #   pynxtools
+    #   pynxtools-xps (pyproject.toml)
 zarr==2.16.1
     # via hyperspy
 zipfile37==0.1.3
     # via pynxtools
 zipp==3.17.0
-    # via
-    #   importlib-metadata
-    #   importlib-resources
+    # via importlib-metadata
 
 # The following packages are considered to be unsafe in a requirements file:
 # pip

examples/README.md

@@ -1,22 +1,17 @@
-# XPS Reader
+# Examples for the XPS
 
-## What is this reader?
+Here, you can find working examples of XPS-to-NeXus conversion using the latest [NXmpes](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html#nxmpes) application definition and the latest development version of [pynxtools-xps](https://github.com/FAIRmat-NFDI/pynxtools-xps).
 
-This reader supports converting X-ray photoelectron spectroscopy into a NeXus formatted file. The application definiton it follows is [NXmpes](https://fairmat-nfdi.github.io/nexus_definitions/classes/contributed_definitions/NXmpes.html#nxmpes).
+This reader supports converting X-ray photoelectron spectroscopy into a NeXus formatted file. You can find examples for the supported file formats in the individual subfolders.
 
-## Supported file formats
-The reader decides which parser to use based on the file extension of the files provided. For the main XPS files, the following file extensions are supported:
-- .sle: [SpecsLabProdigy](https://www.specs-group.com/nc/specs/products/detail/prodigy/) files, propietary format of SPECS GmbH (v1.6)
-- .xml: SpecsLab 2files, XML format from SPECS GmbH (v1.6)
-- .vms: VAMAS files, ISO standard data transfer format ([ISO 14976](https://www.iso.org/standard/24269.html)), both in regular and irregular format
+## Existing examples
+- .sle: [SpecsLabProdigy](https://www.specs-group.com/nc/specs/products/detail/prodigy/) file, propietary format of SPECS GmbH (software version: v1.6)
+- .xml: SpecsLab 2files, XML format from SPECS GmbH (software version: v4.63)
+- .vms: VAMAS files, ISO standard data transfer format ([ISO 14976](https://www.iso.org/standard/24269.html)), in regular format
 - .xy: SpecsLabProdigy export format in XY format (including all export settings)
 - .txt:
   - exported by [Scienta Omicron](https://scientaomicron.com/en) instruments
   - exported by [CasaXPS](https://www.casaxps.com/) analysis software
 
-```console
-user@box:~$ dataconverter --params-file params.yaml
-```
-
-## Contact person in FAIRmat for this reader
+## Contact person in FAIRmat for these example
 Lukas Pielsticker

examples/sle/README.md

@@ -0,0 +1,12 @@
+# Example for .sle data
+
+This is an example for [SpecsLabProdigy](https://www.specs-group.com/nc/specs/products/detail/prodigy/) .sle files, which is the propietary format of SPECS GmbH. The SpecsLabProdigy software version that was used to measure this data is v4.63.1. The example conversion can be run with the following command.
+
+```console
+user@box:~$ dataconverter --params-file params.yaml
+```
+
+Note that the `params.yaml` file contains the `remove_align` keyword which is special for the SLE parser. It allows removal of alignment spectra that were taken during the experiment. For this example, it considerably speeds up the conversion.
+
+## Contact person in FAIRmat for this example
+Lukas Pielsticker