Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vamas sub-reader: enhance data extraction, add IRREGULAR parsing #10

Merged

Conversation

lukaspie
Copy link
Collaborator

@lukaspie lukaspie commented Feb 20, 2024

This enhances the capabilities of the existing Vamas XPS reader.
The Vamas reader reads .vms files, which is the current ISO standard data transfer format (ISO 14976).

VMS specific:

  • Write CPS data to file
  • Extract units for quantities
  • Parse header comments -> additional metadata (depends on spectrometer software that writes the vms file)
  • Parse comment lines -> data analysis
  • Parse vamas files of type IRREGULAR
  • Update config file (will be done again when we have the [*] syntax from the MultiFormatReader

General:

  • align mapper dictionary keys

@lukaspie lukaspie linked an issue Feb 20, 2024 that may be closed by this pull request
@lukaspie lukaspie changed the title XPS vamas sub-reader: enhance data extraction, add IRREGULAR parsing Vamas sub-reader: enhance data extraction, add IRREGULAR parsing Feb 22, 2024
@lukaspie lukaspie added bug Something isn't working enhancement New feature or request sub-reader labels Mar 4, 2024
@coveralls
Copy link

coveralls commented Mar 5, 2024

Pull Request Test Coverage Report for Build 8294709433

Details

  • 356 of 480 (74.17%) changed or added relevant lines in 8 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+14.8%) to 48.748%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pynxtools_xps/sle/sle_specs.py 0 2 0.0%
pynxtools_xps/txt/txt_scienta.py 0 2 0.0%
pynxtools_xps/txt/txt_vamas_export.py 0 2 0.0%
pynxtools_xps/xy/xy_specs.py 0 2 0.0%
pynxtools_xps/reader_utils.py 27 31 87.1%
pynxtools_xps/vms/vamas.py 198 239 82.85%
pynxtools_xps/vms/casa_data_model.py 93 164 56.71%
Totals Coverage Status
Change from base Build 8293922186: 14.8%
Covered Lines: 1285
Relevant Lines: 2636

💛 - Coveralls

@lukaspie lukaspie requested review from domna and RubelMozumder March 8, 2024 16:03
@lukaspie lukaspie marked this pull request as ready for review March 11, 2024 09:47
@RubelMozumder
Copy link
Contributor

Here are the some change requests.
I see this is a big code base. So, you can add some tests specific to the reader according to the file extension.

@lukaspie lukaspie force-pushed the 9-xps-vamas-sub-reader-add-regular-parsing-enhance-data-extraction branch from 3ac7f2a to 4c26b37 Compare March 15, 2024 09:41
pynxtools_xps/vms/vamas.py Show resolved Hide resolved
pynxtools_xps/vms/vamas.py Show resolved Hide resolved
pynxtools_xps/vms/vamas.py Show resolved Hide resolved
pynxtools_xps/vms/vamas.py Outdated Show resolved Hide resolved
pynxtools_xps/vms/vamas.py Outdated Show resolved Hide resolved
pynxtools_xps/vms/vamas.py Outdated Show resolved Hide resolved
@RubelMozumder
Copy link
Contributor

Sorry for the confusion. I reviewed some code on the first review but some of them submitted and some of them I did not submit. So, now you see new change requests. Which I am going to resolve now. I think there a few comment you may add in the future especially pynxtools_xps/vms/vamas.py where you assumed all the data will come in a chain. But if one of them is broken or missing other fields will read the wrong data. So in another PR you can take care of it.

@lukaspie
Copy link
Collaborator Author

Sorry for the confusion. I reviewed some code on the first review but some of them submitted and some of them I did not submit. So, now you see new change requests. Which I am going to resolve now. I think there a few comment you may add in the future especially pynxtools_xps/vms/vamas.py where you assumed all the data will come in a chain. But if one of them is broken or missing other fields will read the wrong data. So in another PR you can take care of it.

There is really no other way around then reading the data in a list. The Vamas format does not allow for keywords in the file and it alway assumes that all of the elements are at the correct positions (it is an ISO standard afterall, so if you don't have the right structure, it cannot be a vamas file).

@lukaspie lukaspie merged commit ac2768b into main Mar 15, 2024
6 checks passed
@lukaspie lukaspie deleted the 9-xps-vamas-sub-reader-add-regular-parsing-enhance-data-extraction branch March 15, 2024 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request sub-reader
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Vamas sub-reader: add regular parsing, enhance data extraction
4 participants