Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Text format (e.g. IridasCube) parsing optimizations #2074

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

aras-p
Copy link

@aras-p aras-p commented Oct 1, 2024

Primarily driven by a wish to increase performance of parsing .cube files. But the functions that are added or changed are used across parsing of all/most of text based formats.

With these changes, parsing "Khronos PBR Neutral" Iridas .cube file (5.4MB) on Ryzen 5950X / VS2022 Release build: 167ms -> 126ms. Parsing the same file in CLF format: 150ms -> 137ms.

What the PR does is:

  • Add locale independent IsSpace(char). Somewhat similar to 3aab90d, whitespace trimming perhaps should not be locale dependent.
  • Add IsEmptyOrWhiteSpace() and use that inside ParseUtils::nextline, instead of doing Trim(line).empty().
  • Add StringUtils::StartsWith(char) and use that in various parsers that were constructing whole std::string object just to check for a single character.
  • When building for C++17 or later, NumberUtils can use standard <charconv> from_chars functions (except on Apple platforms, where those are not implemented for floating point types as of Xcode 15). This has advantage of not having to deal with errno or locales. Saves some thread local storage accesses and function calls (e.g. on Windows errno is actually a function call).
    • Turns out that NumberUtils::from_chars was only indirectly covered by tests, and as XML parser utils tests point out,
      it should be able to handle hex floats and so on. So all that is implemented there, and explicitly covered by
      tests/utils/NumberUtils_tests.cpp.
    • There's a CMake setup change that adds /Zc:__cplusplus flag for MSVC; for backwards compat reasons it does not report proper C++ version detection defines otherwise.

Primarily driven by a wish to increase performance of parsing .cube
files. But the functions that are added or changed are used across
parsing of all/most of text based formats.

With these changes, parsing "Khronos PBR Neutral" .cube file (5.4MB)
on Ryzen 5950X / VS2022 Release build: 167ms -> 123ms.

- Add locale independent IsSpace(char). Somewhat similar to 3aab90d,
  whitespace trimming perhaps should not be locale dependent.
- Add IsEmptyOrWhiteSpace() and use that inside ParseUtils::nextline,
  instead of doing Trim(line).empty().
- Add StringUtils::StartsWith(char) and use that in various parsers
  that were constructing whole std::string object just to check for a
  single character.
- When building for C++17 or later, NumberUtils can use standard
  <charconv> from_chars functions (except on Apple platforms, where
  those are not implemented for floating point types as of Xcode 15).
  This has advantage of not having to deal with errno or locales. Saves
  some thread local storage accesses and function calls (e.g. on Windows
  errno is actually a function call).
- There's a CMake setup change that adds /Zc:__cplusplus flag for MSVC;
  for backwards compat reasons it does not report proper C++ version
  detection defines otherwise.

Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
Currently it was only tested indirectly via XMLReaderUtils_tests and
file format tests

Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
…kip optional whitespace

To match the pre-C++17 behavior that was there before

Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
@aras-p aras-p changed the title Iridas .cube parser optimizations Text format (e.g. IridasCube) parsing optimizations Oct 2, 2024
Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
@remia
Copy link
Collaborator

remia commented Oct 2, 2024

Apologies for the delay in CI @aras-p, we have to manually approve the workflows due to our current GitHub settings. Trying to see if we can update that or merge your other PR which should also work.

@aras-p
Copy link
Author

aras-p commented Oct 2, 2024

@remia thanks! I admit I did not expect that this PR would need so many failed iterations. <charconv> is a tough beast, it seems :)

Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants