-
Notifications
You must be signed in to change notification settings - Fork 452
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Text format (e.g. IridasCube) parsing optimizations #2074
Open
aras-p
wants to merge
7
commits into
AcademySoftwareFoundation:main
Choose a base branch
from
aras-p:cube-parser-opt
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Primarily driven by a wish to increase performance of parsing .cube files. But the functions that are added or changed are used across parsing of all/most of text based formats. With these changes, parsing "Khronos PBR Neutral" .cube file (5.4MB) on Ryzen 5950X / VS2022 Release build: 167ms -> 123ms. - Add locale independent IsSpace(char). Somewhat similar to 3aab90d, whitespace trimming perhaps should not be locale dependent. - Add IsEmptyOrWhiteSpace() and use that inside ParseUtils::nextline, instead of doing Trim(line).empty(). - Add StringUtils::StartsWith(char) and use that in various parsers that were constructing whole std::string object just to check for a single character. - When building for C++17 or later, NumberUtils can use standard <charconv> from_chars functions (except on Apple platforms, where those are not implemented for floating point types as of Xcode 15). This has advantage of not having to deal with errno or locales. Saves some thread local storage accesses and function calls (e.g. on Windows errno is actually a function call). - There's a CMake setup change that adds /Zc:__cplusplus flag for MSVC; for backwards compat reasons it does not report proper C++ version detection defines otherwise. Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
aras-p
force-pushed
the
cube-parser-opt
branch
from
October 1, 2024 10:35
dab9471
to
769e887
Compare
Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
Currently it was only tested indirectly via XMLReaderUtils_tests and file format tests Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
…kip optional whitespace To match the pre-C++17 behavior that was there before Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
aras-p
force-pushed
the
cube-parser-opt
branch
from
October 2, 2024 08:08
0e81b9b
to
8428381
Compare
aras-p
changed the title
Iridas .cube parser optimizations
Text format (e.g. IridasCube) parsing optimizations
Oct 2, 2024
Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
aras-p
force-pushed
the
cube-parser-opt
branch
from
October 2, 2024 09:54
4ecef8f
to
3353ac4
Compare
Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
Apologies for the delay in CI @aras-p, we have to manually approve the workflows due to our current GitHub settings. Trying to see if we can update that or merge your other PR which should also work. |
@remia thanks! I admit I did not expect that this PR would need so many failed iterations. |
Signed-off-by: Aras Pranckevicius <aras@nesnausk.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Primarily driven by a wish to increase performance of parsing .cube files. But the functions that are added or changed are used across parsing of all/most of text based formats.
With these changes, parsing "Khronos PBR Neutral" Iridas .cube file (5.4MB) on Ryzen 5950X / VS2022 Release build: 167ms -> 126ms. Parsing the same file in CLF format: 150ms -> 137ms.
What the PR does is:
IsSpace(char)
. Somewhat similar to 3aab90d, whitespace trimming perhaps should not be locale dependent.IsEmptyOrWhiteSpace()
and use that insideParseUtils::nextline
, instead of doingTrim(line).empty()
.StringUtils::StartsWith(char)
and use that in various parsers that were constructing wholestd::string
object just to check for a single character.NumberUtils
can use standard<charconv>
from_chars
functions (except on Apple platforms, where those are not implemented for floating point types as of Xcode 15). This has advantage of not having to deal witherrno
or locales. Saves some thread local storage accesses and function calls (e.g. on Windowserrno
is actually a function call).NumberUtils::from_chars
was only indirectly covered by tests, and as XML parser utils tests point out,it should be able to handle hex floats and so on. So all that is implemented there, and explicitly covered by
tests/utils/NumberUtils_tests.cpp
./Zc:__cplusplus
flag for MSVC; for backwards compat reasons it does not report proper C++ version detection defines otherwise.