[tmva][sofie] Parse generated code at test runtime #21184
Conversation
guitargeek
force-pushed
the
sofie_tests
branch
2 times, most recently
from
cfba928 to
a47de91
Compare
Test Results 22 files 22 suites 3d 15h 35m 27s ⏱️ Results for commit 2f82ccb. ♻️ This comment has been updated with latest results. |
TMVA SOFIE development is challenging sometimes, because of how the tests are structured. The tests that covers many possible models imported from ONNX or ROOT have the issue that they includes **all** emitted code in the compiled executables. This means that one gets a build failure on the first model that generated invalid code, and that was it. Therefore, it's difficult to debug what is going wrong. This commit suggests include the generated code with the interpreter instead. Then, one can check for each individual model if the code was valid, and if not, skip over to the next test a print the emitted code that failed to compile. It has some performance overhead, but the tests still only take about 6 seconds. The drastically improved debugging experience justifies these few extra seconds spent on testing. This was motivated by the effort to refactor the SOFIE-emitted code to make it differentiable with Clad.
guitargeek
force-pushed
the
sofie_tests
branch
from
a47de91 to
2f82ccb
Compare
lmoneta
left a comment
lmoneta
left a comment
There was a problem hiding this comment.
Hi,
Thank you Jonas for this PR. I agree this is an improvement in the testing, allowing for better flexibilility and debugging. It is something I wanted to do, but did not find the time.
The only doubt is that we have a class designed for parsong and compiling a model on the flay and then execute the inference (RSofieReader). Maybe to avoid some code duplication at some point we could refactor the tests to use the RSofieReader class, but this can be done later.
|
Thanks for the review! Good point with the RSofieReader 👍 That would be a good followup, but from a practical point of view I'm happy with how it is right now for the development that I want to do. |
2 participants
TMVA SOFIE development is challenging sometimes, because of how the tests are structured.
The tests that covers many possible models imported from ONNX or ROOT have the issue that they includes all emitted code in the compiled executables. This means that one gets a build failure on the first model that generated invalid code, and that was it. Therefore, it's difficult to debug what is going wrong.
This commit suggests include the generated code with the interpreter instead. Then, one can check for each individual model if the code was valid, and if not, skip over to the next test a print the emitted code that failed to compile.
It has some performance overhead, but the tests still only take about 6 seconds. The drastically improved debugging experience justifies these few extra seconds spent on testing. But it total, testing is still faster, because compilation of
the test executable takes about 12 seconds, and now it doesn't need to be rebuilt at test time.
This was motivated by the effort to refactor the SOFIE-emitted code to make it differentiable with Clad.