-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Switch to CTest #87
Switch to CTest #87
Conversation
* use a test driver to run test cases in parallel * configure test cases with labels and arguments * only show test case output on test failure * allow skipped test cases (error code 5)
The CTest tool is described in more detail in the the "Mastering CMake" book, chapter Testing With CMake and CTest. The CMake community provides detailed instructions on Using CTest and CDash without CMake. I skipped the CDash part, since I don't have much experience with the dashboard and wasn't sure how it would integrate in GitHub Actions. Most pyMBE and ESPResSo users are already familiar with CTest, and its ASCII-based reporting should be sufficient for our needs. Long tests are sorted by decreasing runtime. This way, the first time While this PR adds CMake as a dependency, I have no intention of making pyMBE a CMake project. CMake is required by CTest, but anyone can edit the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you very much for this PR and the useful links @jngrad! I did some testing in my office computer and the new framework for tests seems to work very well and is way faster than before, which will save both on our Github quota and on the time spending debugging code. I also think that it should be clear from your notes how to add new tests with the new framework, but @paobtorres should take a look into this PR since she is the one that is going to develop tests in the near future.
I think that we should note in the requierements that cmake
is a dependency only for testing. Since we are working in a virtual environment it should in principle not be a problem to have it installed by default, but just in case it might be a potential issue for some of our users. I added a similar note on tqdm
which is actually only needed because we use it in some of our sample scripts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, thank you for the detailed explanation @jngrad. I took a look into poetry
and I agree with you: it is an interesting tool but not what we need at the moment. I do not think that it is worth for now to add another layer of complexity to our library. Thanks for the refactoring of the HH tests, the ut.TestCases
sounds like a quite convenient framework. We can try to use it more for our other tests!
Replace the Makefile-based test driver by the CTest driver. Tests can now run in parallel with
make unit_tests -j8
.