-
-
Notifications
You must be signed in to change notification settings - Fork 173
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add automated QEMU-based testing to CI passes (x86_64 only for now) #1041
Add automated QEMU-based testing to CI passes (x86_64 only for now) #1041
Conversation
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
e1cd362
to
1fd09ea
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally looks great, thanks so much! I left a few small comments.
One question -- how long does this take? I noticed a 10-min timeout in the action, hence my curiosity. Our existing CI passes already take over 5 min or so for basic PRs, so I'm wondering if there's a way to speed this up, e.g., by caching build results across separate action passes, and/or disabling actions when a PR's changeset doesn't touch files that require them to be re-run.
The most recent commit took 7 min 27 sec. EDIT: the most most recent commit took 5 min 26 sec, without any significant changes so 🤷 .
It's possible, but I'm not sure how significant the speed up will be. I don't think the time it takes to run CI is particularly important. CI speed is only relevant if a cleanup commit was pushed right before the PR is merged but Github has an auto-merge feature. |
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
Added the feature shenanigans in this commit. Not quite sure if that's the best way to do it. |
Well that's my primary use case, I do that for almost every PR so it would be nice to not make CI passes super slow. But nbd, i can wait 5-7 mins.
Ah i see, bitten yet again by the need for features to be additive. Yes, looking at that commit I do agree it's a bit weird, but we should be able to get around that by simply enabling |
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
Should be good. |
Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com>
* Added `qemu_test` application that automatically runs all tests and returns a specific exit code from QEMU based on test output. * Added a `test` Make target that enables and runs `qemu_test`. * Added a CI workflow that runs the `test` target * Changed `test_mlx5`, `test_ixgbe`, and `test_block_io` to return an exit code of 0 if the required devices aren't connected. * Changed `test_identity_mapping`, `test_aligned_page_allocation`, and `test_filerw` to fail rather than print if they encounter an error. * Renamed `tls_test` to `test_tls` so it is detected by `qemu_test`. * Changed `test_channel` and `test_tls` to run all tests rather than specifying specific tests in the arguments. * Renamed `test_serial_echo` to `serial_echo` because it isn't really a test with defined success and failure conditions. * Changed `test_task_cancel` to always return 0, because task cancellation is not yet implemented in the mainline. * Skip `test_channel`, as it currently does not work. Signed-off-by: Klimenty Tsoutsman <klim@tsoutsman.com> 7a8d64a
qemu_test
application that automatically runs all tests and returns a specific exit code from QEMU based on output of tests.test
Make targettest
targettest_mlx5
,test_ixgbe
, andtest_block_io
to return an exit code of 0 if the associated devices aren't connected. An alternative would be to add them to the skipped tests inqemu_test
test_identity_mapping
,test_aligned_page_allocation
, andtest_filerw
to fail rather than print if they encounter an error.tls_test
totest_tls
so it is detected byqemu_test
.test_channel
andtest_tls
to run all tests rather than specifying specific tests in the arguments.test_serial_echo
toserial_echo
because it isn't really a test with defined success and failure conditions.test_task_cancel
to always return 0 as task cancellation is not yet implemented.I've also temporarily added
test_channel
to the list of skipped tests, because there seems to be a bug that causes deadlock. I'll need to look into the bug, but it's outside of the scope of this PR.