Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[draft] CI Improvements #125

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
Draft

[draft] CI Improvements #125

wants to merge 12 commits into from

Conversation

brenns10
Copy link
Member

@brenns10 brenns10 commented Oct 30, 2024 •
edited
Loading

This pull request contains several improvements to our CI.

  1. Removing tox, which hasn't been useful, and it's a bit confusing and inflexible. This simplifies a fair bit of our CI and speeds things up too.
  2. Enabling CTF testing in the two live test case type (litevm and heavyvm). We weren't actually doing any CTF testing against live kernels in CI previously.
  3. Dropping dependencies for the test suite. The goal is that the tests that developers may want to run (litevm, vmcore) should only require python, drgn, and pytest, all of which could even be satisfied by the Yum versions. To do this, I've dropped the pytest-cov (which was causing issues in CI for no benefit), junitparser (it can be replaced by a few lines of XML parsing).
  4. Discovered that CTF is broken for UEK4, enough that we shouldn't mark it as supported.
  5. Added CTF testing in Github, by building drgn from source. (This necessitated removing Python 3.6 from the Github tests, but is well covered elsewhere)

It's not ready. Still remaining:

  • separating out the OCI SDK from the vmcore test runner

@brenns10
testing: remove coverage
bd70e2c 
The Gitlab CI tests have intermittently failed for vmcore tests due to
race conditions with the parallel execution for coverage. The fact of
the matter is that the coverage information is not currently reported by
any of our CI tests, and no developers use it locally. It's just more
trouble than it's worth, so remove it.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
testing: rpm: call sys.executable
6ae9c36 
Using "python3" runs the risk of a different python version getting
used. The whole point of testing/rpm.py is to run *within* the current
Python environment, so use sys.executable to make that certain.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
testing: unpin oci-cli
82a9d6d 
There have been many releases since the annotated ones. It should be
fine to unpin now.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
tests: heavyvm: run with CTF as well
728df1c 
Currently the only CTF-related tests for CI are the ones which run
against vmcores. The heavyvm tests can easily support CTF, so run it. It
should not add enough time to cause any issues with CI.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
testing: heavyvm: skip CTF for UEK4
c815e74 
The CTF data for UEK4 is fully broken; we cannot support it. Skip
running live tests for UEK4 CTF.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
conftest: fix test suite names for heavyvm CTF
05c2e97 
The test suite name was being reported as "CTF" only if the tests were
run against vmcores. With heavyvm (and possibly litevm) tests running
against CTF, this means that the XML test reports no longer contain the
correct debuginfo in their name.

Live tests also simply report "live" which makes it very difficult to
understand which test suite is for which version. Report the uname in
this case, which includes both UEK and OL version, though it's not as
nice as simply saying "OLX UEKY".

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
testing: stop using tox
5a72332 
The value proposition of tox is allowing you to run your tests against a
lot of different Python versions at the same time. It can be really
useful, but honestly it hasn't been helpful in drgn-tools. We typically
only have one (maybe two) Python versions available, and frequently we
can only run tests against one anyway. Testing multiple Python versions
is done in Github CI, but tricking tox into only running tests for one
version is actually a pain!

All told, it's caused more pain than gain, so let's do away with it in
testing. The tox.ini still exists and is used for generating docs, but
we can eliminate that smaller issue in a subsequent commit.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
testing: remove junitparser
22bf9fe 
This is a totally unnecessary dependency. We only need to have one
operation: concatenating two test results. Thankfully, that can be done
directly with the python XML support. Drop the dependency.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
testing.rpm: remove the --coverage and --xml
7cb28e7 
The coverage option has been removed from the rest of the tests. It
looks like the xml option was simply never implemented.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
Mark UEK4 as incompatible with CTF
f767864 
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Oct 30, 2024
@brenns10 brenns10 force-pushed the remove_coverage branch 3 times, most recently from 98c828b to 6b3df3f Compare October 30, 2024 22:33
@brenns10
actions: Remove 3.6 testing, add CTF testing
720269a 
Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
@brenns10
testing: litevm: fix litevm runner
ec0f0cd 
It got broken by the removal of tox. It also needs to handle CTF in the
same run for some cases too. Fix it all up.

Signed-off-by: Stephen Brennan <stephen.s.brennan@oracle.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@brenns10