tests(e2e): Bootstrap end-to-end tests #803
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GabrielNagy
force-pushed
the
prepare-e2e
branch
from
d8e0d91
to
5d37bd8
Compare
Codecov Report
@@ Coverage Diff @@
## main #803 +/- ##
=======================================
Coverage 86.07% 86.07%
=======================================
Files 77 77
Lines 8552 8552
=======================================
Hits 7361 7361
Misses 868 868
Partials 323 323 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
GabrielNagy
force-pushed
the
prepare-e2e
branch
from
5d37bd8
to
e2624c4
Compare
Add various internal packages that will help us in our e2e workflows: - CLI command bootstrapping - azure command execution - remote command execution (SSH + SFTP) - inventory/state management
Add a series of scripts to be used for building image templates on Azure. This is accomplished through running 3 scripts in succession: - 00_check_vm_image: acquire the Azure image URN to use as a base image - 01_prepare_base_vm: provision required dependencies on the base image - 02_create_vm_template: turn the customized VM into an image template Detailed usage and requirements are described in each of the scripts.
This requires versions of ppp and sstp-client which are not yet available in the Ubuntu archive so we're temporarily using a PPA. The calling workflows are responsible for passing the required inputs as secrets to the action.
This workflow runs on a scheduled basis and can also be triggered manually if needed. By default it'll figure out the currently supported Ubuntu versions and build templates for them unless suitable templates already exist (in other words, templates will only be built when a release transitions from development to stable, and when no template - stable or developmental - exists for the release at all). This behavior can be overridden using the manual workflow inputs, which provide a way to force refreshing the template builds, and configuring the matrix of codenames to build.
This set of scripts is responsible for building the current working directory of adsys as a deb package for the given codename, then provisioning a VM from our custom VM image template, joining the integration tests realm and installing the previously built adsys deb(s). The package build is done via Docker containers and tests are disabled both to speed up the build process and to weed out false positives since adsys package tests were not written to be run in a Docker container. For Ubuntu releases where adsys doesn't build cleanly, such as jammy, we provide patches to satisfy the build process.
The structure of these GPOs is described in the ADSys QA Plan document. The GPOs are added in XML format to be converted to POL before copying them on the AD controller (using the xml_to_pol.py) script.
We don't measure coverage on end-to-end tests, and the code there doesn't run in production, so it shouldn't affect our coverage status.
And use apt-get build-dep instead of mk-build-deps.
Previously, the approach was to:
- always build an image if no custom image exists
- build stable image if we only have daily custom images
- never build again if a custom stable image already exists
Now, we take into consideration the build number of the Azure
Marketplace VM image, and ensure we refresh our custom images when new
base images are published. This means we have to track the build number
as part of the custom image version to ensure idempotency of the build.
Thus, the version format of our custom images becomes the following:
- X: major version, 0 for development releases, 1 for stable releases
- Y: minor version, replicates build number of the Marketplace VM, e.g.
202310110
- Z: patch version, incremented for consecutive builds of the same minor
version, starts at 0 (in practice this won't be encountered unless
we force a build with the --force param)
Additionally, the behavior of the 00_check_vm_image script is to no
longer error when there is no image to be printed - this is to be able
to properly differentiate between a valid behavior and an actual error.
Now, the script always exits with 0 unless an actual error occurs. If
there is no image URN to be printed, stdout will be empty so the
consumer will know that the image for the given codename is already up
to date.
And use `keep` instead of `preserve`.
Use eatmydata and the dpkg force-unsafe-io setting to speed up package installation during base VM preparation. Pass --unattended to the realm command when joining the domain. Pass set -eu to the provision script to make it more resilient.
We don't really care how these fields are serialized (as long as they are serialized), it's also less work when we decide to add/remove a field.
GabrielNagy
force-pushed
the
prepare-e2e
branch
from
e2624c4
to
14a9875
Compare
|
@didrocks if you would be so kind as to have a look on the last 5 commits I just pushed, full diff is here https://github.com/ubuntu/adsys/compare/e2624c41eef4285f103e7ac51d5e4d24ae4f85f5..14a98755049024d0e48edd6acc420dec0151f14e It's what we discussed in our calls, hopefully I haven't missed anything. An updated run of the E2E image template build workflow can be seen here. |
|
Pushed a change to reflect what we discussed regarding the structure of the cmd files. |
didrocks
requested changes
Oct 18, 2023
As opposed to a flat structure.
Prefer apt-get instead of apt in scripts, and only set DEBIAN_FRONTEND where needed.
GabrielNagy
force-pushed
the
prepare-e2e
branch
from
b6e158b
to
5ca44a6
Compare
didrocks
approved these changes
Oct 18, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a comprehensive set of changes best reviewed commit by commit.
It handles the following parts of the e2e test process:
The following parts are not covered:
End-to-end tests are structured in suites of steps meant to be run in succession to accomplish a task, e.g. creating an image template or provisioning the client VM. Between steps, the current state is saved to a file (
inventory.yaml) which will be used in the subsequent step of the suite. These suites are written in a semi-portable way so they can easily be replicated on any developer's machine (provided they have a VPN connection set up).A run of the GitHub workflow that builds VM templates can be seen here: https://github.com/GabrielNagy/adsys/actions/runs/6440737893
Fixes UDENG-1358 and UDENG-1441 and UDENG-1362