Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests(e2e): Bootstrap end-to-end tests #803

Merged
merged 16 commits into from
Oct 19, 2023
Merged

tests(e2e): Bootstrap end-to-end tests #803

merged 16 commits into from
Oct 19, 2023

Commits on Oct 17, 2023

  1. Add e2e helpers

    Add various internal packages that will help us in our e2e workflows:
    
    - CLI command bootstrapping
    - azure command execution
    - remote command execution (SSH + SFTP)
    - inventory/state management
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    8f62415 View commit details
    Browse the repository at this point in the history
  2. Add scripts to build image templates on Azure

    Add a series of scripts to be used for building image templates on
    Azure. This is accomplished through running 3 scripts in succession:
    
    - 00_check_vm_image: acquire the Azure image URN to use as a base image
    - 01_prepare_base_vm: provision required dependencies on the base image
    - 02_create_vm_template: turn the customized VM into an image template
    
    Detailed usage and requirements are described in each of the scripts.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    353da54 View commit details
    Browse the repository at this point in the history
  3. Add action to establish SSTP VPN connection

    This requires versions of ppp and sstp-client which are not yet
    available in the Ubuntu archive so we're temporarily using a PPA.
    
    The calling workflows are responsible for passing the required inputs as
    secrets to the action.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    043b221 View commit details
    Browse the repository at this point in the history
  4. Add workflow to build image templates on Azure

    This workflow runs on a scheduled basis and can also be triggered
    manually if needed.
    
    By default it'll figure out the currently supported Ubuntu versions and
    build templates for them unless suitable templates already exist (in
    other words, templates will only be built when a release transitions
    from development to stable, and when no template - stable or
    developmental - exists for the release at all).
    
    This behavior can be overridden using the manual workflow inputs, which
    provide a way to force refreshing the template builds, and configuring
    the matrix of codenames to build.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    3b5377b View commit details
    Browse the repository at this point in the history
  5. Add scripts to provision client VM on Azure

    This set of scripts is responsible for building the current working
    directory of adsys as a deb package for the given codename, then
    provisioning a VM from our custom VM image template, joining the
    integration tests realm and installing the previously built adsys
    deb(s).
    
    The package build is done via Docker containers and tests are disabled
    both to speed up the build process and to weed out false positives since
    adsys package tests were not written to be run in a Docker container.
    
    For Ubuntu releases where adsys doesn't build cleanly, such as jammy, we
    provide patches to satisfy the build process.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    996a349 View commit details
    Browse the repository at this point in the history
  6. Add integration tests GPOs and conversion script

    The structure of these GPOs is described in the ADSys QA Plan document.
    
    The GPOs are added in XML format to be converted to POL before copying
    them on the AD controller (using the xml_to_pol.py) script.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    0368a3d View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    ea55cb7 View commit details
    Browse the repository at this point in the history
  8. Update go.mod and go.sum

    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    1867076 View commit details
    Browse the repository at this point in the history
  9. Exclude e2e directory from coverage

    We don't measure coverage on end-to-end tests, and the code there
    doesn't run in production, so it shouldn't affect our coverage status.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    3e1199b View commit details
    Browse the repository at this point in the history
  10. Pass DEBIAN_FRONTEND=noninteractive to apt

    And use apt-get build-dep instead of mk-build-deps.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    b1fb1a6 View commit details
    Browse the repository at this point in the history
  11. Rework base image check logic

    Previously, the approach was to:
    - always build an image if no custom image exists
    - build stable image if we only have daily custom images
    - never build again if a custom stable image already exists
    
    Now, we take into consideration the build number of the Azure
    Marketplace VM image, and ensure we refresh our custom images when new
    base images are published. This means we have to track the build number
    as part of the custom image version to ensure idempotency of the build.
    
    Thus, the version format of our custom images becomes the following:
    - X: major version, 0 for development releases, 1 for stable releases
    - Y: minor version, replicates build number of the Marketplace VM, e.g.
         202310110
    - Z: patch version, incremented for consecutive builds of the same minor
         version, starts at 0 (in practice this won't be encountered unless
         we force a build with the --force param)
    
    Additionally, the behavior of the 00_check_vm_image script is to no
    longer error when there is no image to be printed - this is to be able
    to properly differentiate between a valid behavior and an actual error.
    
    Now, the script always exits with 0 unless an actual error occurs. If
    there is no image URN to be printed, stdout will be empty so the
    consumer will know that the image for the given codename is already up
    to date.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    435de3f View commit details
    Browse the repository at this point in the history
  12. Rename to E2E where possible

    And use `keep` instead of `preserve`.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    5c7a44f View commit details
    Browse the repository at this point in the history
  13. Image provisioning updates

    Use eatmydata and the dpkg force-unsafe-io setting to speed up package
    installation during base VM preparation.
    
    Pass --unattended to the realm command when joining the domain.
    
    Pass set -eu to the provision script to make it more resilient.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    399b520 View commit details
    Browse the repository at this point in the history
  14. Remove struct tags from Inventory

    We don't really care how these fields are serialized (as long as they
    are serialized), it's also less work when we decide to add/remove a
    field.
    GabrielNagy committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    14a9875 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2023

  1. Organize suites in subdirectories

    As opposed to a flat structure.
    GabrielNagy committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    f9f6fae View commit details
    Browse the repository at this point in the history
  2. Standardize calls to apt

    Prefer apt-get instead of apt in scripts, and only set DEBIAN_FRONTEND
    where needed.
    GabrielNagy committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    5ca44a6 View commit details
    Browse the repository at this point in the history