Build Qt Using AWS

Previous: Continuously running unit tests in a local VM

minicoin supports local VM providers such as VirtualBox or VMware Fusion, but through the relevant vagrant plugins it also can talk to machines running in a public cloud. For AWS in particular, minicoin includes several features that make it easy to set up and manage a cloud instance.

This tutorial assumes that you have an AWS account.

Preparations

First, let's install the AWS vagrant plugin and the AWS cli. The former is done via

$ vagrant plugin install vagrant-aws

and for the latter, use your system's package manager, e.g. probably one of

$ brew awscli
$ sudo apt-get install -y awscli
$ pip3 install aws-cli

or perhaps

C:\> msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi

With the credentials for your AWS account at hand (you'll need the Access Key ID and the Secret, which you can generate and get from the AWS web console), run

$ aws configure

to authenticate. As part of the configuration you'll have to select a default region for your account as well; some of the AMIs that are used by default or in this tutorial are only available in the eu-west-1 region, i.e. Ireland.

Defining a cloud machine

Let's add a machine that runs in AWS:

$ minicoin machine add linux-aws

This will add a ready-made machine to the current user's minicoin configuration. Alternatively, we can add such a machine to our ~/minicoin/minicoin.yml file:

machines:
  - name: linux-aws
    box: minicoin/linux-cloud
    provider: aws

This basic configuration makes sure that a minicoin up linux-aws always uses the aws provider to bring up a machine based on the minicoin/linux-cloud vagrant box. That box uses the AMI for Ubuntu Server 20.04, and sets the AWS instance type to t3.2xlarge, which is a rather beefy (and thus also not super-cheap) machine with 8 vCPUs and 32 GB of RAM.

Unless we used the ready-made machine, we need to install some software on it as part of the provisioning step before we can do anything with it:

    roles:
      - qt-builder
      - linux-desktop
      - vnc-server

This will install the build tools and libraries required to build Qt; the default Linux desktop for Ubuntu; and a VNC server through which we can connect to the UI of the machine.

Thanks to our earlier configuration of minicoin, the mutagen role for sharing our ~/qt/dev branch with all minicoin machines is implicitly set. Configurations such as cpu or memory do not apply to cloud VMs.

AWS instances are based on machine images (AMIs), which come with just enough storage for the OS and perhaps some additional packages. In this case, the system disk is merely 8 GB large, which is enough for the software we want to install through our provisioning steps, but not enough for building Qt.

Adding storage space requires a provider specific configuration in the roles list:

      - aws:
          storage: 100

This adds one disk with 100 GB capacity to the AWS instance. minicoin will automatically initialize the disk with a file system, and mount it as the home partition on Linux (on a Windows machine, it would become drive D:, which minicoin would then use as the working directory when running jobs).

Run the job

Let's make sure that everything is set up correctly:

$ minicoin validate linux-aws

minicoin will now validate the AWS account, and if necessary create a minicoin security group that grants access to machines from your current network through a list of ports (SSH, WinRM, VNC, RDP).

Together with key-based authentication to the machine itself, this makes the machine sufficiently secure for our purposes (but don't use such a machine to run critical production workloads or to store confidential information).

With everything working, we can now run a build on this machine just like with any local VM:

$ cd ~/qt/dev
$ minicoin run build --target Widgets linux-aws

Note how the run command brings up the machine for us, and then waits for the mutagen file system syncing to finish before starting the job. Depending on how many submodules of Qt you have checked out in your ~/qt/dev worktree, the job might take a little. But your machine won't spend any CPU or RAM on it, so you can kick of a parallel build on your host, and on your locally running VMs.