Skip to content

Commit

Permalink
Switch from docker-compose to "docker compose" (#6599)
Browse files Browse the repository at this point in the history
Switch from standalone docker-compose binary to the "docker compose" subcommand everywhere.
  • Loading branch information
pgporada committed Jan 30, 2023
1 parent 9d3f7d8 commit c091e64
Show file tree
Hide file tree
Showing 12 changed files with 45 additions and 41 deletions.
18 changes: 9 additions & 9 deletions .github/workflows/boulder-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

name: Boulder CI

# Controls when the action will run.
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches:
branches:
- main
- release-branch-*
pull_request:
Expand Down Expand Up @@ -38,7 +38,7 @@ jobs:
BOULDER_TOOLS_TAG:
- go1.19.2_2023-01-10
- go1.19.5_2023-01-10
# Tests command definitions. Use the entire docker-compose command you want to run.
# Tests command definitions. Use the entire "docker compose" command you want to run.
tests:
# Run ./test.sh --help for a description of each of the flags.
- "./t.sh --lints --generate --make-artifacts"
Expand All @@ -48,8 +48,8 @@ jobs:
# production can be made in `test/config-next/<component>.json`.
#
# Testing DB Schema Changes:
# Database migrations in `sa/_db-next/migrations` are only performed
# when `docker-compose` is called using `-f docker-compose.yml -f
# Database migrations in `sa/_db-next/migrations` are only performed
# when `docker compose` is called using `-f docker-compose.yml -f
# docker-compose.next.yml`.
- "./tn.sh --integration"
- "./t.sh --unit --enable-race-detection"
Expand All @@ -59,7 +59,7 @@ jobs:
# because it needs to fetch packages from GitHub et. al., which
# is incompatible with the DNS server override in the boulder
# container (used for service discovery).
- "docker-compose run --use-aliases netaccess ./test.sh --gomod-vendor"
- "docker compose run --use-aliases netaccess ./test.sh --gomod-vendor"

# This sets the docker image tag for the boulder-tools repository to
# use in tests. It will be set appropriately for each tag in the list
Expand Down Expand Up @@ -93,9 +93,9 @@ jobs:
run: echo "Using BOULDER_TOOLS_TAG ${BOULDER_TOOLS_TAG}"

# Pre-pull the docker containers before running the tests.
- name: docker-compose pull
run: docker-compose pull
- name: docker compose pull
run: docker compose pull

# Run the test matrix. This will run
- name: "Run Test: ${{ matrix.tests }}"
run: ${{ matrix.tests }}
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/update-psl.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,12 @@ jobs:

- run: git checkout -b ${{ steps.branch.outputs.name }}

# We run these inside docker-compose to ensure we use the same Go version
# We run these inside "docker compose" to ensure we use the same Go version
# as elsewhere. They're run inside the netaccess container so they can
# download the dependency files.
- run: docker-compose run netaccess go get github.com/weppos/publicsuffix-go@main
- run: docker-compose run netaccess go mod vendor
- run: docker-compose run netaccess go mod tidy
- run: docker compose run netaccess go get github.com/weppos/publicsuffix-go@main
- run: docker compose run netaccess go mod vendor
- run: docker compose run netaccess go mod tidy

- run: git diff --numstat vendor/github.com/weppos/publicsuffix-go/publicsuffix/rules.go | awk '{ print "::set-output name=body::" $1 " additions and " $2 " removals." }'
id: diff
Expand Down
24 changes: 12 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,48 +121,48 @@ non-obvious ways.
To start Boulder in a Docker container, run:

```shell
docker-compose up
docker compose up
```

To run our standard battery of tests (lints, unit, integration):

```shell
docker-compose run --use-aliases boulder ./test.sh
docker compose run --use-aliases boulder ./test.sh
```

To run all unit tests:

```shell
docker-compose run --use-aliases boulder ./test.sh --unit
docker compose run --use-aliases boulder ./test.sh --unit
```

To run specific unit tests (example is of the ./va directory):

```shell
docker-compose run --use-aliases boulder ./test.sh --unit --filter=./va
docker compose run --use-aliases boulder ./test.sh --unit --filter=./va
```

To run all integration tests:

```shell
docker-compose run --use-aliases boulder ./test.sh --integration
docker compose run --use-aliases boulder ./test.sh --integration
```

To run specific integration tests (example runs TestAkamaiPurgerDrainQueueFails and TestWFECORS):

```shell
docker-compose run --use-aliases boulder ./test.sh --filter TestAkamaiPurgerDrainQueueFails/TestWFECORS
docker compose run --use-aliases boulder ./test.sh --filter TestAkamaiPurgerDrainQueueFails/TestWFECORS
```

To get a list of available integration tests:

```shell
docker-compose run --use-aliases boulder ./test.sh --list-integration-tests
docker compose run --use-aliases boulder ./test.sh --list-integration-tests
```

The configuration in docker-compose.yml mounts your boulder checkout at
/boulder so you can edit code on your host and it will be immediately
reflected inside the Docker containers run with docker-compose.
reflected inside the Docker containers run with `docker compose`.

If you have problems with Docker, you may want to try [removing all
containers and
Expand All @@ -186,27 +186,27 @@ environmental variable using -e (replace 172.17.0.1 with the host IPv4
address found in the command above)

```shell
docker-compose run --use-aliases -e FAKE_DNS=172.17.0.1 --service-ports boulder ./start.py
docker compose run --use-aliases -e FAKE_DNS=172.17.0.1 --service-ports boulder ./start.py
```

Running tests without the `./test.sh` wrapper:

Run all unit tests

```shell
docker-compose run --use-aliases boulder go test -p 1 ./...
docker compose run --use-aliases boulder go test -p 1 ./...
```

Run unit tests for a specific directory:

```shell
docker-compose run --use-aliases boulder go test <DIRECTORY>
docker compose run --use-aliases boulder go test <DIRECTORY>
```

Run integration tests (omit `--filter <REGEX>` to run all):

```shell
docker-compose run --use-aliases boulder python3 test/integration-test.py --chisel --gotest --filter <REGEX>
docker compose run --use-aliases boulder python3 test/integration-test.py --chisel --gotest --filter <REGEX>
```

Boulder's default VA configuration (`test/config/va.json`) is configured to
Expand Down
10 changes: 5 additions & 5 deletions docs/load-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Set up a SoftHSM instance running pkcs11-daemon on some remote host with more
CPUs than your local machine. Easiest way to do this is to clone the Boulder
repo, and on the remote machine run:

remote-machine$ docker-compose run -p 5657:5657 bhsm
remote-machine$ docker compose run -p 5657:5657 bhsm

Check that the port is open:

Expand All @@ -22,12 +22,12 @@ Edit docker-compose.yml to change these in the "boulder" section's "env":

Run the pkcs11key benchmark to check raw signing speed at various settings for SESSIONS:

local-machine$ docker-compose run -e SESSIONS=4 -e MODULE=/usr/local/lib/softhsm/libsofthsm2.so --entrypoint /go/src/github.com/letsencrypt/pkcs11key/test.sh boulder
local-machine$ docker compose run -e SESSIONS=4 -e MODULE=/usr/local/lib/softhsm/libsofthsm2.so --entrypoint /go/src/github.com/letsencrypt/pkcs11key/test.sh boulder

Initialize the tokens for use by Boulder:

local-machine$ docker-compose run --entrypoint "softhsm --module /usr/local/lib/softhsm/libsofthsm2.so --init-token --pin 5678 --so-pin 1234 --slot 0 --label intermediate" boulder
local-machine$ docker-compose run --entrypoint "softhsm --module /usr/local/lib/softhsm/libsofthsm2.so --init-token --pin 5678 --so-pin 1234 --slot 1 --label root" boulder
local-machine$ docker compose run --entrypoint "softhsm --module /usr/local/lib/softhsm/libsofthsm2.so --init-token --pin 5678 --so-pin 1234 --slot 0 --label intermediate" boulder
local-machine$ docker compose run --entrypoint "softhsm --module /usr/local/lib/softhsm/libsofthsm2.so --init-token --pin 5678 --so-pin 1234 --slot 1 --label root" boulder

Configure Boulder to always consider all OCSP responses instantly stale, so it
will sign new ones as fast as it can. Edit "ocspMinTimeToExpiry" in
Expand All @@ -37,7 +37,7 @@ test/config/ocsp-updater.json (or test/config-next/ocsp-updater.json):

Run a local Boulder instance:

local-machine$ docker-compose up
local-machine$ docker compose up

Issue a bunch of certificates with chisel.py, ideally a few thousand
(corresponding to the default batch size of 5000 in ocsp-updater.json, to make
Expand Down
2 changes: 1 addition & 1 deletion docs/redis.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ script that handles all that for you: `test/redis-cli.sh`. First, make sure your
redis cluster is running:

```
docker-compose up bredis_clusterer
docker compose up bredis_clusterer
```

Then, in a different window, run:
Expand Down
2 changes: 1 addition & 1 deletion start.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,6 @@
print("\nstopping servers.")
except OSError as v:
# Ignore EINTR, which happens when we get SIGTERM or SIGINT (i.e. when
# someone hits Ctrl-C after running docker-compose up or start.py.
# someone hits Ctrl-C after running `docker compose up` or start.py.
if v.errno != errno.EINTR:
raise
4 changes: 3 additions & 1 deletion t.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@
#
# Outer wrapper for invoking test.sh inside docker-compose.
#

if type realpath >/dev/null 2>&1 ; then
cd "$(realpath -- $(dirname -- "$0"))"
fi

# Use a predictable name for the container so we can grab the logs later
# for use when testing logs analysis tools.
docker rm boulder_tests
exec docker-compose run --name boulder_tests boulder ./test.sh "$@"
exec docker compose run --name boulder_tests boulder ./test.sh "$@"
6 changes: 3 additions & 3 deletions test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ USAGE="$(cat -- <<-EOM
Usage:
Boulder test suite CLI, intended to be run inside of a Docker container:
docker-compose run --use-aliases boulder ./$(basename "${0}") [OPTION]...
docker compose run --use-aliases boulder ./$(basename "${0}") [OPTION]...
With no options passed, runs standard battery of tests (lint, unit, and integration)
Expand Down Expand Up @@ -154,7 +154,7 @@ while getopts lueciosvgmnhp:f:-: OPT; do
done
shift $((OPTIND-1)) # remove parsed options and args from $@ list

# The list of segments to run. Order doesn't matter. Note: gomod-vendor
# The list of segments to run. Order doesn't matter. Note: gomod-vendor
# is specifically left out of the defaults, because we don't want to run
# it locally (it could delete local state).
if [ -z "${RUN[@]+x}" ]
Expand Down Expand Up @@ -241,7 +241,7 @@ if [[ "${RUN[@]}" =~ "$STAGE" ]] ; then
fi

# Test that just ./start.py works, which is a proxy for testing that
# `docker-compose up` works, since that just runs start.py (via entrypoint.sh).
# `docker compose up` works, since that just runs start.py (via entrypoint.sh).
STAGE="start"
if [[ "${RUN[@]}" =~ "$STAGE" ]] ; then
print_heading "Running Start Test"
Expand Down
2 changes: 1 addition & 1 deletion test/boulder-tools/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ apt-get autoremove -y libssl-dev ruby-dev cmake pkg-config libtool autoconf auto
apt-get clean -y

# Tell git to trust the directory where the boulder repo volume is mounted
# by docker-compose.
# by `docker compose`.
git config --global --add safe.directory /boulder

rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
4 changes: 2 additions & 2 deletions test/entrypoint-netaccess.sh
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
#!/bin/bash
# For the boulder container, we want to run entrypoint.sh and start.py by
# default (when no command is passed on a "docker-compose run" command line).
# default (when no command is passed on a "docker compose run" command line).
# However, we want the netaccess container to run nothing by default.
# Otherwise it would race with boulder container's entrypoint.sh to run
# migrations, and one or the other would fail randomly. Also, it would compete
# with the boulder container for ports. This is a variant of entrypoint.sh that
# exits if it is not given an argument.
if [[ "$@" = "" ]]; then
echo "Not needed as part of 'docker-compose up'. Exiting normally."
echo "Not needed as part of 'docker compose up'. Exiting normally."
exit 0
fi
"$@"
2 changes: 1 addition & 1 deletion test/redis-cli.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,4 @@ ARGS="--tls \
--user replication-user \
--pass 435e9c4225f08813ef3af7c725f0d30d263b9cd3"

exec docker-compose exec bredis_clusterer redis-cli "${ARGS}" "${@}"
exec docker compose exec bredis_clusterer redis-cli "${ARGS}" "${@}"
4 changes: 3 additions & 1 deletion tn.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@
#
# Outer wrapper for invoking test.sh with config-next inside docker-compose.
#

if type realpath >/dev/null 2>&1 ; then
cd "$(realpath -- $(dirname -- "$0"))"
fi
exec docker-compose -f docker-compose.yml -f docker-compose.next.yml run boulder ./test.sh "$@"

exec docker compose -f docker-compose.yml -f docker-compose.next.yml run boulder ./test.sh "$@"

0 comments on commit c091e64

Please sign in to comment.