Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Continue building base/k8s when tag version is below v20 #15638

Merged
merged 10 commits into from
Apr 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/docker_build_images.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
name: Docker Build Images
name: Docker Build Images (v20+)
on:
push:
branches:
- main
tags:
- '*'
- 'v[2-9][0-9]*.*' # run only on tags greater or equal to v20.0.0

concurrency:
group: format('{0}-{1}', ${{ github.ref }}, 'Docker Build Images')
group: format('{0}-{1}', ${{ github.ref }}, 'Docker Build Images (v20+)')
cancel-in-progress: true

permissions: read-all
Expand Down Expand Up @@ -77,6 +77,7 @@ jobs:

build_and_push_components:
name: Build and push vitess components Docker images
needs: build_and_push_lite
runs-on: gh-hosted-runners-16cores-1
if: github.repository == 'vitessio/vitess'

Expand Down
259 changes: 259 additions & 0 deletions .github/workflows/docker_build_old_base.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,259 @@
name: Docker Build Base (<v20)
on:
push:
Comment on lines +1 to +3
Copy link
Member Author

@frouioui frouioui Apr 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pure copy of what we used to have before #15620, only the workflow name and the tags-ignore field on lines 4 and 5 changed.

tags-ignore:
- 'v[2-9][0-9]*.*' # ignore all the tags greater or equal to v20.0.0

concurrency:
group: format('{0}-{1}', ${{ github.ref }}, 'Docker Build Base (<v20)')
cancel-in-progress: true

permissions: read-all

jobs:
build_and_push_base:
name: Build and push vitess/base Docker images
runs-on: gh-hosted-runners-16cores-1
if: github.repository == 'vitessio/vitess'

strategy:
fail-fast: true
matrix:
branch: [ latest, mysql57, percona57, percona80 ]

steps:
- name: Check out code
uses: actions/checkout@v4

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Set Dockerfile path
run: |
if [[ "${{ matrix.branch }}" == "latest" ]]; then
echo "DOCKERFILE=./docker/base/Dockerfile" >> $GITHUB_ENV
else
echo "DOCKERFILE=./docker/base/Dockerfile.${{ matrix.branch }}" >> $GITHUB_ENV
fi

- name: Build and push on main
if: github.ref == 'refs/heads/main'
uses: docker/build-push-action@v5
with:
context: .
file: ${{ env.DOCKERFILE }}
push: true
tags: vitess/base:${{ matrix.branch }}

######
# All code below only applies to new tags
######

- name: Get the Git tag
if: startsWith(github.ref, 'refs/tags/')
run: echo "TAG_NAME=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV

- name: Set Docker tag name
if: startsWith(github.ref, 'refs/tags/') && matrix.branch == 'latest'
run: |
if [[ "${{ matrix.branch }}" == "latest" ]]; then
echo "DOCKER_TAG=vitess/base:${TAG_NAME}" >> $GITHUB_ENV
fi

- name: Build and push on tags
if: startsWith(github.ref, 'refs/tags/') && matrix.branch == 'latest'
uses: docker/build-push-action@v5
with:
context: .
file: ${{ env.DOCKERFILE }}
push: true
tags: ${{ env.DOCKER_TAG }}

build_and_push_k8s:
needs: build_and_push_base
name: Build and push vitess/k8s image
runs-on: gh-hosted-runners-16cores-1
if: github.repository == 'vitessio/vitess'

strategy:
fail-fast: true
matrix:
debian: [ bullseye, bookworm ]

steps:
- name: Check out code
uses: actions/checkout@v4

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Set Docker context path
run: |
echo "DOCKER_CTX=./docker/k8s" >> $GITHUB_ENV

- name: Build and push on main latest tag
if: github.ref == 'refs/heads/main' && matrix.debian == 'bookworm'
uses: docker/build-push-action@v5
with:
context: ${{ env.DOCKER_CTX }}
push: true
tags: vitess/k8s:latest
build-args: |
VT_BASE_VER=latest
DEBIAN_VER=${{ matrix.debian }}-slim

- name: Build and push on main debian specific tag
if: github.ref == 'refs/heads/main'
uses: docker/build-push-action@v5
with:
context: ${{ env.DOCKER_CTX }}
push: true
tags: vitess/k8s:latest-${{ matrix.debian }}
build-args: |
VT_BASE_VER=latest
DEBIAN_VER=${{ matrix.debian }}-slim

######
# All code below only applies to new tags
######

- name: Get the Git tag
if: startsWith(github.ref, 'refs/tags/')
run: echo "TAG_NAME=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV

# We push git-tag-based k8s image to three tags, i.e. for 'v19.0.0' we push to:
#
# vitess/k8s:v19.0.0 (DOCKER_TAG_DEFAULT_DEBIAN)
# vitess/k8s:v19.0.0-bookworm (DOCKER_TAG)
# vitess/k8s:v19.0.0-bullseye (DOCKER_TAG)
#
- name: Set Docker tag name
if: startsWith(github.ref, 'refs/tags/')
run: |
echo "DOCKER_TAG_DEFAULT_DEBIAN=vitess/k8s:${TAG_NAME}" >> $GITHUB_ENV
echo "DOCKER_TAG=vitess/k8s:${TAG_NAME}-${{ matrix.debian }}" >> $GITHUB_ENV

# Build and Push component image to DOCKER_TAG, applies to both debian version
- name: Build and push on tags using Debian extension
if: startsWith(github.ref, 'refs/tags/')
uses: docker/build-push-action@v5
with:
context: ${{ env.DOCKER_CTX }}
push: true
tags: ${{ env.DOCKER_TAG }}
build-args: |
VT_BASE_VER=${{ env.TAG_NAME }}
DEBIAN_VER=${{ matrix.debian }}-slim

# Build and Push component image to DOCKER_TAG_DEFAULT_DEBIAN, only applies when building the default Debian version (bookworm)
# It is fine to build a second time here when "matrix.debian == 'bookworm'" as we have cached the first build already
- name: Build and push on tags without Debian extension
if: startsWith(github.ref, 'refs/tags/') && matrix.debian == 'bookworm'
uses: docker/build-push-action@v5
with:
context: ${{ env.DOCKER_CTX }}
push: true
tags: ${{ env.DOCKER_TAG_DEFAULT_DEBIAN }}
build-args: |
VT_BASE_VER=${{ env.TAG_NAME }}
DEBIAN_VER=${{ matrix.debian }}-slim


build_and_push_components:
needs: build_and_push_k8s
name: Build and push vitess components Docker images
runs-on: gh-hosted-runners-16cores-1
if: github.repository == 'vitessio/vitess'

strategy:
fail-fast: true
matrix:
debian: [ bullseye, bookworm ]
component: [ vtadmin, vtorc, vtgate, vttablet, mysqlctld, mysqlctl, vtctl, vtctlclient, vtctld, logrotate, logtail, vtbackup, vtexplain ]

steps:
- name: Check out code
uses: actions/checkout@v4

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Set Docker context path
run: |
echo "DOCKER_CTX=./docker/k8s/${{ matrix.component }}" >> $GITHUB_ENV

- name: Build and push on main latest tag
if: github.ref == 'refs/heads/main' && matrix.debian == 'bookworm'
uses: docker/build-push-action@v5
with:
context: ${{ env.DOCKER_CTX }}
push: true
tags: vitess/${{ matrix.component }}:latest
build-args: |
VT_BASE_VER=latest
DEBIAN_VER=${{ matrix.debian }}-slim

- name: Build and push on main debian specific tag
if: github.ref == 'refs/heads/main'
uses: docker/build-push-action@v5
with:
context: ${{ env.DOCKER_CTX }}
push: true
tags: vitess/${{ matrix.component }}:latest-${{ matrix.debian }}
build-args: |
VT_BASE_VER=latest
DEBIAN_VER=${{ matrix.debian }}-slim

######
# All code below only applies to new tags
######

- name: Get the Git tag
if: startsWith(github.ref, 'refs/tags/')
run: echo "TAG_NAME=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV

# We push git-tag-based images to three tags, i.e. for 'v19.0.0' we push to:
#
# vitess/${{ matrix.component }}:v19.0.0 (DOCKER_TAG_DEFAULT_DEBIAN)
# vitess/${{ matrix.component }}:v19.0.0-bookworm (DOCKER_TAG)
# vitess/${{ matrix.component }}:v19.0.0-bullseye (DOCKER_TAG)
#
- name: Set Docker tag name
if: startsWith(github.ref, 'refs/tags/')
run: |
echo "DOCKER_TAG_DEFAULT_DEBIAN=vitess/${{ matrix.component }}:${TAG_NAME}" >> $GITHUB_ENV
echo "DOCKER_TAG=vitess/${{ matrix.component }}:${TAG_NAME}-${{ matrix.debian }}" >> $GITHUB_ENV

# Build and Push component image to DOCKER_TAG, applies to both debian version
- name: Build and push on tags using Debian extension
if: startsWith(github.ref, 'refs/tags/')
uses: docker/build-push-action@v5
with:
context: ${{ env.DOCKER_CTX }}
push: true
tags: ${{ env.DOCKER_TAG }}
build-args: |
VT_BASE_VER=${{ env.TAG_NAME }}
DEBIAN_VER=${{ matrix.debian }}-slim

# Build and Push component image to DOCKER_TAG_DEFAULT_DEBIAN, only applies when building the default Debian version (bookworm)
# It is fine to build a second time here when "matrix.debian == 'bookworm'" as we have cached the first build already
- name: Build and push on tags without Debian extension
if: startsWith(github.ref, 'refs/tags/') && matrix.debian == 'bookworm'
uses: docker/build-push-action@v5
with:
context: ${{ env.DOCKER_CTX }}
push: true
tags: ${{ env.DOCKER_TAG_DEFAULT_DEBIAN }}
build-args: |
VT_BASE_VER=${{ env.TAG_NAME }}
DEBIAN_VER=${{ matrix.debian }}-slim
71 changes: 71 additions & 0 deletions .github/workflows/docker_build_old_lite.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Docker Build Lite (<v20)
on:
push:
Comment on lines +1 to +3
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

tags-ignore:
- 'v[2-9][0-9]*.*' # ignore all the tags greater or equal to v20.0.0

concurrency:
group: format('{0}-{1}', ${{ github.ref }}, 'Docker Build Lite (<v20)')
cancel-in-progress: true

permissions: read-all

jobs:
build_and_push:
name: Build and push vitess/lite Docker images
runs-on: gh-hosted-runners-16cores-1
if: github.repository == 'vitessio/vitess'

strategy:
fail-fast: true
matrix:
branch: [ latest ]

steps:
- name: Check out code
uses: actions/checkout@v4

- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Set Dockerfile path
run: |
if [[ "${{ matrix.branch }}" == "latest" ]]; then
echo "DOCKERFILE=./docker/lite/Dockerfile" >> $GITHUB_ENV
else
echo "DOCKERFILE=./docker/lite/Dockerfile.${{ matrix.branch }}" >> $GITHUB_ENV
fi

- name: Build and push on main
if: github.ref == 'refs/heads/main'
uses: docker/build-push-action@v5
with:
context: .
file: ${{ env.DOCKERFILE }}
push: true
tags: vitess/lite:${{ matrix.branch }}

- name: Get the Git tag
if: startsWith(github.ref, 'refs/tags/')
run: echo "TAG_NAME=${GITHUB_REF#refs/tags/}" >> $GITHUB_ENV

- name: Set Docker tag name
if: startsWith(github.ref, 'refs/tags/')
run: |
if [[ "${{ matrix.branch }}" == "latest" ]]; then
echo "DOCKER_TAG=vitess/lite:${TAG_NAME}" >> $GITHUB_ENV
else
echo "DOCKER_TAG=vitess/lite:${TAG_NAME}-${{ matrix.branch }}" >> $GITHUB_ENV
fi

- name: Build and push on tags
if: startsWith(github.ref, 'refs/tags/')
uses: docker/build-push-action@v5
with:
context: .
file: ${{ env.DOCKERFILE }}
push: true
tags: ${{ env.DOCKER_TAG }}
1 change: 1 addition & 0 deletions docker/bootstrap/Dockerfile.common
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ FROM --platform=linux/amd64 golang:1.22.1-bullseye
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
# TODO(mberlin): Group these to make it easier to understand which library actually requires them.
ant \
ca-certificates \
chromium \
curl \
default-jdk \
Expand Down
3 changes: 3 additions & 0 deletions docker/lite/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,11 @@ ENV VTDATAROOT /vt/vtdataroot
ENV PATH $VTROOT/bin:$PATH

# Copy artifacts from builder layer.
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=builder --chown=vitess:vitess /vt/install /vt
COPY --from=builder --chown=vitess:vitess /vt/src/vitess.io/vitess/web/vtadmin /vt/web/vtadmin
COPY --from=builder --chown=vitess:vitess /vt/src/vitess.io/vitess/config/init_db.sql /vt/config/
COPY --from=builder --chown=vitess:vitess /vt/src/vitess.io/vitess/config/mycnf /vt/config/

# Create mount point for actual data (e.g. MySQL data dir)
VOLUME /vt/vtdataroot
Expand Down
3 changes: 3 additions & 0 deletions docker/lite/Dockerfile.testing
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,11 @@ ENV VTDATAROOT /vt/vtdataroot
ENV PATH $VTROOT/bin:$PATH

# Copy artifacts from builder layer.
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=builder --chown=vitess:vitess /vt/install /vt
COPY --from=builder --chown=vitess:vitess /vt/src/vitess.io/vitess/web/vtadmin /vt/web/vtadmin
COPY --from=builder --chown=vitess:vitess /vt/src/vitess.io/vitess/config/init_db.sql /vt/config/
COPY --from=builder --chown=vitess:vitess /vt/src/vitess.io/vitess/config/mycnf /vt/config/

# Create mount point for actual data (e.g. MySQL data dir)
VOLUME /vt/vtdataroot
Expand Down
Loading