Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(web): dockerize web application #1253

Merged
merged 5 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci_web.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,13 @@ jobs:
run: yarn i18n --fail-on-update
- name: Build
run: yarn build

# TODO: Remove after dockerizing the web.
- name: Pack
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/release'
run: mv dist reearth-cms-web && tar -zcvf reearth-cms-web.tar.gz reearth-cms-web

# TODO: Remove after dockerizing the web.
- uses: actions/upload-artifact@v4
if: github.ref == 'refs/heads/main' || github.ref == 'refs/heads/release'
with:
Expand Down
57 changes: 47 additions & 10 deletions .github/workflows/deploy_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,29 @@ concurrency:
group: ${{ github.workflow }}-${{ github.event.workflow_run.head_branch }}-${{github.event.workflow_run.name}}
cancel-in-progress: true
env:
GCP_REGION: us-central1

# TODO: Remove after dockerizing the web.
GCS_DEST: gs://cms.test.reearth.dev

# server
IMAGE: reearth/reearth-cms:nightly
IMAGE_NAME: us-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/reearth/reearth-cms:nightly
IMAGE_GCP: us-central1-docker.pkg.dev/reearth-oss/reearth/reearth-cms:nightly
SERVER_IMAGE: reearth/reearth-cms:nightly
SERVER_IMAGE_NAME: us-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/reearth/reearth-cms:nightly
SERVER_IMAGE_GCP: us-central1-docker.pkg.dev/reearth-oss/reearth/reearth-cms:nightly

# web
WEB_IMAGE: reearth/reearth-cms-web:nightly
WEB_IMAGE_NAME: us-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/reearth/reearth-cms-worker:nightly
WEB_IMAGE_GCP: us-central1-docker.pkg.dev/reearth-oss/reearth/reearth-cms-worker:nightly

# worker
WORKER_IMAGE: reearth/reearth-cms-worker:nightly
WORKER_IMAGE_NAME: us-central1-docker.pkg.dev/${{ secrets.GCP_PROJECT_ID }}/reearth/reearth-cms-worker:nightly
WORKER_IMAGE_GCP: us-central1-docker.pkg.dev/reearth-oss/reearth/reearth-cms-worker:nightly
GCP_REGION: us-central1
GCS_DEST: gs://cms.test.reearth.dev

jobs:
deploy_web:
# TODO: Remove after dockerizing the web.
deploy_web_gcs:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

I changed it so that I could remove this block entirely after verifying our new Docker image without renaming the web (Docker) job.

name: Deploy web to test env
if: github.event.repository.full_name == 'reearth/reearth-cms' && github.event.workflow_run.name == 'ci-web' && github.event.workflow_run.conclusion != 'failure' && github.event.workflow_run.head_branch == 'main'
runs-on: ubuntu-latest
Expand Down Expand Up @@ -67,13 +77,40 @@ jobs:
run: gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
- name: docker push
run: |
docker pull $IMAGE
docker tag $IMAGE $IMAGE_GCP
docker push $IMAGE_GCP
docker pull $SERVER_IMAGE
docker tag $SERVER_IMAGE $SERVER_IMAGE_GCP
docker push $SERVER_IMAGE_GCP
- name: Deploy to Cloud Run
run: |
gcloud run deploy reearth-cms-backend \
--image $IMAGE_GCP \
--image $SERVER_IMAGE_GCP \
--region $GCP_REGION \
--platform managed \
--quiet
deploy_web:
name: Deploy web to test env
runs-on: ubuntu-latest
if: github.event.repository.full_name == 'reearth/reearth-cms' && github.event.workflow_run.name == 'web-build' && github.event.workflow_run.conclusion != 'failure' && github.event.workflow_run.head_branch == 'main'
steps:
- uses: actions/checkout@v4

- uses: google-github-actions/auth@v2
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workload Identity is not used... Later we want to change

- name: Set up Cloud SDK
uses: google-github-actions/setup-gcloud@v2
- name: Configure docker
run: gcloud auth configure-docker us-central1-docker.pkg.dev --quiet
- name: docker push
run: |
docker pull $WEB_IMAGE
docker tag $WEB_IMAGE $WEB_IMAGE_GCP
docker push $WEB_IMAGE_GCP
- name: Deploy to Cloud Run
run: |
gcloud run deploy reearth-cms-web \
--image $WEB_IMAGE_GCP \
--region $GCP_REGION \
--platform managed \
--quiet
Expand Down
107 changes: 107 additions & 0 deletions .github/workflows/web_build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
name: web-build
on:
workflow_run:
workflows: [ci-web]
types: [completed]
branches: [main, release]
concurrency:
group: ${{ github.workflow }}-${{ github.event.workflow_run.head_branch }}
cancel-in-progress: true

jobs:
info:
name: Collect information
runs-on: ubuntu-latest
if: github.event.workflow_run.conclusion != 'failure' && github.event.repository.full_name == 'reearth/reearth-cms' && (github.event.workflow_run.head_branch == 'release' || !startsWith(github.event.head_commit.message, 'v'))
outputs:
sha_short: ${{ steps.info.outputs.sha_short }}
new_tag: ${{ steps.info.outputs.new_tag }}
new_tag_short: ${{ steps.info.outputs.new_tag_short }}
name: ${{ steps.info.outputs.name }}
steps:
- name: checkout
uses: actions/checkout@v4
- name: Fetch tags
run: git fetch --prune --unshallow --tags
- name: Get info
id: info
# The tag name should be retrieved lazily, as tagging may be delayed.
env:
BRANCH: ${{ github.event.workflow_run.head_branch }}
run: |
echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"
if [[ "$BRANCH" = "release" ]]; then
TAG=$(git tag --points-at HEAD)
if [[ ! -z "$TAG" ]]; then
echo "::set-output name=new_tag::$TAG"
echo "::set-output name=new_tag_short::${TAG#v}"
else
echo "::set-output name=name::rc"
fi
else
echo "::set-output name=name::nightly"
fi
- name: Show info
env:
SHA_SHORT: ${{ steps.info.outputs.sha_short }}
NEW_TAG: ${{ steps.info.outputs.new_tag }}
NEW_TAG_SHORT: ${{ steps.info.outputs.new_tag_short }}
NAME: ${{ steps.info.outputs.name }}
run: echo "sha_short=$SHA_SHORT, new_tag=$NEW_TAG, new_tag_short=$NEW_TAG_SHORT, name=$NAME"

docker:
name: Build and push Docker image
runs-on: ubuntu-latest
needs:
- info
if: needs.info.outputs.name || needs.info.outputs.new_tag
env:
IMAGE_NAME: reearth/reearth-cms-web
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Get options
id: options
env:
TAG: ${{ needs.info.outputs.tag_short }}
NAME: ${{ needs.info.outputs.name }}
SHA: ${{ needs.info.outputs.sha_short }}
run: |
if [[ -n $TAG ]]; then
PLATFORMS=linux/amd64,linux/arm64
VERSION=$TAG
TAGS=$IMAGE_NAME:$TAG
if [[ ! $TAG =~ '-' ]]; then
TAGS+=,${IMAGE_NAME}:${TAG%.*}
TAGS+=,${IMAGE_NAME}:${TAG%%.*}
TAGS+=,${IMAGE_NAME}:latest
fi
else
PLATFORMS=linux/amd64
VERSION=$SHA
TAGS=$IMAGE_NAME:$NAME
fi
echo "::set-output name=platforms::$PLATFORMS"
echo "::set-output name=version::$VERSION"
echo "::set-output name=tags::$TAGS"
- name: Build and push docker image
uses: docker/build-push-action@v6
with:
context: ./web
platforms: ${{ steps.options.outputs.platforms }}
push: true
build-args: |
GITHUB_SHA=${{ needs.info.outputs.sha_short }}
VERSION=${{ steps.options.outputs.version }}
tags: ${{ steps.options.outputs.tags }}
cache-from: type=gha
cache-to: type=gha,mode=max
11 changes: 11 additions & 0 deletions web/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
*
Copy link
Contributor Author

@KeisukeYamashita KeisukeYamashita Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Allow listing is always good and safe.


!docker/
!src/

!index.html
!i18next-parser.config.js
!tsconfig.json
!package.json
!vite.config.ts
!yarn.lock
43 changes: 43 additions & 0 deletions web/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
FROM node:20.18.0-slim AS builder
WORKDIR /app

ARG NODE_OPTIONS="--max-old-space-size=4096"
ARG GITHUB_SHA
ENV NODE_OPTIONS=$NODE_OPTIONS
ENV GITHUB_SHA=$GITHUB_SHA

RUN --mount=type=bind,source=package.json,target=package.json \
--mount=type=bind,source=yarn.lock,target=yarn.lock \
--mount=type=cache,target=/root/.yarn,sharing=locked \
yarn install --frozen-lockfile --production=false

RUN --mount=type=bind,source=package.json,target=package.json \
--mount=type=bind,source=yarn.lock,target=yarn.lock \
--mount=type=bind,source=index.html,target=index.html \
--mount=type=bind,source=tsconfig.json,target=tsconfig.json \
--mount=type=bind,source=vite.config.ts,target=vite.config.ts \
--mount=type=bind,source=src,target=src \
--mount=type=cache,target=/root/.yarn,sharing=locked \
yarn build

FROM nginx:1.27-alpine
WORKDIR /app

# Quite the Nginx startup logs.
ENV NGINX_ENTRYPOINT_QUIET_LOGS=true

# Default to Cloud Run port.
# Ref: https://cloud.google.com/run/docs/reference/container-contract#port
ENV PORT=8080

# Defaults Google Cloud Load Balancer header.
# Ref: https://cloud.google.com/load-balancing/docs/https#target-proxies
ENV REAL_IP_HEADER=X-Forwarded-For
Comment on lines +29 to +35
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

IMHO, vendor-specific configurations should be configurable since we are an OSS product; in this case, we can pass -e to override on execution.


COPY --from=builder --chown=nginx:nginx /app/dist /usr/share/nginx/html
COPY --chown=nginx:nginx docker/nginx.conf.template /etc/nginx/templates/nginx.conf.template
COPY --chown=nginx:nginx docker/40-envsubst-on-reearth-config.sh /docker-entrypoint.d
COPY --chown=nginx:nginx docker/reearth_config.json.template /opt/reearth-cms/reearth_config.json.template

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["nginx", "-g", "daemon off;"]
8 changes: 8 additions & 0 deletions web/docker/40-envsubst-on-reearth-config.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/sh

set -e

_REEARTH_CONFIG_TEMPLATE_FILE="/opt/reearth-cms/reearth_config.json.template"
_REEARTH_CONFIG_OUTPUT_FILE="/usr/share/nginx/html/reearth_config.json"

envsubst < "$_REEARTH_CONFIG_TEMPLATE_FILE" > "$_REEARTH_CONFIG_OUTPUT_FILE"
35 changes: 35 additions & 0 deletions web/docker/nginx.conf.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
log_format json escape=json '{'
'"body_bytes_sent": "$body_bytes_sent",'
'"http_referer": "$http_referer",'
'"http_user_agent": "$http_user_agent",'
'"remote_ip": "$remote_addr",'
'"remote_user": "$remote_user",'
'"request": "$request",'
'"request_id": "$request_id",'
'"request_method": "$request_method",'
'"request_time": "$request_time",'
'"request_uri": "$request_uri",'
'"server_name": "$server_name",'
'"status": "$status",'
'"time": "$time_iso8601"'
'}';
Comment on lines +1 to +15
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I accessed the container via curl and it will be like this:

{"body_bytes_sent": "739","http_referer": "","http_user_agent": "curl/8.7.1","remote_ip": "xxx.xxx.xxx.xxx","remote_user": "","request": "GET / HTTP/1.1","request_id": "fde3377155ab9765825dff57cb2a83f6","request_method": "GET","request_time": "0.000","request_uri": "/","server_name": "_","status": "200","time": "2024-10-08T22:48:41+00:00"}


real_ip_header ${REAL_IP_HEADER};

server {
listen ${PORT};
root /usr/share/nginx/html;
server_name _;

access_log /dev/stdout json;
error_log /dev/stderr warn;

location / {
try_files $uri /index.html =404;
}

error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
5 changes: 5 additions & 0 deletions web/docker/reearth_config.json.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"auth0Audience": "$AUTH0_AUDIENCE",
"auth0ClientId": "$AUTH0_CLIENT_ID",
"auth0Domain": "$AUTH0_DOMAIN"
}
10 changes: 1 addition & 9 deletions web/vite.config.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
/// <reference types="vite/client" />
/// <reference types="vitest" />

import { execSync } from "child_process";
import { readFileSync } from "fs";
import { resolve } from "path";

Expand All @@ -15,13 +14,6 @@ import { configDefaults } from "vitest/config";

import pkg from "./package.json";

let commitHash = "";
try {
commitHash = execSync("git rev-parse HEAD").toString().trimEnd();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Because we built in Docker, it has no Git information.

} catch {
// noop
}

const cesiumPackageJson = JSON.parse(
readFileSync(resolve(__dirname, "node_modules", "cesium", "package.json"), "utf-8"),
);
Expand All @@ -35,7 +27,7 @@ export default defineConfig({
envPrefix: "REEARTH_CMS_",
define: {
__APP_VERSION__: JSON.stringify(pkg.version),
__REEARTH_COMMIT_HASH__: JSON.stringify(commitHash)
__REEARTH_COMMIT_HASH__: JSON.stringify(process.env.GITHUB_SHA || commitHash)
},
plugins: [
react(),
Expand Down
Loading