Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: optimise ci build time #955

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

ci: optimise ci build time #955

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
2e46f27
ci: reusable setup
nexxeln Feb 14, 2025
8fcc815
fix setup
nexxeln Feb 14, 2025
1d9c191
switch over to blacksmith arm runner
nexxeln Feb 19, 2025
3299327
switch to x64 runner
nexxeln Feb 19, 2025
89c1a55
trying 4vcpu
nexxeln Feb 19, 2025
924a63c
try to make chrome work on arm
nexxeln Feb 19, 2025
75aaaec
cache the chrome binary
nexxeln Feb 19, 2025
3c1d508
install chromium from registry
nexxeln Feb 19, 2025
5d3507a
go back to x64 now with 8vpcu
nexxeln Feb 20, 2025
01f6866
go back to 4vpcu
nexxeln Feb 20, 2025
6bbf135
cache playwright
nexxeln Feb 20, 2025
b523ae0
empty commit
nexxeln Feb 20, 2025
8ad349b
rm playwright caching and add .next caching
nexxeln Feb 20, 2025
9e47554
empty commit
nexxeln Feb 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/actions/setup-playwright/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
name: "Setup Playwright"
description: "Setup Playwright with pnpm (should already be installed)"
runs:
using: "composite"
steps:
- name: Install Playwright
shell: bash
run: pnpm exec playwright install --with-deps chromium
19 changes: 19 additions & 0 deletions .github/actions/setup/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: 'Setup Environment'
description: 'Sets up Node.js and pnpm'
runs:
using: composite
steps:
- name: Install pnpm
uses: pnpm/action-setup@v4
with:
version: 9.3.0

- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'pnpm'

- name: Install dependencies
shell: bash
run: pnpm install --frozen-lockfile
93 changes: 27 additions & 66 deletions .github/workflows/test.yaml
View file
Open in desktop
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the reason why it failed with arm64 is because CI would pull the amd64 and not arm build of chrome. The problem is that ARM builds are not hosted officially by Chrome or MS, but there are community-made builds:

https://github.com/chromium-for-lambda/chromium-binaries/releases/tag/x86_64-amazon-linux-2023-chromium-127.0.6533

Original file line number Diff line number Diff line change
Expand Up @@ -7,20 +7,10 @@ env:

jobs:
coverage:
runs-on: ubuntu-latest
runs-on: blacksmith-4vcpu-ubuntu-2204
steps:
- uses: actions/checkout@v4

- name: Install pnpm
run: npm i -g --force corepack && corepack enable pnpm

- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'pnpm'

- run: pnpm install --frozen-lockfile
- uses: ./.github/actions/setup

- name: Get contract addresses
run: 'parallel --lb --halt now,success=1,fail=1 ::: \
Expand All @@ -30,20 +20,19 @@ jobs:
- run: pnpm test:coverage

build-stateless:
runs-on: ubuntu-latest
runs-on: blacksmith-4vcpu-ubuntu-2204
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup

- name: Install pnpm
run: npm i -g --force corepack && corepack enable pnpm

- name: Install Node.js
uses: actions/setup-node@v4
- name: Cache Next.js build
uses: useblacksmith/cache@v5
with:
node-version: 20
cache: 'pnpm'

- run: pnpm install --frozen-lockfile
path: |
${{ github.workspace }}/.next/cache
key: ${{ runner.os }}-nextjs-${{ hashFiles('**/pnpm-lock.yaml') }}-${{ hashFiles('**/*.js', '**/*.jsx', '**/*.ts', '**/*.tsx') }}
restore-keys: |
${{ runner.os }}-nextjs-${{ hashFiles('**/pnpm-lock.yaml') }}-

- name: Get contract addresses
run: 'parallel --lb --halt now,success=1,fail=1 ::: \
Expand All @@ -63,20 +52,19 @@ jobs:
path: stateless-build.tar

build-stateful:
runs-on: ubuntu-latest
runs-on: blacksmith-4vcpu-ubuntu-2204
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup

- name: Install pnpm
run: npm i -g --force corepack && corepack enable pnpm

- name: Install Node.js
uses: actions/setup-node@v4
- name: Cache Next.js build
uses: useblacksmith/cache@v5
with:
node-version: 20
cache: 'pnpm'

- run: pnpm install --frozen-lockfile
path: |
${{ github.workspace }}/.next/cache
key: ${{ runner.os }}-nextjs-${{ hashFiles('**/pnpm-lock.yaml') }}-${{ hashFiles('**/*.js', '**/*.jsx', '**/*.ts', '**/*.tsx') }}
restore-keys: |
${{ runner.os }}-nextjs-${{ hashFiles('**/pnpm-lock.yaml') }}-

- name: Build stateful and export
run: pnpm build && pnpm export
Expand All @@ -94,23 +82,14 @@ jobs:
name: stateless (${{matrix.shard}})
needs: build-stateless
timeout-minutes: 20
runs-on: ubuntu-latest
runs-on: blacksmith-4vcpu-ubuntu-2204
strategy:
matrix:
shard: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]
steps:
- uses: actions/checkout@v4

- name: Install pnpm
run: npm i -g --force corepack && corepack enable pnpm

- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'pnpm'

- run: pnpm install --frozen-lockfile
- uses: ./.github/actions/setup
- uses: ./.github/actions/setup-playwright

- run: pnpm rebuild -r

Expand All @@ -124,9 +103,6 @@ jobs:
- name: Untar files
run: tar -xvf stateless-build.tar

- name: Install playwright
run: npx playwright install chromium

- name: Run tests
run: |
PLAYWRIGHT_SHARD=${{matrix.shard}} PLAYWRIGHT_TOTAL=${{strategy.job-total}} pnpm e2e:ci --no-build
Expand All @@ -143,25 +119,14 @@ jobs:
name: stateful (${{matrix.shard}})
needs: build-stateful
timeout-minutes: 10
runs-on: ubuntu-latest
runs-on: blacksmith-4vcpu-ubuntu-2204
strategy:
matrix:
shard: [1, 2, 3]

steps:
- uses: actions/checkout@v4
- run: ./scripts/check-chrome.sh

- name: Install pnpm
run: npm i -g --force corepack && corepack enable pnpm

- name: Install Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'pnpm'

- run: pnpm install --frozen-lockfile
- uses: ./.github/actions/setup
- uses: ./.github/actions/setup-playwright

- run: pnpm rebuild -r

Expand All @@ -174,21 +139,17 @@ jobs:
- name: Untar files
run: tar -xvf stateful-build.tar

- name: Install playwright
run: npx playwright install chromium

- name: Run tests
run: |
parallel --lb --halt now,success=1,fail=1 ::: \
"pnpm wrangle" \
"pnpm wait-on http://127.0.0.1:8788 && npx playwright test --shard=${{matrix.shard}}/${{strategy.job-total}} --project=stateful"
env:
# NEXT_PUBLIC_GRAPH_URI: https://api.studio.thegraph.com/query/49574/ensholesky/version/latest
SECRET_WORDS: ${{ secrets.SECRET_WORDS }}

- uses: actions/upload-artifact@v4
if: always()
with:
name: stateful-report-${{matrix.shard}}
path: playwright-report/
retention-days: 30
retention-days: 30