Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(NODE-6578): add misc tooling to ZSTD repo #36

Merged
merged 10 commits into from
Nov 25, 2024
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/docker/Dockerfile.glibc
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ RUN python3 --version

RUN npm run install-zstd
RUN npm install
RUN npm run prebuild

ARG RUN_TEST
RUN if [ -n "$RUN_TEST" ]; then npm test ; else echo "skipping tests" ; fi

FROM scratch

COPY --from=build /zstd/prebuilds/ /
8 changes: 7 additions & 1 deletion .github/docker/Dockerfile.musl
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,19 @@
ARG PLATFORM=arm64
ARG NODE_VERSION=16.20.1

FROM ${PLATFORM}/node:${NODE_VERSION}-alpine AS node
FROM ${PLATFORM}/node:${NODE_VERSION}-alpine AS build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the motivation behind this change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The last line in the dockerfile:

COPY --from=build /zstd/prebuilds/ /

Requires that tag used in --from is the same as the tag here. I chose to change this to build to match Dockerfile.glibc instead of adjusting the tag below to --from=node.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good


WORKDIR /zstd
COPY . .

RUN apk --no-cache add make g++ libc-dev curl bash python3 py3-pip vim cmake
RUN npm run install-zstd && npm i
RUN npm run prebuild

ARG RUN_TEST
RUN if [ -n "$RUN_TEST" ]; then npm test ; else echo "skipping tests" ; fi


FROM scratch

COPY --from=build /zstd/prebuilds/ /
121 changes: 121 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
on:
pull_request:
branches: [main]
workflow_dispatch: {}
workflow_call: {}

name: Build

jobs:
host_tests:
strategy:
matrix:
os: [macos-latest, windows-2019]
fail-fast: false
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 16.20.1
cache: "npm"
registry-url: "https://registry.npmjs.org"

- name: Install zstd
run: npm run install-zstd
shell: bash

- name: install dependencies and compile
run: npm install --loglevel verbose && npm run prebuild
shell: bash

- id: upload
name: Upload prebuild
uses: actions/upload-artifact@v4
with:
name: build-${{ matrix.os }}
path: prebuilds/
if-no-files-found: "error"
retention-days: 1
compression-level: 0

container_tests_glibc:
runs-on: ubuntu-latest
strategy:
matrix:
linux_arch: [s390x, arm64, amd64]
fail-fast: false
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Run Buildx
run: |
docker buildx create --name builder --bootstrap --use
docker buildx build \
--platform linux/${{ matrix.linux_arch }} \
--build-arg="NODE_ARCH=${{ matrix.linux_arch == 'amd64' && 'x64' || matrix.linux_arch }}" \
--build-arg="NODE_VERSION=16.20.1" \
--output type=local,dest=./prebuilds,platform-split=false \
-f ./.github/docker/Dockerfile.glibc \
.

- id: upload
name: Upload prebuild
uses: actions/upload-artifact@v4
with:
name: build-linux-glibc-${{ matrix.linux_arch }}
path: prebuilds/
if-no-files-found: "error"
retention-days: 1
compression-level: 0

container_tests_musl:
runs-on: ubuntu-latest
strategy:
matrix:
linux_arch: [amd64, arm64]
fail-fast: false
steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Run Buildx
run: |
docker buildx create --name builder --bootstrap --use
docker --debug buildx build --progress=plain --no-cache \
--platform linux/${{ matrix.linux_arch }} \
--build-arg="PLATFORM=${{ matrix.linux_arch == 'arm64' && 'arm64v8' || matrix.linux_arch }}" \
--build-arg="NODE_VERSION=16.20.1" \
--output type=local,dest=./prebuilds,platform-split=false \
-f ./.github/docker/Dockerfile.musl \
.

- id: upload
name: Upload prebuild
uses: actions/upload-artifact@v4
with:
name: build-linux-musl-${{ matrix.linux_arch }}
path: prebuilds/
if-no-files-found: "error"
retention-days: 1
compression-level: 0
54 changes: 54 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: "CodeQL"

on:
push:
branches: ["main"]
pull_request:
branches: ["main"]

jobs:
analyze:
name: Analyze (${{ matrix.language }})
runs-on: "ubuntu-latest"
timeout-minutes: 360
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL only takes about 2 minutes, is there a reason we put a timeout here and chose 360?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not really, no.

This is directly copied from mongodb-client-encryption, which was auto-generated by Github for us. I could adjust it if you want

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to change it if it matches the existing code for mongodb-client-encryption!

permissions:
# required for all workflows
security-events: write

# required to fetch internal or private CodeQL packs
packages: read

# only required for workflows in private repositories
actions: read
contents: read

strategy:
fail-fast: false
matrix:
include:
- language: "c-cpp"
build-mode: "manual"
sourceDirectory: "./addon"
- language: "javascript-typescript"
build-mode: "none"
sourceDirectory: "./lib"
steps:
- name: Checkout repository
uses: actions/checkout@v4

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
source-root: ${{ matrix.sourceDirectory }}

- if: matrix.build-mode == 'manual'
shell: bash
run: npm i --ignore-scripts && npm run install-zstd && npm run compile

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
3 changes: 3 additions & 0 deletions .release-please-manifest.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{

Check warning on line 1 in .release-please-manifest.json

View workflow job for this annotation

GitHub Actions / typescript

File ignored by default.
".": "2.0.0-beta.0"
}
3 changes: 2 additions & 1 deletion etc/install-zstd.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ clean_deps() {
download_zstd() {
rm -rf deps
mkdir -p deps/zstd
ZSTD_VERSION=$(node -p "require('./package.json').zstd_version")

curl -L "https://github.com/facebook/zstd/releases/download/v1.5.6/zstd-1.5.6.tar.gz" \
curl -L "https://github.com/facebook/zstd/releases/download/v$ZSTD_VERSION/zstd-$ZSTD_VERSION.tar.gz" \
| tar -zxf - -C deps/zstd --strip-components 1
}

Expand Down
8 changes: 5 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@
"install-zstd": "bash etc/install-zstd.sh",
"check:eslint": "ESLINT_USE_FLAT_CONFIG=false eslint *ts lib/*.js test/*.js .*.json",
"clang-format": "clang-format --style=file:.clang-format --Werror -i addon/*",
"check:clang-format": "clang-format --style=file:.clang-format --dry-run --Werror addon/*"
"check:clang-format": "clang-format --style=file:.clang-format --dry-run --Werror addon/*",
"prebuild": "prebuild --runtime napi --strip --verbose --all"
},
"overrides": {
"prebuild": {
Expand All @@ -49,5 +50,6 @@
"napi_versions": [
4
]
}
}
},
"zstd_version": "1.5.6"
nbbeeken marked this conversation as resolved.
Show resolved Hide resolved
}
14 changes: 14 additions & 0 deletions release-please-config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"pull-request-header": "Please run the release_notes action before releasing to generate release highlights",
"packages": {
".": {
"include-component-in-tag": false,
"changelog-path": "HISTORY.md",
"release-type": "node",
"bump-minor-pre-major": false,
"bump-patch-for-minor-pre-major": false,
"draft": false
}
},
"$schema": "https://raw.githubusercontent.com/googleapis/release-please/main/schemas/config.json"
}
Loading