diff --git a/CHANGELOG.md b/CHANGELOG.md index 9fe49e4..531e69b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,10 +16,6 @@ ## [1.0.0](https://github.com/agilecustoms/release/compare/v0.17.1...v1.0.0) (2025-08-24) -### ⚠ BREAKING CHANGES - -* first beta - ### Features * add java-version input - pass it into setup-maven-codeartifact action ([e707ff4](https://github.com/agilecustoms/release/commit/e707ff4254776e8a6694e1021713b3122a7d2a5c)) diff --git a/README.md b/README.md index f9792d1..6845106 100644 --- a/README.md +++ b/README.md @@ -4,16 +4,14 @@ Release software artifacts in AWS (S3, ECR, CodeArtifact) and NPM with consisten ![Cover](docs/images/cover.png) -You can release **any combination** of software packages, binary files, docker images and raw repo files +You can release **any combination** of software packages, binary files, docker images, and raw repo files -This is especially useful in microservices where the release is a _binary_ + _IaC_ versioned via git tag +This is especially useful in microservices where the releases are _binary_ + _IaC_ versioned via git tag The action comes with an **ecosystem**: - Terraform modules to provide AWS roles and policies to [read](https://registry.terraform.io/modules/agilecustoms/ci-builder/aws/latest) and [publish](https://registry.terraform.io/modules/agilecustoms/ci-publisher/aws/latest) artifacts -- (Fall 2025) Terraform module to create security-aware GitHub repo -- (Fall 2025) Terraform module to access terraform modules from corporate private GH repos -- GitHub actions to use in build workflows ex. [setup-maven-codeartifact](https://github.com/agilecustoms/setup-maven-codeartifact) -- documentation / examples for all supported [artifact types](./docs/artifact-types/index.md) +- GitHub actions to use in build workflows, e.g., [setup-maven-codeartifact](https://github.com/agilecustoms/setup-maven-codeartifact) +- documentation and examples for all supported [artifact types](./docs/artifact-types/index.md) - [Authorization and Security](./docs/authorization.md) — how to make releases secure, including self-service (dev-releases) - Release workflow [best practices](./docs/best-practices.md) - Articles: [Software distribution in AWS](https://www.linkedin.com/pulse/software-distribution-aws-alexey-chekulaev-ubl0e) @@ -26,18 +24,18 @@ The action comes with an **ecosystem**: - [maintenance releases](./docs/features/maintenance-release.md) — made from branch like `1.x.x` (given `2.x.x` development is in `main`) - [prereleases](./docs/features/prerelease.md) — develop a next (often major, sometimes minor) version, typically made from a branch `next` - [dev-release](./docs/features/dev-release.md) — ability to publish artifacts for dev testing when testing on a local machine is impossible/complicated -- [idempotency](./docs/features/idempotency.md) — ability to re-run the action w/o side effects +- [idempotency](./docs/features/idempotency.md) — ability to re-run the action without side effects - GitHub release ## Artifact types ⇔ features -| Artifact type | floating tags | idempotency | dev-release — auto cleanup | -|---------------------------------------------------------------------------|---------------|-------------|----------------------------| -| [git](./docs/artifact-types/git.md) | ✅ | ✅ | ✅ — ❌️ | -| [AWS S3](./docs/artifact-types/aws-s3.md) | ✅ | ✅ | ✅ — ✅ | -| [AWS ECR](./docs/artifact-types/aws-ecr.md) | ✅ | ✅ | ✅ — ✅ | -| [AWS CodeArtifact maven](./docs/artifact-types/aws-codeartifact-maven.md) | ❌️ | ⚠️ | ❌️ | -| [npmjs](./docs/artifact-types/npmjs.md) | ✅ | ⚠️ | ❌️ | +| Artifact type | floating tags | idempotency | dev-release | auto cleanup | +|---------------------------------------------------------------------------|---------------|-------------|-------------|--------------| +| [git](./docs/artifact-types/git.md) | ✅ | ✅ | ✅ ️ | ✅ | +| [AWS S3](./docs/artifact-types/aws-s3.md) | ✅ | ✅ | ✅ | ✅ | +| [AWS ECR](./docs/artifact-types/aws-ecr.md) | ✅ | ✅ | ✅ | ✅ | +| [AWS CodeArtifact maven](./docs/artifact-types/aws-codeartifact-maven.md) | ❌️ | ⚠️ | ❌️ | N/A | +| [npmjs](./docs/artifact-types/npmjs.md) | ✅ | ⚠️ | ❌️ | N/A | _See the respective artifact type to learn about idempotency limitations ⚠️_ @@ -45,7 +43,7 @@ _See the respective artifact type to learn about idempotency limitations ⚠️_ All examples are structured by [artifact types](./docs/artifact-types/index.md) and [features](./README.md#features) -The example below shows how to publish binaries in S3 +The example below shows how to publish binaries in S3: ```yaml name: Release @@ -58,10 +56,10 @@ on: jobs: Release: runs-on: ubuntu-latest - environment: release + environment: release # has secret GH_TOKEN - a PAT with permission to bypass branch protection rule permissions: - contents: read - id-token: write + contents: read # to checkout code + id-token: write # to assume AWS role via OIDC steps: # (example) package AWS Lambda code as a zip archive in ./s3 directory @@ -77,61 +75,61 @@ jobs: ``` Assume: -- you store artifacts in AWS account "Dist" and its number stored in GH org variable `AWS_ACCOUNT_DIST` +- you store artifacts in AWS account "Dist" and its number is stored in GH org variable `AWS_ACCOUNT_DIST` - you have an S3 bucket `mycompany-dist` in `us-east-1` region -- there is a role `ci/publisher` with permissions to upload files in this S3 bucket +- there is a role `ci/publisher` with permissions to upload files in this S3 bucket and trust policy that allows to assume this role from GH action - you have repo `mycompany/myapp` -- current release branch `main` has protection rule so all changes must be done via PR +- current release branch `main` has a protection rule so all changes must be done via PR - you have a GH environment `release` associated with branch `main` -- There is a PAT (Personal Access Token) with permission to bypass branch protection rule. This PAT is stored as environment secret `GH_TOKEN` -- latest tag is `v1.2.3` +- There is a PAT (Personal Access Token) with permission to bypass the branch protection rule. This PAT is stored as environment secret `GH_TOKEN` +- the latest tag is `v1.2.3` Scenario: -- developer made a feature branch and a commit with message `feat: new-feature` +- a developer made a feature branch and a commit with message `feat: new-feature` (alternatively use input [version-bump](./docs/features/version-generation.md#version-bump) for default minor/patch bump) - the developer created and merged a PR which triggered a `Release` workflow - build steps (omitted) produced a directory `./s3` with files (like a zip archive for AWS Lambda) The action will: -- generate a new version `v1.3.0` +- generate a new version `v1.3.0` (minor bump based on commit message prefix `feat:`) - upload files from `./s3` directory to S3 bucket `mycompany-dist` at path `/myapp/v1.3.0/` - update `CHANGELOG.md` with release notes - push tags `v1.3.0`, `v1.3`, `v1` and `latest` to the remote repository -- create GH Release tied to tag `v1.3.0` +- create a GH Release tied to tag `v1.3.0` ## Inputs -_There are no required inputs. The action only controls that combination of inputs is valid_ - -| Name | Default | Description | -|-----------------------------|-------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| aws-account | | AWS account to publish artifacts to. Not needed if there are no artifacts, just git tag | -| aws-codeartifact-domain | | AWS CodeArtifact domain name, ex. `mycompany` | -| aws-codeartifact-repository | | AWS CodeArtifact repository name, ex. `maven` | -| aws-codeartifact-maven | | If true, then publish maven artifacts to AWS CodeArtifact | -| aws-ecr | | If true, then push docker image to AWS ECR, [example](./docs/artifact-types/aws-ecr.md) | -| aws-region | | AWS region | -| aws-role | | AWS IAM role to assume to publish, ex. `/ci/publisher` | -| aws-s3-bucket | | AWS S3 bucket to upload artifacts to | -| aws-s3-dir | | Allows to specify AWS S3 bucket directory to upload artifacts to. By default just place in `bucket/{repo-name}/{version}/*` | -| changelog-file | CHANGELOG.md | Changelog file path. Pass empty string to disable changelog generation | -| changelog-title | # Changelog | Title of the changelog file (first line of the file) | -| dev-branch-prefix | feature/ | Allows to enforce branch prefix for dev-releases, this help to write auto-disposal rules. Empty string disables enforcement | -| dev-release | false | Allows to create temporary named release, mainly for dev testing. Implementation is different for all supported artifact types | -| floating-tags | true | When next version to be released is `1.2.4`, then also release `1.2`, `1` and `latest`. Not desired for public terraform modules | -| npm-extra-deps | | Additional npm dependencies, needed to use non-default commit analyzer preset, ex. `conventional-changelog-conventionalcommits@9.1.0` use white space or new line to specify multiple deps (extremely rare) | -| npm-visibility | public | Used together with env variable `NPM_TOKEN` to publish npm package. Specifies package visibility: public or private (not tested yet), [example](./docs/artifact-types/npmjs.md) | -| node-version | 22 | Node.js version to publish npm packages, default is 22 because it is highest pre-cached in Ubuntu 24 | -| java-version | 21 | Java version to use with input `aws-codeartifact-maven`, [example](./docs/artifact-types/aws-codeartifact-maven.md) | -| pre-publish-script | | Custom sh script that allows to update version in arbitrary file(s), not only files governed by build tool (pom.xml, package.json, etc). In this script you can use variable `$version`. See example in [npmjs](./docs/artifact-types/npmjs.md) | -| release-branches | (see description) | Semantic-release [branches](https://semantic-release.gitbook.io/semantic-release/usage/configuration?utm_source=chatgpt.com#branches), mainly used to support [maintenance releases](./docs/features/maintenance-release.md) and [prereleases](./docs/features/prerelease.md) | -| release-channel | | Repeat `.releaserc.json` `channel` behavior when set `version` explicitly, see [floating-tags](./docs/features/floating-tags.md) for details | -| release-gh | true | If true, then create a GitHub release | -| release-plugins | (see description) | Semantic-release "plugins" configuration, see [details](./docs/configuration.md) | -| summary | (see description) | Text to print in workflow 'Release summary'. Default is `### Released ${version}`. Set empty string to omit summary generation | -| tag-format | v${version} | Default tag format is `v1.0.0` _(default is in code level, not input value)_. Use `${version}` to remove `v` prefix | -| version | | [Explicit version](./docs/features/version-generation.md#explicit-version) to use instead of auto-generation | -| version-bump | | Allows to [bump a version](./docs/features/version-generation.md#version-bump) w/o semantic commits | +_There are no required inputs. The action only controls that the combination of inputs is valid_ + +| Name | Default | Description | +|-----------------------------|-------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| aws-account | | AWS account to publish artifacts to. Not needed if there are no artifacts, just a git tag | +| aws-codeartifact-domain | | AWS CodeArtifact domain name, e.g., `mycompany` | +| aws-codeartifact-repository | (see description) | AWS CodeArtifact repository name, e.g., `maven`. If `aws-codeartifact-maven` is true, then default to `maven` | +| aws-codeartifact-maven | | If true, then publish maven artifacts to AWS CodeArtifact | +| aws-ecr | | If true, then push docker image to AWS ECR, [example](./docs/artifact-types/aws-ecr.md) | +| aws-region | | AWS region | +| aws-role | | AWS IAM role to assume to publish, e.g., `ci/publisher` | +| aws-s3-bucket | | AWS S3 bucket to upload artifacts to | +| aws-s3-dir | | Allows you to specify AWS S3 bucket directory to upload artifacts to. By default, just place in `bucket/{repo-name}/{version}/*` | +| changelog-file | CHANGELOG.md | Changelog file path. Pass an empty string to disable changelog generation | +| changelog-title | # Changelog | Title of the changelog file (first line of the file) | +| dev-branch-prefix | feature/ | Allows you to enforce branch prefix for dev-releases; this helps to write auto-disposal rules. Empty string disables enforcement | +| dev-release | false | Allows you to create a temporary named release, mainly for dev testing. Implementation is different for all supported artifact types | +| floating-tags | true | When next version to be released is `1.2.4`, then also release `1.2`, `1` and `latest`. Not desired for public terraform modules | +| npm-extra-deps | | Additional npm dependencies needed to use non-default commit analyzer preset, e.g., `conventional-changelog-conventionalcommits@9.1.0`. Use white space or new line to specify multiple deps (extremely rare) | +| npm-visibility | public | Used together with env variable `NPM_TOKEN` to publish npm package. Specifies package visibility: public or private (not tested yet). [Example](./docs/artifact-types/npmjs.md) | +| node-version | 22 | Node.js version to publish npm packages. Default is 22 because it is the highest pre-cached in Ubuntu 24 (latest at time of writing) | +| java-version | 21 | Java version to use with input `aws-codeartifact-maven`. [Example](./docs/artifact-types/aws-codeartifact-maven.md) | +| pre-publish-script | | Custom shell script that allows you to update version in arbitrary file(s), not only files governed by build tool (pom.xml, package.json, etc.). In this script you can use variable `$version`. See example in [npmjs](./docs/artifact-types/npmjs.md) | +| release-branches | (see description) | Semantic-release [branches](https://semantic-release.gitbook.io/semantic-release/usage/configuration#branches), mainly used to support [maintenance releases](./docs/features/maintenance-release.md) and [prereleases](./docs/features/prerelease.md) | +| release-channel | | Repeat `.releaserc.json` `channel` behavior when `version` is set explicitly. See [floating-tags](./docs/features/floating-tags.md) for details | +| release-gh | true | If true, then create a GitHub release | +| release-plugins | (see description) | Semantic-release "plugins" configuration, see [details](./docs/configuration.md) | +| summary | (see description) | Text to print in step summary. Can use `${version}` placeholder. Default is `### Released ${version}`. Set to an empty string to omit summary generation | +| tag-format | v${version} | Default tag format is `v1.0.0` _(default is in code level, not input value)_. Use `${version}` to remove `v` prefix | +| version | | [Explicit version](./docs/features/version-generation.md#explicit-version) to use instead of auto-generation | +| version-bump | | Allows you to [bump a version](./docs/features/version-generation.md#version-bump) without semantic commits | ## Outputs @@ -141,10 +139,10 @@ _There are no required inputs. The action only controls that combination of inpu ## Environment variables -| Name | Description | -|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| GH_TOKEN | Takes GH PAT with permission to bypass branch protection rule. Required if `release-gh: true` (default). See details in [Authorization and Security](./docs/authorization.md) | -| NPM_TOKEN | If specified — publish npm package in npmjs repo. See [details](./docs/artifact-types/npmjs.md) | +| Name | Description | +|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------| +| GH_TOKEN | Takes GH PAT with permission to bypass the branch and tags protection rules. See details in [Authorization and Security](./docs/authorization.md) | +| NPM_TOKEN | If specified, publish npm package in npmjs repo. See [details](./docs/artifact-types/npmjs.md) | ## Misc diff --git a/action.yml b/action.yml index 84ee8d6..60e7a98 100644 --- a/action.yml +++ b/action.yml @@ -1,6 +1,10 @@ name: 'Consistent Release' author: 'Alexey Chekulaev' -description: 'Release software artifacts in AWS (S3, ECR, CodeArtifact) and NPM with consistent versioning' +description: > + Automate consistent, secure releases of software artifacts to AWS (S3, ECR, CodeArtifact) and npm. + Supports semantic versioning, changelog generation, floating tags, prereleases, and dev-releases. + Publish binaries, Docker images, IaC, and raw files with idempotent, repeatable workflows — ideal + for microservices and multi-artifact projects. branding: icon: package color: green @@ -12,13 +16,13 @@ inputs: description: 'AWS region' required: false aws-role: - description: 'IAM role to assume to publish, ex. `/ci/publisher`' + description: 'IAM role to assume to publish, e.g., `ci/publisher`' required: false aws-codeartifact-domain: - description: 'CodeArtifact domain name, ex. `mycompany`' + description: 'CodeArtifact domain name, e.g., `mycompany`' required: false aws-codeartifact-repository: - description: 'CodeArtifact repository name, ex. `maven`' + description: 'CodeArtifact repository name, e.g., `maven`' required: false aws-codeartifact-maven: description: 'If true, then publish maven artifacts to AWS CodeArtifact' @@ -30,7 +34,7 @@ inputs: description: 'Required if uploading to S3 (i.e. s3/ directory exists)' required: false aws-s3-dir: - description: 'Allows to specify S3 bucket directory to upload artifacts to. By default just place in `bucket/{repo-name}/{version}/*`' + description: 'Allows you to specify S3 bucket directory to upload artifacts to. By default, just place in `bucket/{repo-name}/{version}/*`' required: false changelog-file: description: 'Changelog file path. Pass empty string to disable changelog generation' @@ -41,15 +45,15 @@ inputs: required: false default: '# Changelog' dev-branch-prefix: - description: 'Allows to enforce branch prefix for dev-releases, this help to write auto-disposal rules. Empty string disables enforcement' + description: 'Allows you to enforce branch prefix for dev-releases; this helps to write auto-disposal rules. Empty string disables enforcement' required: false default: 'feature/' dev-release: - description: 'Allows to create temporary named release, mainly for dev testing. Implementation is different for all supported artifact types' + description: 'Allows you to create a temporary named release, mainly for dev testing. Implementation is different for all supported artifact types' required: false default: 'false' floating-tags: - description: 'When next version to be released is 1.2.4, then also release 1, 1.2 and latest. Not desired for public terraform modules' + description: 'When the next version to be released is 1.2.4, then also release 1, 1.2, and latest. Not desired for public terraform modules' required: false default: 'true' java-version: @@ -57,7 +61,7 @@ inputs: required: false default: '21' node-version: - description: 'Node.js version to publish npm packages, default is 22 because it is highest pre-cached in Ubuntu 24' + description: 'Node.js version to publish npm packages, default is 22 because it is highest pre-cached in Ubuntu 24 (latest at time of writing)' required: false default: '22' npm-extra-deps: @@ -68,7 +72,7 @@ inputs: required: false default: 'public' pre-publish-script: - description: 'sh script that allows to update version in custom file(s), not only files governed by build tool (pom.xml, package.json, etc)' + description: 'Shell script that allows you to update version in custom file(s), not only files governed by build tool (pom.xml, package.json, etc)' required: false release-branches: description: 'semantic-release "branches" configuration' @@ -84,17 +88,17 @@ inputs: description: 'semantic-release plugins configuration' required: false summary: - description: 'If true, then print summary to the GitHub Actions summary page' + description: 'Text to print in step summary. Can use `${version}` placeholder. Default is `### Released ${version}`. Set empty string to omit summary generation' required: false default: '### Released ${version}' tag-format: - description: 'By-default tag (version) has format `v1.0.0`. Use `${version}` to remove `v` prefix' + description: 'By default, the tag (version) has format `v1.0.0`. Use `${version}` to remove `v` prefix' required: false version: description: 'Explicit version to use instead of auto-generation' required: false version-bump: - description: 'Allows to bump a version w/o semantic commits' + description: 'Allows you to bump a version without semantic commits' required: false runs: @@ -117,7 +121,7 @@ runs: if: inputs.release-gh == 'true' && inputs.dev-release != 'true' && !env.GH_TOKEN shell: bash run: | - echo "Error: env: GH_TOKEN is required to create GH release, you can pass $ {{ github.token }} or PAT" >&2; exit 1 + echo "Error: env variable GH_TOKEN is required to create GH release, you can pass $ {{ github.token }} or PAT" >&2; exit 1 - name: Validate npm visibility if : env.NPM_TOKEN && ! (inputs.npm-visibility == 'public' || inputs.npm-visibility == 'private') @@ -161,8 +165,8 @@ runs: fi if [ "${{ inputs.aws-codeartifact-maven }}" = "true" ]; then - if [ -z "${{ inputs.aws-codeartifact-domain }}" ] || [ -z "${{ inputs.aws-codeartifact-repository }}" ]; then - echo "Error: aws-codeartifact-domain and aws-codeartifact-repository must be provided when aws-codeartifact-maven is true" >&2 + if [ -z "${{ inputs.aws-codeartifact-domain }}" ]; then + echo "Error: aws-codeartifact-domain must be provided when aws-codeartifact-maven is true" >&2 exit 1 fi fi @@ -174,9 +178,9 @@ runs: ## Phase 2 - Release generation - # by default (job permissions: contents: write) .git/config already pre-configured with a ${{ github.token }} + # by default (job permissions: contents: write) .git/config already pre-configured with a ${{ github.token }} see misc/.git-config for example # alternatively user may want to use an explicit PAT, then need to configure git to use it. - # Besides 'git push', release_gen _probes_ git push, that's why it is in front of release_gen + # Besides 'git push', the release_gen _probes_ git push, that's why it is in front of release_gen - name: Git authorization if: env.GH_TOKEN shell: bash @@ -248,7 +252,7 @@ runs: aws-region: ${{ inputs.aws-region }} aws-role: ${{ inputs.aws-role }} aws-codeartifact-domain: ${{ inputs.aws-codeartifact-domain }} - aws-codeartifact-repository: ${{ inputs.aws-codeartifact-repository }} + aws-codeartifact-repository: ${{ inputs.aws-codeartifact-repository || 'maven' }} aws-login: false # we already logged in AWS in a previous step java-version: ${{ inputs.java-version }} diff --git a/docs/about.md b/docs/about.md index 49f29c1..bd15458 100644 --- a/docs/about.md +++ b/docs/about.md @@ -6,14 +6,14 @@ In 2023, I (Alexey Chekulaev) started to work on a project that consists of mult First, I did not find a good GH action to upload files in S3, so I did one myself. Then I felt a lack of GH action to publish Maven packages in AWS CodeArtifact, so I had to develop two more actions: one to publish and one to resolve existing packages. -Spring 2025 I started my second project, the number of services grew and so grew a volume of similar code in release pipelines. +In spring 2025 I started my second project, the number of services grew and so grew a volume of similar code in release pipelines. Then I combined all of them into a single action `agilecustoms/gha-release`. But then (summer 2025) I decided to make it public and extracted stuff not specific to AgileCustoms into a separate action `agilecustoms/release` ## Current state Currently (July 2025) this action is being used in two private AgileCustoms projects with over 20+ repositories, -which allows me (author) to have good coverage on different release scenarios +which allows me as the author to have good coverage of different release scenarios Did I think about a plugin system? Yes, I did. Having every type of artifact as a plugin would be great. The problem is that right now this GH action is a combination of other GH actions, my custom GH actions (composite and Node.js ones) and some shell scripts. diff --git a/docs/artifact-types/aws-ecr.md b/docs/artifact-types/aws-ecr.md index c0afe06..1ff3852 100644 --- a/docs/artifact-types/aws-ecr.md +++ b/docs/artifact-types/aws-ecr.md @@ -6,9 +6,9 @@ _Note: all examples use shared patterns: two workflows: Build and Release — co - [AWS Lambda running Spring Boot application in Docker](#aws-lambda-running-spring-boot-application-in-docker) - [Workflow reuse](#workflow-reuse) - [Application code and IaC](#application-code-and-iac) - - [dev-release](#dev-release) - [explicit version](#explicit-version) - [setup-maven-codeartifact](#setup-maven-codeartifact) +- [dev-release](#dev-release) ## AWS Lambda running Spring Boot application in Docker @@ -101,22 +101,6 @@ module "env_cleanup" { Note file `infrastructure/vars.tf` has variable `aVersion` which is used in `infrastructure/lambda.tf` to set ECR image URL -### dev-release - -ECR supports [dev-releases](../features/dev-release.md): Docker image gets published with tag equal to branch name -(branch name `feature/login` becomes ECR tag `feature-login`). -`agilecustoms/release` action enforces branch name prefix. Itt can be configured with `dev-branch-prefix` input (default is `feature/`). -ECR allows you to configure lifecycle rules by tag prefix, this allows to remove dev-releases automatically in few days! - -There is one important gotcha: you can't re-run dev-release for immutable ECR repo: -- immutable ECR repo prevents pushing image for existing tag -- tag can be deleted, but IAM doesn't allow to configure permissions selectively for tag prefix, -so if you allow deleting tags — it is a risk that developer can remove some non-dev tag -- if you make ECR repo mutable — it is a risk that developer can override non-dev tag - -So if you need to re-run dev-release shortly (before old tag expired) the workaround is just to create a new branch -(simply increment a number in branch name) - ### explicit version This example showcases how to use [explicit version](../features/version-generation.md#explicit-version). @@ -146,3 +130,20 @@ see [build.yml](../examples/env-cleanup/.github/workflows/build.yml) workflow fi ``` It also uses OIDC to assume IAM role and access CodeArtifact in read-only mode with role `ci/builder` + +## dev-release + +ECR supports [dev-release](../features/dev-release.md): Docker image gets published with tag equal to branch name +(branch name `feature/login` becomes ECR tag `feature-login`). +`agilecustoms/release` action enforces branch name prefix (configured via `dev-branch-prefix`, default is `feature/`). +On the other hand, the ECR supports lifecycle rules by tag prefix. +So if you apply both — you can be sure that all self-service artifacts are removed automatically in a few days! + +There is one important gotcha: you can't re-run dev-release for immutable ECR repo: +- immutable ECR repo prevents pushing image for an existing tag +- tag can be deleted, but IAM doesn't allow to configure permissions selectively for tag prefix, + so if you allow deleting tags — it is a risk that a developer can remove some non-dev tag +- if you make ECR repo mutable — it is a risk that a developer can override a non-dev tag + +So if you need to re-run dev-release shortly (before old tag expired), the workaround is just to create a new branch: +`feature/login` -> `feature/login-2` and run dev-release from it diff --git a/docs/artifact-types/aws-s3.md b/docs/artifact-types/aws-s3.md index a31063e..8b52d6a 100644 --- a/docs/artifact-types/aws-s3.md +++ b/docs/artifact-types/aws-s3.md @@ -3,7 +3,7 @@ _Note: all examples use shared patterns: two workflows: Build and Release — covered in [Best practices](../best-practices.md); "release" GitHub environment and AWS IAM authorization — covered in [Authorization and security](../authorization.md)_ -AWS S3 is very powerful, and specifically it works very well for software distribution. +AWS S3 is very powerful, and specifically, it works very well for software distribution. S3 allows flexible read and write permissions by prefix and by tags. And it has rules for expiration so you can auto remove temporary artifacts For publishing in S3 the `agilecustoms/release` uses a simple convention: @@ -16,9 +16,8 @@ Additionally, you can specify `aws-s3-dir`, then files will be uploaded to `{aws In this section we'll cover some examples of releasing software artifacts in S3: - [Python lambda function](#python-lambda-function) - - [Application code and IaC](#application-code-and-iac) -- [Go lambda function](#go-lambda-function) - [Static website](#static-website) +- [Go lambda function](#go-lambda-function) - [Java CLI application](#java-cli-application) - [dev-release](#dev-release) @@ -55,7 +54,7 @@ jobs: - name: Download artifacts uses: actions/download-artifact@v4 - # this will download `app.zip` in `s3` directory (see Build workflow) + # this will download `app.zip` and place it in `s3` directory (see Build workflow) # next step recognize that `s3` directory exists and upload all files from it to S3 - name: Release @@ -78,7 +77,7 @@ When developer merges a PR, the Release workflow is triggered: 4. upload `s3/app.zip` to `agilecustoms-dist/env-api/{version}/app.zip` 5. push git commit and tags to the remote repository -### Application code and IaC +### IaC This is an example of a microservice that consists of an application (Python) code and IaC (Terraform in `infrastructure` directory). Upon release the `agilecustoms/release` generates a new version, and it is used as git tag and S3 prefix. @@ -93,11 +92,84 @@ module "env_api" { Note file `infrastructure/vars.tf` has variable `aVersion` which is used in `infrastructure/lambda.tf` as part of `s3_key` -## Go lambda function +## Static website -TBD +Example: [tt-web](../examples/tt-web) — it is from AgileCustoms repository with all code removed, only workflows left -## Static website +``` + +├── .github/ +├── dist/ <-- created in Build workflow +│ ├── assets/ +│ └── index.html +├── infrastructure/ <-- terraform code +├── src/ <-- TypeScript code +└── package.json +``` + +In this example a standard npm is used to manage dependencies and `vite` to build static files. +Note: there is no 'package' phase like in Python. That's because AWS offers static website hosting directly from S3 bucket, +and it also offers "copy files" API available in Terraform as `aws_s3_object_copy` resource. +So the distribution format (how release files are stored) should match how S3 serves static files. + +Also, this example showcases use of corporate action `mycompany/gha-release` which is a thin wrapper around `agilecustoms/release`. +In this action you provide all defaults, so your release workflow gets even simpler, see [details](../best-practices.md#company-specific-gha-release-wrapper) + +```yaml +jobs: + Build: + uses: ./.github/workflows/build.yml + # ... + + Release: + needs: Build + # ... + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Download artifacts + uses: actions/download-artifact@v4 + + # this will download static assets and place them in `s3` directory (see Build workflow) + # next step recognize that `s3` directory exists and upload all files from it to S3 + + - name: Release + uses: mycompany/gha-release@main + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} +``` + +When developer merges a PR, the Release workflow is triggered: +1. Release workflow calls Build workflow +2. Build workflow uses `vite` to compile TypeScript in JavaScript and place final bundled code in `dist` directory, then Build workflow uploads `dist` directory as an artifact named `s3` +3. Release workflow downloads the artifact, so you get `s3/{dist content}` +4. Release workflow calls `mycompany/release-gha` action +5. `mycompany/release-gha` calls `agilecustoms/release` passing lots of defaults: `aws-account`, `aws-region`, `aws-s3-bucket` and others +6. `aguilecustoms/release` action then: + 1. generate next version based on commit messages + 2. authorize in AWS with role `ci/publisher`, see [Authorization and security](../authorization.md) + 3. update version in `package.json` + 4. upload `s3/*` to `mycompany-dist/tt-web/{version}/*` + 5. push git commit and tags to the remote repository + +### IaC + +This is an example of a microservice that consists of an application (TypeScript) code and IaC (Terraform in `infrastructure` directory). +Upon release the `agilecustoms/release` generates a new version, and it is used as git tag and S3 prefix. +So your code and infrastructure are in sync! Now you can deploy infra and code like this: + +```hcl +module "tt_web" { + source = "git::https://github.com/agilecustoms/tt-web.git//infrastructure?ref=1.2.3" + aVersion = "1.2.3" +} +``` + +Note file `infrastructure/main.tf` has variable `aVersion` which is datasource `aws_s3_objects` to access (download) files +from dist S3 bucket and then upload them in static website using resource `aws_s3_object_copy` + +## Go lambda function TBD @@ -107,7 +179,11 @@ TBD ## dev-release -TBD +S3 supports [dev-release](../features/dev-release.md). Branch name `feature/login` becomes a version `feature-login`, +and files uploaded at `{aws-s3-bucket}/{current-repo-name}/feature-login/` -Publish files in `{aws-s3-bucket}/{current-repo-name}/{current-branch-name}/` directory. -Each S3 file will be tagged with `Release=false`, so you can set up lifecycle rule to delete such files after 30 days! +`agilecustoms/release` action adds tag `Release` to each S3 object. In normal mode `Release=true`, in dev-release mode `Release=false`. +It is important for security and cleanup: +- you can configure S3 lifecycle rule to auto-remove objects with tag `Release=false` after 30 days +- IAM role (e.g. `ci/publisher-dev`) used in dev-release workflow can distinguish between normal release and dev-release: + it allows to override `Release=false` objects and deny to override `Release=true` diff --git a/docs/artifact-types/git.md b/docs/artifact-types/git.md index 8b402f3..a347605 100644 --- a/docs/artifact-types/git.md +++ b/docs/artifact-types/git.md @@ -6,6 +6,11 @@ This is the simplest scenario for `agilecustoms/release` as there is no need to _Note: all examples use shared patterns: two workflows: Build and Release; parameter `npm-extra-deps` — covered in [Best practices](../best-practices.md); "release" GitHub environment — covered in [Authorization and security](../authorization.md)_ +- [Terraform module](#terraform-module) +- [Composite GitHub Action](#composite-github-action) +- [Node.js GitHub Action](#nodejs-github-action) +- [dev-release](#dev-release) + ## Terraform module Example: [terraform-aws-ci-publisher](https://github.com/agilecustoms/terraform-aws-ci-builder) @@ -86,3 +91,14 @@ When developer merges a PR, the Release workflow is triggered: 3. Release workflow downloads the artifact and calls `agilecustoms/release` action, then action: 1. commit `dist/index.js` 2. push commit and tags to the remote repository + +## dev-release + +Git formally supports [dev-release](../features/dev-release.md). It just means you can create a feature branch +and run `agilecustoms/release` action in `dev-release` mode on this branch. +Git acts like glue for all other artifacts: branch `feature/login` becomes `feature-login` version. +IaC files such as Terraform are accessible via `?ref=feature/login` + +Input `dev-branch-prefix` (default is `feature/`) enforces that only branches with this prefix +can run `agilecustoms/release` in dev-release mode. +This helps with [security](../features/dev-release.md#security) and auto cleanup diff --git a/docs/artifact-types/index.md b/docs/artifact-types/index.md index b0b3f9c..162deb7 100644 --- a/docs/artifact-types/index.md +++ b/docs/artifact-types/index.md @@ -5,5 +5,5 @@ - [Publish in AWS ECR](./aws-ecr.md) - [Publish in AWS CodeArtifact Maven repository](./aws-codeartifact-maven.md) - Non-AWS - - [Publish in npmjs repo](./npmjs.md) + - [Publish in npmjs](./npmjs.md) - [No artifacts, just git tags](./git.md) diff --git a/docs/authorization.md b/docs/authorization.md index a5c7d12..0e2f744 100644 --- a/docs/authorization.md +++ b/docs/authorization.md @@ -1,6 +1,6 @@ # Authorization and security -In this document we'll cover next topics: +In this document we'll cover the following topics: - [GitHub Authorization and security](#github-authorization-and-security) - [AWS Authorization](#aws-authorization) - [dev-release security](#dev-release-security) @@ -13,24 +13,24 @@ These are characteristics for a typical GitHub project ([TLDR](#final-gh-repo-se - release workflow creates new tag (for now forget about automated changes such CHANGELOG.md) - developer can push changes in non-protected branch such as `feature/login` -Now look into last two points: there is a hidden **security implication**. For a developer to do their job, +Now look at the last two points: there is a hidden **security implication**. For a developer to do their job, you need to grant them a `contents: write` permission, but this permission also allows to push arbitrary git tags! -Solution? Use GitHub feature "tag ruleset" to prohibit all tags creation, update and deletion. +Solution? Use the GitHub feature "tag ruleset" to prohibit all tags creation, update and deletion. Now we need to grant a permission to bypass this rule to a release workflow so it can create tags! This is where PAT ([Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)) comes into play: in this "tag ruleset" you configure a Role that can bypass it, -then a person possessing this role needs to create a PAT to use it in the release workflow +then a person possessing this role needs to create a PAT to use it in the release workflow: - either a fine-grained PAT with `Contents "Read and write"` - or classic PAT with `repo` scope -Next big question: **how to ensure this PAT is not compromised**? -Like developer make a mistake/malicious workflow in a feature branch to print PAT in file and then upload it as artifact. +Next big question: **how do you ensure this PAT is not compromised**? +Like when a developer makes a mistake/malicious workflow in a feature branch to print PAT in file and then upload it as artifact. There are two solutions: For private/internal repositories you can use **push ruleset** to prohibit any changes in `.github/**/*`. This restriction is even stronger than `CODEOWNERS`: GitHub will reject any push that attempts to change workflow files! -So you can put the PAT in organization level secret and access it from all repos. Problem 1: if any repo miss this rule, +So you can put the PAT in an organization-level secret and access it from all repos. Problem 1: if any repo misses this rule, the token still can leak. Problem 2: it is not scalable — only admins can change `.github/**/*` files, other developers can't even create a PR for improvements! @@ -38,7 +38,21 @@ The better option is to create a **GitHub environment** (lets call it `release`) Then configure this environment so that only protected branches can use it. Finally, in the release workflow (assuming it is run on push in protected branch) you specify `environment: release` to access a secret -One problem left: what if two "bad" developers act together: one creates a PR to change release workflow to print PAT, +`agilecustoms/release` action takes this PAT via env variable `GH_TOKEN`: +```yaml +jobs: + Release: + environment: release + # ... + steps: + # ... + - name: Release + uses: agilecustoms/release@v1 + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} # secret can have any name, use `GH_TOKEN` for consistency +``` + +One problem remains: what if two "bad" developers act together: one creates a PR to change release workflow to print PAT, and second approves it? To mitigate this risk you configure `CODEOWNERS` so that only trusted people can approve changes in `.github/**/*` ### Final GH repo setup @@ -53,7 +67,7 @@ At this point GitHub security should be in a good shape. Only problem — it is tag protection rules, create environment and configure its access from protected branches, configure CODEOWNERS. In the world of microservices it is quite common to have 50+ repositories, so it is a lot of work to do it manually. You can automate this via provisioning GitHub repos via Terraform (there is a [GitHub provider](https://registry.terraform.io/providers/integrations/github/latest/docs)). -Fall 2025 I plan to release a Terraform module that will do all this work for you. +Fall 2025 I plan to release a Terraform module that will do all this work for you ## AWS Authorization @@ -115,29 +129,10 @@ so if you need to read repository content, make sure to add `contents: read` exp ## dev-release security -TBD - -Dev-release does not require PAT because it does not need to make a direct push in the protected branch - -```yaml -jobs: - Release: - runs-on: ubuntu-latest - environment: release - permissions: # required only to publish artifacts in AWS - contents: read - id-token: write - steps: - - name: Checkout - uses: actions/checkout@v4 - - # ... - - - name: Release - uses: agilecustoms/gha-release@main - with: - dev-release: true -``` +`agilecustoms/release` has major feature [dev-release](./features/dev-release.md). +It is basically a release self-service for developers. +Use of this feature assumes all security measures described above are already in place! +Additional dev-release specific security considerations are placed in the [dev-release security](./features/dev-release.md#security) section ## Advanced diff --git a/docs/best-practices.md b/docs/best-practices.md index 96e2520..ba2caac 100644 --- a/docs/best-practices.md +++ b/docs/best-practices.md @@ -1,7 +1,7 @@ # Best practices - [Build and Release](#build-and-release) -- [conventional commits](#conventional-commits) +- [Conventional commits](#conventional-commits) - [Company-specific gha-release wrapper](#company-specific-gha-release-wrapper) ## Build and Release @@ -14,9 +14,9 @@ At this point Build workflow does not produce any artifacts, it just tells you i Once it is good, you typically create a PR and get approvals Upon PR merge, the Release workflow is triggered (it is configured to run on push in `main` branch). -Then Release workflow calls Build workflow again, this time intent is to get artifacts! +Then the Release workflow calls Build workflow again, this time intent is to get artifacts! -These are typical setup for these two workflows: +There is a typical setup for these two workflows: `build.yml` ```yaml @@ -52,7 +52,7 @@ jobs: # Build/Package the artifact(s) - name: Upload artifacts - if: inputs.artifacts # no sense to upload artifacts on every push in a feature branch + if: inputs.artifacts # no point in uploading artifacts on every push in a feature branch uses: actions/upload-artifact@v4 with: # configure what to upload ``` @@ -79,7 +79,7 @@ jobs: Release: needs: Build runs-on: ubuntu-latest - environment: release # hold `GH_TOKEN` secret, available only for `main` branch + environment: release # holds the `GH_TOKEN` secret, available only for `main` branch permissions: contents: read # required for checkout id-token: write # not always required, only if at some step you assume AWS role via OIDC @@ -97,7 +97,7 @@ jobs: GH_TOKEN: ${{ secrets.GH_TOKEN }} # required to push commit and tags ``` -## conventional commits +## Conventional commits `agilecustoms/release` uses [semantic-release](https://github.com/semantic-release/semantic-release) for next version generation. semantic-release has several presets for different commit message conventions. @@ -135,8 +135,26 @@ Non-default presets require additional npm dependency. This is why in many examp npm-extra-deps: conventional-changelog-conventionalcommits@9.1.0 ``` -For more details see [semantic commits](./features/semantic-commits.md) +For more details see [Semantic commits](./features/semantic-commits.md) ## Company-specific gha-release wrapper -TBD +In many [examples](./examples) you may some inputs repeat, such as `aws-account`, `aws-region`, `aws-role`. +You can reduce code repetition by creating your own wrapper around `agilecustoms/release`. +For this just create a new GH repo with single file `action.yml`, see example [gha-release](./examples/gha-release). +Here you provide company-specific defaults, see comment `# company-specific defaults`. +For parameters that may vary - you add them in `inputs` section and pass them through, see comment `# pass through inputs`. +For pass-through inputs you can use either same defaults as in `agilecustoms/release` (like `floating-tags` default is true) +and for others you may want to set your own defaults (like `summary`). + +Especially useful to provide default value for input `release-plugins`. +This is the recommended way to have shared release configuration among multiple repos. +Alternative is to have `.releaserc.json` file in each repo, but then you need to maintain it in multiple places. +See [configuration](./configuration.md) for more details on configuration options. + +Security note. In "dev-release" mode the `aws-role` is set to `ci/publisher-dev` which has limited permissions, +see [dev-release security](./authorization.md#dev-release-security) for details + +Tech note. Once you create your custom GH action wrapper, make sure other repos can access it. +For this go to your repo Settings > Actions > General > "Workflow permissions" > "Access" > +set "Accessible from repositories in the '_mycompany_' organization" diff --git a/docs/configuration.md b/docs/configuration.md index 49c6fcd..7f35e87 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -23,9 +23,9 @@ to create your company-specific GH action wrapper and put common configuration a Bottom line, these are only supported configuration options for semantic-release: -| config file section | GH action input | purpose | -|----------------------------------------------------------------------------------------------------------------------------------|--------------------|------------------------------------------------------------------------------------------------| -| `branches` ([details](https://semantic-release.gitbook.io/semantic-release/usage/configuration?utm_source=chatgpt.com#branches)) | `release-branches` | mainly for [maintenance releases](features/maintenance-release.md) and [prereleases](features/prerelease.md) | -| `plugins` | `release-plugins` | mainly for [semantic versioning](features/semantic-commits.md) | -| `tagFormat` ([details](https://semantic-release.gitbook.io/semantic-release/usage/configuration#tagformat)) | `tag-format` | tag format = version format (`v1.2.3`, `1.2.3`, `release-1.2.3`) | -| `branches > .. > channel` | `release-channel` | mainly for [floating tags](features/floating-tags.md) | +| config file section | GH action input | purpose | +|-------------------------------------------------------------------------------------------------------------|--------------------|--------------------------------------------------------------------------------------------------------------| +| `branches` ([details](https://semantic-release.gitbook.io/semantic-release/usage/configuration#branches)) | `release-branches` | mainly for [maintenance releases](features/maintenance-release.md) and [prereleases](features/prerelease.md) | +| `plugins` | `release-plugins` | mainly for [semantic versioning](features/semantic-commits.md) | +| `tagFormat` ([details](https://semantic-release.gitbook.io/semantic-release/usage/configuration#tagformat)) | `tag-format` | tag format = version format (`v1.2.3`, `1.2.3`, `release-1.2.3`) | +| `branches > .. > channel` | `release-channel` | mainly for [floating tags](features/floating-tags.md) | diff --git a/docs/examples/gha-release/action.yml b/docs/examples/gha-release/action.yml new file mode 100644 index 0000000..ec03fc1 --- /dev/null +++ b/docs/examples/gha-release/action.yml @@ -0,0 +1,112 @@ +name: 'Release' +description: 'MyCompany wrapper for GH action agilecustoms/release' +inputs: + aws-codeartifact-maven: + required: false + aws-ecr: + required: false + dev-branch-prefix: + required: false + default: 'feature/' + dev-release: + required: false + default: 'false' + floating-tags: + required: false + default: 'true' + node-version: + required: false + default: '22' + pre-publish-script: + required: false + release-channel: + required: false + release-gh: + required: false + default: 'true' + release-plugins: + required: false + default: | + [ + [ + "@semantic-release/commit-analyzer", + { + "preset": "conventionalcommits", + "releaseRules": [ + { "type": "perf", "release": false }, + { "type": "docs", "release": "patch" }, + { "type": "misc", "release": "patch" } + ] + } + ], + [ + "@semantic-release/release-notes-generator", + { + "preset": "conventionalcommits", + "presetConfig": { + "types": [ + { + "type": "feat", + "section": "Features" + }, + { + "type": "fix", + "section": "Bug Fixes" + }, + { + "type": "docs", + "section": "Documentation" + }, + { + "type": "misc", + "section": "Miscellaneous" + } + ] + } + } + ] + ] + summary: + required: false + default: '### Released ${version} :pushpin:' + tag-format: + required: false + default: '${version}' + version: + required: false + version-bump: + required: false + +runs: + using: "composite" + steps: + - name: release + id: release + uses: agilecustoms/release@v1 + with: + # company-specific defaults + aws-account: 123456789012 + aws-region: us-east-1 + aws-role: ${{ inputs.dev-release == 'true' && 'ci/publisher-dev' || 'ci/publisher' }} + aws-codeartifact-domain: mycompany + aws-s3-bucket: mycompany-dist + npm-extra-deps: conventional-changelog-conventionalcommits@9.1.0 + # pass through inputs + aws-codeartifact-maven: ${{ inputs.aws-codeartifact-maven }} + aws-ecr: ${{ inputs.aws-ecr }} + dev-branch-prefix: ${{ inputs.dev-branch-prefix }} + dev-release: ${{ inputs.dev-release }} + floating-tags: ${{ inputs.floating-tags }} + node-version: ${{ inputs.node-version }} + pre-publish-script: ${{ inputs.pre-publish-script }} + release-channel: ${{ inputs.release-channel }} + release-gh: ${{ inputs.release-gh }} + release-plugins: ${{ inputs.release-plugins }} + summary: ${{ inputs.summary }} + tag-format: ${{ inputs.tag-format }} + version: ${{ inputs.version }} + version-bump: ${{ inputs.version-bump }} + +outputs: + version: + value: ${{ steps.release.outputs.version }} diff --git a/docs/examples/tt-web/.github/workflows/build.yml b/docs/examples/tt-web/.github/workflows/build.yml new file mode 100644 index 0000000..6a46397 --- /dev/null +++ b/docs/examples/tt-web/.github/workflows/build.yml @@ -0,0 +1,38 @@ +name: Build + +on: + push: + branches-ignore: + - main + workflow_call: + inputs: + artifacts: + type: boolean + default: false + +jobs: + Build: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Switch to Node 22 + uses: actions/setup-node@v4 + with: + node-version: 22 + + - name: Install dependencies + run: npm ci + + # Lint and Test ... + + - name: Build + run: npm run build + + - name: Upload artifacts + if: inputs.artifacts + uses: actions/upload-artifact@v4 + with: + path: dist # take everything from dist/ folder + name: s3 # and create an artifact named s3 (so later on the download action will create /dist/ folder) diff --git a/docs/examples/tt-web/.github/workflows/release-dev.yml b/docs/examples/tt-web/.github/workflows/release-dev.yml new file mode 100644 index 0000000..dea08a7 --- /dev/null +++ b/docs/examples/tt-web/.github/workflows/release-dev.yml @@ -0,0 +1,35 @@ +name: Dev Release + +on: + workflow_dispatch: + +jobs: + Build: + uses: ./.github/workflows/build.yml + with: + artifacts: true + secrets: inherit + permissions: + contents: read + id-token: write + + Release: # upload in S3 w/o version, just git hash + needs: Build + runs-on: ubuntu-latest + permissions: + contents: read + id-token: write + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Download artifacts + uses: actions/download-artifact@v4 + + - name: Release + uses: agilecustoms/gha-release@main + with: + dev-release: true + + - name: Summary + run: echo "### Uploaded '$GITHUB_REF_NAME' to S3 :white_check_mark:" >> $GITHUB_STEP_SUMMARY diff --git a/docs/examples/tt-web/.github/workflows/release.yml b/docs/examples/tt-web/.github/workflows/release.yml new file mode 100644 index 0000000..a81d4bc --- /dev/null +++ b/docs/examples/tt-web/.github/workflows/release.yml @@ -0,0 +1,35 @@ +name: Release + +on: + push: + branches: + - main + +jobs: + Build: + uses: ./.github/workflows/build.yml + with: + artifacts: true + secrets: inherit + permissions: + contents: read + id-token: write + + Release: + needs: Build + runs-on: ubuntu-latest + environment: release + permissions: + contents: write + id-token: write + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Download artifacts + uses: actions/download-artifact@v4 + + - name: Release + uses: agilecustoms/gha-release@main + env: + GH_TOKEN: ${{ secrets.GH_TOKEN }} diff --git a/docs/examples/tt-web/infrastructure/main.tf b/docs/examples/tt-web/infrastructure/main.tf new file mode 100644 index 0000000..e4a5db6 --- /dev/null +++ b/docs/examples/tt-web/infrastructure/main.tf @@ -0,0 +1,19 @@ +variable "aVersion" {} +variable "env" {} +variable "dist_bucket" {} + +resource "aws_s3_bucket" "web" { + bucket = "${var.env}.tt.agilecustoms.com" +} + +data "aws_s3_objects" "src" { + bucket = var.dist_bucket + prefix = "tt-web/${var.aVersion}" +} + +resource "aws_s3_object_copy" "files" { + for_each = toset(data.aws_s3_objects.src.keys) + source = "${var.dist_bucket}/${each.key}" + bucket = aws_s3_bucket.web.id + key = each.key +} diff --git a/docs/examples/tt-web/package.json b/docs/examples/tt-web/package.json new file mode 100644 index 0000000..6cf1b44 --- /dev/null +++ b/docs/examples/tt-web/package.json @@ -0,0 +1,16 @@ +{ + "name": "tt-web", + "private": true, + "version": "1.35.20", + "type": "module", + "scripts": { + "build": "vite build" + }, + "dependencies": { + "react": "^19.0.0" + }, + "devDependencies": { + "typescript": "^5.6.2", + "vite": "^7.0.0" + } +} diff --git a/docs/features/dev-release.md b/docs/features/dev-release.md index 7a270c4..7f58c0e 100644 --- a/docs/features/dev-release.md +++ b/docs/features/dev-release.md @@ -1,60 +1,78 @@ # Dev release -Besides normal releases, prereleases and maintenance releases, this action supports a special type of release called **dev-release**. +Besides normal releases, prereleases, and maintenance releases, this action supports a special type of release called **dev-release**. It sounds similar to prerelease, so let's clarify the difference: **prerelease** is an industry standard, though some tools use different terms like "prerelease" (GitHub), distribution tags (npm), suffix "-SNAPSHOT" (maven), suffixes "-beta", "rc" (Gradle, NuGet, PyPI, pip). -Idea: release a version widely available for testing and potential to become a next major release +Idea: release a version widely available for testing and with potential to become the next major release **dev-release** is a way to overcome the inability to spin up the entire env locally. -It allows to temporarily release a version (= branch name), so that now you can deploy it in sandbox or dev environment for testing or POC. +It allows you to temporarily release a version (= branch name), so that you can deploy it in sandbox or dev environment for testing or POC The table below shows a comparison of different release types: -| Name | normal release and maintenance release | prerelease | dev-release | -|-----------------------------|----------------------------------------|---------------------------------------|-----------------------| -| intention | use in production | beta testing | dev testing | -| best use for | software packages and deployable apps | software packages and deployable apps | deployable apps | -| adoption | widely | widely | popular in enterprise | -| version generation | "semantic commits" or "version-bump" | "semantic commits" or "version bump" | version = branch name | -| auto deletion | ❌️ | ❌️ | ✅ | -| number of developers | many | many | typically one | -| release notes and changelog | ✅ | ✅ | ❌️ | -| floating tags | major, minor, latest | alpha/beta/rc | ❌️ | +| Name | normal release and maintenance release | prerelease | dev-release | +|-----------------------------|-----------------------------------------------|-----------------------------------------------|--------------------------| +| intention | use in production | beta testing | dev testing | +| best use for | software packages and deployable apps | software packages and deployable apps | deployable apps | +| adoption | widely | widely | popular in enterprise | +| version generation | "semantic commits" or "version-bump" | "semantic commits" or "version bump" | version = branch name | +| auto deletion | ❌️ | ❌️ | ✅ | +| number of developers | many | many | typically one | +| release notes and changelog | ✅ | ✅ | ❌️ | +| floating tags | major, minor, latest | alpha / beta / rc | ❌️ | +| trigger | push in protected branch (including PR merge) | push in protected branch (including PR merge) | Manual workflow dispatch | Dev release allows publishing artifacts temporarily for testing purposes: you push your changes to the feature branch, the branch name becomes this dev-release version: - SemVer is _not_ generated +- no automated pushes, so no need for PAT - no git tags created, files in branch addressable by branch name -- if branch name is `feature/login` then the version will be `feature-login` +- `/` gets replaced with `-`, so branch `feature/login` gives version `feature-login` - parameter `dev-branch-prefix` (default value is `feature/`) enforces branch naming for dev releases. - This is needed for security and automatic resource disposal. Set to empty string to disable such enforcement (not recommended). -- for each [artifact type](./../artifact-types/index.md), dev-release might have different semantics, see `dev-release` section for each artifact type + This is needed for security and automatic resource disposal. Set to an empty string to disable such enforcement (not recommended) +- for each [artifact type](./../artifact-types/index.md), dev-release might have different semantics, + see "dev-release" section for each artifact type + +dev-release workflow blueprint: -Example of 'dev-release' usage with AWS S3: ```yaml -steps: - - name: Release - uses: agilecustoms/release@v1 - with: - aws-account: ${{ vars.AWS_ACCOUNT_DIST }} - aws-region: us-east-1 - aws-role: 'ci/publisher-dev' # see "Security" section below - aws-s3-bucket: 'mycompany-dist' - dev-release: true - dev-release-prefix: 'feature/' # default +name: Dev Release + +on: + workflow_dispatch: + +jobs: + # ... + Release: + # ... + steps: + # ... + - name: Release + uses: agilecustoms/release@v1 + with: + aws-account: ${{ vars.AWS_ACCOUNT_DIST }} + aws-region: us-east-1 + aws-role: 'ci/publisher-dev' # see "Security" section below + aws-s3-bucket: 'mycompany-dist' + dev-release: true + dev-release-prefix: 'feature/' # default ``` +Complete examples: +- [AWS S3](../examples/env-api/.github/workflows/release-dev.yml), see [details](../artifact-types/aws-s3.md#dev-release) +- [AWS ECR](../examples/env-cleanup/.github/workflows/release-dev.yml), see [details](../artifact-types/aws-ecr.md#dev-release) + ## Motivation -_How did we live w/o a dev-release before? -We do use microservices in our team, and we just have a CI/CD pipeline that can build a feature branch and deploy it to a dev server. +_How did we live without a dev-release before? +We do use microservices in our team, and we just have a CI/CD pipeline that can build a feature branch and deploy it to a dev environment. So we do not need dev-release, right?_ Build-and-deploy is kind of a "shortcut" and it may work in simple scenarios. -The reality is that a system is a _combination_ of multiple services and true deployment takes a combination of services! +The reality is that a system consists of multiple services, and true deployment takes a combination of services! Imagine a system that consists of two services A and B, and there is a repo C storing the current combination: `A@v1.0`, `B@v1.1`. And only C has a "Deploy" button! Now, you want to make a change in service A and test it, but since the only way to deploy a system is to deploy both services together, @@ -68,55 +86,81 @@ These are only parameters respected by dev-release: `aws-account`, `aws-ecr`, `aws-region`, `aws-role`, `aws-s3-bucket`, `aws-s3-dir`, `dev-branch-prefix` There is no error if `dev-release` used with incompatible parameter (like `tag-format` or `floating-tags`). -General principle: ignore unused parameters, so that you can have one corporate gha wrapper for `agilecustoms/release`. +General principle: ignore unused parameters, +so that you can have one [corporate gha wrapper](../best-practices.md#company-specific-gha-release-wrapper) +for `agilecustoms/release` Only parameter that conflicts with `dev-release` is `version` as it looks like a complete mistake ## Supported artifact types -npm and python Poetry only support SemVer versions so no dev-release. -Maven on the other hand, allows arbitrary version format. +npm and python Poetry only support SemVer versions, so no dev-release. +Maven, on the other hand, allows an arbitrary version format. Originally (up to beta-22) it was possible to dev-release maven package in CodeArtifact. -Problem is that there is no way to distinguish normal release from dev-release. -And also there is no way to automatically delete such dev-release artifacts. -Then I (author) decided to not allow dev-release for CodeArtifact entirely +The problem is that there is no way to distinguish normal release from dev-release. +And also, there is no way to automatically delete such dev-release artifacts. +Then I (author) decided to not allow dev-release for CodeArtifact completely. + +So the only artifact types that support dev-release are: +[AWS S3](../artifact-types/aws-s3.md#dev-release), +[AWS ECR](../artifact-types/aws-ecr.md#dev-release) and +[git](../artifact-types/git.md#dev-release) (just because you can create a feature branch) ## Security -Dev-release mode brings self-service release capabilities to developers, -but also brings security risks. Developer may try to use dev-release workflow to: - -1. Create unverified artifact that looks like normal -2. Update (override) existing production artifact -3. Delete production artifact - -| Name | create unverified | update | delete | overwrite | auto cleanup | -|------|-------------------|---------|---------|-----------|--------------| -| S3 | ✅ safe | ✅ safe | ✅ safe | ✅ safe | ✅ | -| ECR | ❌️ unsafe | ✅ safe | ✅ safe | ✅ safe | ✅ | - -On order to mitigate these risks, follow next practices: - -1. configure GitHub branch ruleset, so any changes in `main` branch are made via PR + review -2. configure GitHub tag ruleset, so any tags can be created only by automation -3. configure GitHub push ruleset with "Restrict file paths" `.github/**/*` -4. configure 2 separate IAM roles: - - `ci/publisher` for normal releases (trust only protected branches), with full permissions to publish artifacts to S3, ECR and CodeArtifact - - `ci/publisher-dev` for dev-release (trust any branch), with limited permissions to publish artifacts to S3 and ECR - -For 'publisher-dev' you can use terraform module [terraform-aws-ci-publisher](https://github.com/agilecustoms/terraform-aws-ci-publisher). -Dev mode available from [v1.1.0-beta.1](https://github.com/agilecustoms/terraform-aws-ci-publisher/releases/tag/v1.1.0-beta.1) - -_Note: this documentation will be improved in Sep 2025 to better describe security practices for dev-release_ - -| Name | publisher | publisher-dev | -|------------------------------------|----------------------------------------|-------------------------------------------| -| CodeArtifact PublishPackageVersion | Allow | | -| S3 bucket policy | delete "Release=false" in 7 days | same | -| S3 Create | Allow PutObject | Allow PutObject w/ tag Release=false | -| S3 Update | Allow PutObject | Deny PutObjectTagging w/ tag Release=true | -| S3 Delete | Allow DeleteObject | | -| ECR repo policy | delete "feature-" in 7 days | same | -| ECR Create | Allow PutImage | Allow PutImage | -| ECR Update | Allow PutImage, Allow BatchDeleteImage | | -| ECR Delete | Allow BatchDeleteImage | | +Dev-release mode brings self-service capabilities but also brings security risks + +_You can't guarantee dev-release security if anybody can assume your 'ci/publisher' role. +Or if anybody can use powerful GH PAT to make arbitrary Git changes and then run a malicious workflow on the main branch. +In this section we assume you already have a secure GitHub setup and AWS authorization, see [Authorization and security](../authorization.md)_ + +Now lets focus on risks coming specifically from dev-release. Developers may try to use dev-release workflow to: +1. _Create_ unverified artifact that looks like normal +2. _Update_ (override) existing production artifact +3. _Delete_ production artifact + +Down below these three risks are referred as _Create_, _Update_, _Delete_. +To mitigate these risks, we'll need two separate IAM roles: +- `ci/publisher` for normal releases (trust only protected branches), with full permissions to publish artifacts to S3, ECR and CodeArtifact +- `ci/publisher-dev` for dev-release (trust any branch), with limited permissions to publish artifacts to S3 and ECR + +IAM role has two parts: trust policy and permission policy + +**permission policy** for both roles can be created with terraform module +[ci-publisher](https://registry.terraform.io/modules/agilecustoms/ci-publisher/aws/latest). +For dev policy you would use input `dev = true` and it will create a limited policy for dev-release. +Additionally, you need to configure lifecycle rules to automatically delete dev-release artifacts. +Below is a breakdown of permissions needed: + +| Name | publisher | publisher-dev | +|-------------------|----------------------------------------|-------------------------------------------| +| Role CodeArtifact | Allow PublishPackageVersion | | +| S3 bucket policy | delete "Release=false" in 7 days | _same_ | +| Role S3 Create | Allow PutObject | Allow PutObject w/ tag Release=false | +| Role S3 Update | Allow PutObject | Deny PutObjectTagging w/ tag Release=true | +| Role S3 Delete | Allow DeleteObject | | +| ECR repo policy | delete "feature-" in 7 days | _same_ | +| Role ECR Create | Allow PutImage | Allow PutImage | +| Role ECR Update | Allow PutImage, Allow BatchDeleteImage | | +| Role ECR Delete | Allow BatchDeleteImage | | + +**trust policy** for both roles going to use same OIDC provider, but different `sub` condition. +`ci/publisher-dev` role can trust any branch or only branches starting with `feature/`, up to you. +For `ci/publisher` role you might think you need to trust only protected branches! +But in GitHub if a workflow uses an environment — it takes precedence in OIDC token, +so on AWS side instead of "trust `main` branch" you would need to configure "trust `release` environment" type of trust policy. +See [terraform-aws-ci-publisher](https://github.com/agilecustoms/terraform-aws-ci-publisher) for trust policy examples. +See [GitHub Authorization](../authorization.md#github-authorization-and-security) why environment is needed in the first place + +With these permission policies the risk mitigation looks like this: + +| Name | create unverified | update | delete | +|------|-------------------|---------|---------| +| S3 | ⚠️ safe | ✅ safe | ✅ safe | +| ECR | ⚠️ safe | ✅ safe | ✅ safe | + +Why create unverified is marked as ⚠️? Because IAM policy itself can't 100% prevent creating unverified artifact. +Assume you have a role `ci/publisher-dev` that allows to create artifacts in S3 and ECR, +and this role can be assumed from any feature branch. The risk is that a developer can create a feature branch and +craft a workflow where they assume `ci/publisher-dev` role and create artifact manually, not via `agilecustoms/release` action. +To mitigate this risk, you need to use GitHub rules that any changes in `.github/**/*` must be approved by trusted people diff --git a/docs/roadmap.md b/docs/roadmap.md index 3b3f44d..fd14321 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -1,14 +1,18 @@ # Roadmap -Below is the list of potential future features: +Fall 2025: +- Terraform module to create security-aware GitHub repo +- GitHub action to access terraform modules from corporate private GH repos +- setup-maven-codeartifact support Maven 4 +- publish in Maven central +- publish Python packages (pip and Twine) in CodeArtifact +Potential future features (not prioritized): - publish in AWS CodeArtifact. Every artifact type will require a corresponding GH action like [setup-maven-codeartifact](https://github.com/agilecustoms/setup-maven-codeartifact) - npm (and Yarn) repository - - pip (and Twine) repository - Gradle repository - publish in non-AWS repositories - private npmjs repository - - Maven central - GitHub release - ability to specify a list of files to include release - integration between release and issues/PRs, ex: close issues fixed in a release, see [semantic-release/github](https://github.com/semantic-release/github) diff --git a/misc/.git-config b/misc/.git-config index b2f29cc..dc5fd04 100644 --- a/misc/.git-config +++ b/misc/.git-config @@ -1,3 +1,5 @@ +# example of .git/config file (token in fake) +# used this example to debug/understand step "Git authorization" [core] repositoryformatversion = 0 filemode = true