Skip to content

Commit

Permalink
frantjc/docs (#332)
Browse files Browse the repository at this point in the history
  • Loading branch information
frantjc authored Apr 23, 2024
1 parent 9369ba3 commit 012788d
Show file tree
Hide file tree
Showing 53 changed files with 509 additions and 303 deletions.
4 changes: 4 additions & 0 deletions .forge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,7 @@ resources:
source:
force_version: "true"
log: ok
- name: git
type: git
source:
uri: https://github.com/frantjc/forge
5 changes: 5 additions & 0 deletions .github/actions/mkdocs/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
FROM squidfunk/mkdocs-material:9.4.5
COPY action.sh /action.sh
RUN apk add --no-cache bash \
&& chmod +x /action.sh
ENTRYPOINT ["/action.sh"]
30 changes: 30 additions & 0 deletions .github/actions/mkdocs/action.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/bin/bash

set -e

REQUIREMENTS="$GITHUB_WORKSPACE/docs/requirements.txt"

if [ -f "$REQUIREMENTS" ]; then
pip install -r "$REQUIREMENTS"
fi

if [ -n "$GITHUB_TOKEN" ]; then
REMOTE_REPO="https://x-access-token:$GITHUB_TOKEN@github.com/$GITHUB_REPOSITORY.git"
fi

git config --global user.name "$GITHUB_ACTOR"
git config --global user.email "$GITHUB_ACTOR@users.noreply.github.com"

mkdocs build --config-file "$GITHUB_WORKSPACE/mkdocs.yml"

git clone --branch=gh-pages --single-branch --depth=1 "$REMOTE_REPO" gh-pages
cd gh-pages

# remove current content in branch gh-pages
git rm -r .
# copy new doc.
cp -r ../site/* .
# commit changes
git add .
git commit -m "deploy GitHub Pages"
git push --force --quiet "$REMOTE_REPO" gh-pages > /dev/null 2>&1
8 changes: 8 additions & 0 deletions .github/actions/mkdocs/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
name: Deploy Mkdocs
description: Deploys Mkdocs site
branding:
icon: arrow-up-circle
color: orange
runs:
using: docker
image: Dockerfile
2 changes: 0 additions & 2 deletions .github/workflows/build-action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,6 @@ jobs:
- uses: actions/setup-node@v4
with:
node-version: 20
cache: yarn
cache-dependency-path: .github/actions/setup-forge/yarn.lock
- name: Run make action
run: |
cd .github/actions/setup-forge
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/build-shims.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Build Shims
on: [workflow_dispatch]
on: workflow_dispatch
permissions:
contents: write
pull-requests: write
Expand All @@ -11,7 +11,6 @@ jobs:
- uses: actions/setup-go@v5
with:
go-version: 1.22
cache: ${{ github.ref_name == 'main' }}
- uses: crazy-max/ghaction-upx@v3
with:
install-only: true
Expand Down
3 changes: 0 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,9 @@ jobs:
- uses: actions/setup-node@v4
with:
node-version: 20
cache: yarn
cache-dependency-path: .github/actions/setup-forge/yarn.lock
- uses: actions/setup-go@v5
with:
go-version: 1.22
cache: ${{ github.ref_name == 'main' }}
- uses: golangci/golangci-lint-action@v4.0.0
- run: make download
- run: make test
Expand Down
19 changes: 19 additions & 0 deletions .github/workflows/mkdocs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Mkdocs
on:
push:
branches:
- main
paths:
- .github/actions/mkdocs/**
- .github/workflows/mkdocs.yml
- docs/**
permissions:
contents: write
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/mkdocs
env:
GITHUB_TOKEN: "${{ secrets.GITHUB_TOKEN }}"
7 changes: 7 additions & 0 deletions .github/workflows/run-using-nonstd-dockerfile.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
name: Runs Using Nonstandard Dockerfile
on: workflow_dispatch
jobs:
save-state:
runs-on: ubuntu-latest
steps:
- uses: frantjc/forge/testdata/actions/dockerfile/nonstd@main
2 changes: 1 addition & 1 deletion .github/workflows/save-state.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Save State
on: [workflow_dispatch]
on: workflow_dispatch
jobs:
save-state:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/set-env.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Set Env
on: [workflow_dispatch]
on: workflow_dispatch
jobs:
save-state:
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/set-output.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Set Output
on: [workflow_dispatch]
on: workflow_dispatch
jobs:
save-state:
runs-on: ubuntu-latest
Expand Down
7 changes: 7 additions & 0 deletions .github/workflows/yml-v-yaml.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
name: Runs Using action.yml or action.yaml
on: workflow_dispatch
jobs:
save-state:
runs-on: ubuntu-latest
steps:
- uses: frantjc/forge/testdata/actions/yml-v-yaml@main
6 changes: 4 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
# IDE
.idea/
.vscode/

# macOS
Expand All @@ -10,7 +9,6 @@ vendor/

# dev
__debug_bin
.env

# concourse/mock-resource
rootfs/
Expand All @@ -22,3 +20,7 @@ dist/

# nodejs
node_modules/

# mkdocs
site/
gh-pages/
2 changes: 1 addition & 1 deletion .goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ brews:
- repository:
owner: frantjc
name: homebrew-tap
folder: Formula
directory: Formula
skip_upload: auto
homepage: https://github.com/frantjc/forge
description: Easily run reusable steps from proprietary CI systems.
Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ BIN = /usr/local/bin
GOOS = $(shell $(GO) env GOOS)
GOARCH = $(shell $(GO) env GOARCH)

SEMVER ?= 0.15.0
SEMVER ?= 0.15.1

.DEFAULT: install

Expand Down Expand Up @@ -81,4 +81,4 @@ shim_$(GOARCH): internal/bin/shim_$(GOARCH) internal/bin/fs_$(GOARCH).go

.PHONY: .github/actions/setup-forge .github/actions/setup-forge/ action i install build fmt generate test download vendor verify lint shim shim_$(GOARCH) internal/bin/fs_$(GOARCH).go internal/bin/shim_$(GOARCH) clean gen dl ven ver format release

-include docs/docs.mk
-include docs/gifs.mk
192 changes: 5 additions & 187 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,191 +1,9 @@
# forge [![CI](https://github.com/frantjc/forge/actions/workflows/ci.yml/badge.svg?branch=main&event=push)](https://github.com/frantjc/forge/actions) [![godoc](https://pkg.go.dev/badge/github.com/frantjc/forge.svg)](https://pkg.go.dev/github.com/frantjc/forge) [![goreportcard](https://goreportcard.com/badge/github.com/frantjc/forge)](https://goreportcard.com/report/github.com/frantjc/forge) ![license](https://shields.io/github/license/frantjc/forge)
# forge [![CI](https://github.com/frantjc/forge/actions/workflows/ci.yml/badge.svg?branch=main&event=push)](https://github.com/frantjc/forge/actions) [![godoc](https://pkg.go.dev/badge/github.com/frantjc/forge.svg)](https://pkg.go.dev/github.com/frantjc/forge) [![goreportcard](https://goreportcard.com/badge/github.com/frantjc/forge)](https://goreportcard.com/report/github.com/frantjc/forge)

<p align="center">
<img src="https://raw.githubusercontent.com/frantjc/forge/main/docs/demo.gif">
</p>
Have you ever had to swap to using a new CI system? Twice? Three times? Done with searching for a replacement for each Action, CloudBuilder or resource that you were using in your old one? Tired of waiting minutes for feedback on each iteration of dealing with the quirks of your new one?

Forge is a library and CLI for running reusable steps from various proprietary CI systems using a pluggable container runtime. This, for example, makes the functionality provided to GitHub Actions easily consumable (or testable) by users of other CI systems.

Forge currently exposes running [GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/finding-and-customizing-actions) (e.g. [`actions/setup-go`](https://github.com/actions/setup-go)), [Concourse Resources](https://concourse-ci.org/resources.html) (e.g. [`concourse/git-resource`](https://github.com/concourse/git-resource)) and [Google Cloudbuild Steps](https://cloud.google.com/build/docs/configuring-builds/create-basic-configuration) (e.g. [`gcr.io/cloud-builders/docker`](https://cloud.google.com/build/docs/building/build-containers)).

## install

From a [release](https://github.com/frantjc/forge/releases).

Using `brew`:

```sh
brew install frantjc/tap/forge
```

From source:

```sh
git clone https://github.com/frantjc/forge
cd forge
make
```

Using `go`:

```sh
go install github.com/frantjc/forge/cmd/forge
```

In GitHub Actions:

```yml
- uses: frantjc/forge@v0
```
As a library:
```sh
go get -u github.com/frantjc/forge
```

## usage

### GitHub Actions

For GitHub Actions, Forge will try to source the GitHub Actions variables from the working directory's Git configuration as well as GitHub's [default environment variables](https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables).

```sh
forge use actions/setup-go@v3 -w go-version=1.22
```

Forge mounts the current working directory to the Action's `GITHUB_WORKSPACE` as well as cache directories respecting the [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html) to the Action's `RUNNER_TOOLCACHE` and `RUNNER_TEMP`.

That is to say, after running the above command, `go` should be installed to `XDG_CACHE_HOME/forge/runner/toolcache`.

You can also use local GitHub Actions by starting the reference with `"/"` or `"."` to signify that it is an absolute or relative local filepath, respectively.

```sh
forge use ./testdata/actions/docker
```

For additional debugging, you can attach to the container running the Action:

```sh
forge use -a ./testdata/actions/dockerfile
```

> If the Action runs using a custom image, that image must have `bash` or `sh` on its `PATH` for the attach to work.
> Local Actions cannot refer to files outside of the action metadata file's directory.
### Concourse Resources

For Concourse Resources, Forge will source `resource_types` and `resources` from the working directory's [`.forge.yml`](.forge.yml) (overridable with `-c`). This schema is conveniently compatible with [Concourse's pipeline schema](https://concourse-ci.org/pipelines.html).
Forge is here to help.

> Just like Concourse itself, Forge ships with [some Resource Types builtin](concourse/builtin.go) that can be overridden.
```sh
forge get mock -v version=v0.0.0
```

Forge mounts the current working directory to the resource's.

You can also attach to the container executing the Resource to snoop around:

```sh
forge get -a mock -v version=v0.0.0
```

> The Resource's image must have `bash` or `sh` on its `PATH` for the attach to work.
### Google Cloudbuild steps

For Google Cloudbuild, Forge will try to source the default substitutions from the working directory's Git configuration as well as `~/.config/gcloud`.

```sh
# The `--` is important to signify to forge that the rest of the arguments are meant to be passed to the underlying command, not parsed by `forge` itself.
# The `''` are important to keep your shell from doing the substitution before `forge` can get ahold of it to the substitution itself.
forge cloudbuild gcr.io/cloud-builders/docker -- build -t 'gcr.io/${PROJECT_ID}/my-image:${SHORT_SHA}' .
```

Using a different entrypoint:

```sh
# The `""` are important to pass the entire `docker` command
# as the value to `bash`'s `-c` flag.
forge cloudbuild --entrypoint bash gcr.io/cloud-builders/docker -- -c "docker build -t 'gcr.io/${PROJECT_ID}/my-image:${SHORT_SHA}' ."
```

Forge mounts the current working directory to the step's as well as a cache directory respecting the XDG Base Directory Specification to the step's `/workspace`.

For additional debugging, you can attach to the container running the step:

```sh
forge cloudbuild -a gcr.io/cloud-builders/docker -- build -t 'gcr.io/${PROJECT_ID}/my-image:${SHORT_SHA}' .
```

> The step's image must have `bash` or `sh` on its `PATH` for the attach to work.
### as a library

```go
import (
// Some std imports omitted for brevity.

"github.com/docker/docker/client"
"github.com/frantjc/forge"
"github.com/frantjc/forge/githubactions"
"github.com/frantjc/forge/ore"
"github.com/frantjc/forge/runtime/docker"
)

func main() {
cli, err := client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
if err != nil {
fmt.Println(err)
os.Exit(1)
}

globalContext := githubactions.NewGlobalContextFromEnv()
globalContext.SecretsContext[githubactions.SecretActionsStepDebug] = githubactions.SecretActionsStepDebugValue
globalContext.SecretsContext[githubactions.SecretRunnerDebug] = githubactions.SecretRunnerDebugValue
globalContext.SecretsContext[githubactions.SecretActionsRunnerDebug] = githubactions.SecretActionsRunnerDebugValue
globalContext.GitHubContext.Repository = "frantjc/forge"

// Checkout https://github.com/frantjc/forge, using
// https://github.com/actions/checkout, grepping to
// only print debug logs.
if err = forge.NewFoundry(docker.New(cli)).Process(
ctx,
&ore.Lava{
From: &ore.Action{
Uses: "actions/checkout@v4",
GlobalContext: globalContext,
},
To: &ore.Pure{
Image: "alpine:3.19",
Entrypoint: []string{"grep", "debug"},
},
},
forge.StdDrains(),
); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
```

## why?

Automation begins with a shell script that executes a bunch of CLI commands often to test, build and publish some code. The next step is to set up some continuous integration (CI) system that executes that script in response to some event such as a commit to a Git repository's `main` branch. Such CI systems tend to identify that all of the scripts that they are executing do a lot of the same things--checkout a Git repository, setup a tool and so on.

In an effort to make their platform easier to use and to refactor the shared functionality out of all of the aforementioned scripts, CI systems in the past have introduced reusable "plugins"/"Actions"/"Resources"/"Tasks"/"Orbs" which take minimal configuration to do a complex task. GitHub Actions' [`actions/checkout`](https://github.com/actions/checkout), for example, takes one short line of code to invoke and accepts a bunch of optional configuration to fulfill many related use cases.

Unfortunately, using such powerful plugins outside of the the system they were built for can be wildly difficult. This makes debugging the use of these plugins require long feedback loops. It also makes migrating from one CI system to another treacherous, having to replace uses of one system's plugins with another's.

Forge aims to remedy this.

## developing
Forge is a library and CLI for running reusable steps from various proprietary CI systems using a pluggable container runtime. This, for example, makes the functionality provided to GitHub Actions easily consumable (or testable) by users of other CI systems.

- `git` is _required_
- `make` is _required_
- `go` 1.20 is _required_ for multi-error handling
- `docker` is _required_ to test as it is its only runtime
- [`upx`](https://github.com/upx/upx) is _required_ for compressing [`shim`](internal/cmd/shim/main.go)
- `node` 20 is _required_ for developing the [`action`](.github/action)
(**Learn more**)[docs/index.md].
2 changes: 1 addition & 1 deletion command/cache.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (

var (
runnerTmpArgs = []string{"runner_temp", "runnertemp", "runner_tmp", "runnertmp", "temp", "tmp"}
runnerToolCacheArgs = []string{"runner_tool_cache", "runner_toolcache", "runnertoolcache", "toolcache"}
runnerToolCacheArgs = []string{"runner_tool_cache", "runner_toolcache", "runnertoolcache", "toolcache", "tc"}
actionsArgs = []string{"github", "github_actions", "githubactions", "gha", "action", "actions"}
cloudbuildWorkspaceArgs = []string{"cloudbuild", "cb", "workspace"}
)
Expand Down
Loading

0 comments on commit 012788d

Please sign in to comment.