Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CP-19492: golang based cloudzero-chart validator #1

Merged
merged 9 commits into from
Jul 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @cloudzero/cirrus
38 changes: 38 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
name: Report an issue
about: Create a bug report to fix an existing issue.
title: ''
labels: ''
assignees: ''

---
**Please do not report security vulnerabilities here**. Please disclose all security issues to [security@cloudzero.com](mailto:security@cloudzero.com).

**Thank you in advance for helping us to improve this project!** Please read through the template below and answer all relevant questions. Your additional work here is greatly appreciated and will help us respond as quickly as possible. For general support or usage questions, use [CloudZero Support](mailto:support@cloudzero.com). Finally, to avoid duplicates, please search existing Issues before submitting one here.

By submitting an Issue to this repository, you agree to the terms within the [CloudZero Code of Conduct](https://github.com/cloudzero/template-cloudzero-open-source/blob/master/CODE-OF-CONDUCT.md).

### Description

> Provide a clear and concise description of the issue, including what you expected to happen.

### Reproduction

> Detail the steps taken to reproduce this error, what was expected, and whether this issue can be reproduced consistently or if it is intermittent.
>
> Where applicable, please include:
>
> - Code sample to reproduce the issue
> - Log files (redact/remove sensitive information)
> - Application settings (redact/remove sensitive information)
> - Screenshots

### Environment

> Please provide the following:

- **Version of this project used:**
- **Version of the platform or framework used, if applicable:**
- **Other relevant versions (language, server software, OS, browser):**
- **Cloud provider and cloud services used (EKS, K8S, EC2, Etc...)**
- **Other modules/plugins/libraries that might be involved:**
29 changes: 29 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
name: Feature request
about: Suggest new functionality for this project.
title: ''
labels: ''
assignees: ''

---
**Please do not report security vulnerabilities here**. Please disclose all security issues to [security@cloudzero.com](mailto:security@cloudzero.com).

**Thank you in advance for helping us to improve this project!** Please read through the template below and answer all relevant questions. Your additional work here is greatly appreciated and will help us respond as quickly as possible. For general support or usage questions, use [CloudZero Support](mailto:support@cloudzero.com). Finally, to avoid duplicates, please search existing Issues before submitting one here.

By submitting an Issue to this repository, you agree to the terms within the [CloudZero Code of Conduct](https://github.com/cloudzero/template-cloudzero-open-source/blob/master/CODE-OF-CONDUCT.md).

### Describe the problem you'd like to have solved

> A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

### Describe the ideal solution

> A clear and concise description of what you want to happen.

## Alternatives and current work-arounds

> A clear and concise description of any alternatives you've considered or any work-arounds that are currently in place.

### Additional context

> Add any other context or screenshots about the feature request here.
18 changes: 18 additions & 0 deletions .github/dependabot.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
version: 2

updates:
- package-ecosystem: gomod
directory: "/"
schedule:
interval: "daily"
open-pull-requests-limit: 10
labels:
- "dependencies"

- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
open-pull-requests-limit: 10
labels:
- "dependencies"
77 changes: 77 additions & 0 deletions .github/workflows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# Test driving workflows

To run the GitHub Actions workflows locally using the act utility, you can follow these steps:

## 1. Get the `act` tool

You can install act using Homebrew on macOS, or download it directly for other platforms.

```sh
brew install act
```

Or download it from the [GitHub releases page](https://github.com/nektos/act).

## 2. Run the workflows manually using act:

act allows you to simulate the GitHub Actions environment and execute the workflows as if they were running on GitHub.

**_Note - for the following commands, it is assumed you are in the base directory of the repository - and have the following environment variables set:_**

* `GH_USER` - set to your github user name (such as josephbarnett)
* `GH_PAT` - set to your github personal access token. This token should have repo write permissions, and package write permissions

Now, you can run the following workflow simulations.

### Manually Trigger the Merge Workflow

The manual merge workflow [release-to-main.yml](release-to-main.yml) will perform a sync merge from the `develop` branch to `main`.

All releases are based on `main`, where as `develop` is incrementally changing until we are ready to release.

To manual trigger the workflow, use the following command:

```sh
act --container-architecture linux/arm64 \
-a $GH_USER --secret GITHUB_TOKEN=$GH_PAT \
-j release-to-main
```

### Manually Trigger the DockerBuild Workflow:

For the DockerBuild workflow [docker-build.yml](docker-build.yml), simulate a push to the main branch, develop branch, and a new release tag. It will automatically build a docker image from the repository code, scan it for security vulnerabilties, then if on main or develop - will publish the docker image to the public GHCR repository associated with the repository.

The following sub-sections allow you to simulate these events and run each permuation.

#### 1. Simulate a Push to develop Branch

> Don't forget to update the `.json` file first before running the command!

```sh
act --container-architecture linux/arm64 \
-a $GH_USER --secret GITHUB_TOKEN=$GH_PAT \
-j docker --eventpath .github/workflows/events/develop-push-event.json
```

#### 2. Simulate a Push to main Branch

> Don't forget to update the `.json` file first!

```sh
act --container-architecture linux/arm64 \
-a $GH_USER --secret GITHUB_TOKEN=$GH_PAT \
-j docker --eventpath .github/workflows/events/main-push-event.json
```

#### 3. Simulate a Release Event

> Don't forget to update the `.json` file first!

```sh
act --container-architecture linux/arm64 \
-a $GH_USER --secret GITHUB_TOKEN=$GH_PAT \
-j docker --eventpath .github/workflows/events/release-event.json
```


By using these commands, you can test your workflows locally and verify their functionality before pushing them to GitHub. This ensures that your workflows are working correctly without needing to trigger them on the actual repository.
172 changes: 172 additions & 0 deletions .github/workflows/docker-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
name: DockerBuild
on:
push:
branches:
- develop
- main
tags:
- '*'
# tag is pr-<number>
pull_request:
release:
types:
- created
- published
- released

env:
REGISTRY_LOCAL_ADDR: localhost:5000
REGISTRY_PROD_ADDR: ghcr.io
IMAGE_NAME: ${{ github.repository }}/cloudzero-agent-validator

jobs:
# This job lints the chart
josephbarnett marked this conversation as resolved.
Show resolved Hide resolved
docker:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
id-token: write
services:
registry:
image: registry:2
ports:
- 5000:5000
steps:
# Checkout the repository code
- name: SETUP - Checkout
id: checkout_code
uses: actions/checkout@v4

- # Install buildx for multi-platform builds
name: SETUP - Docker Buildx
id: install_buildx
uses: docker/setup-buildx-action@4fd812986e6c8c2a69e18311145f9371337f27d4 # v3.4.0
with:
driver-opts: network=host

# Sanity Check: Validate the k8s and Registry is Running
- name: SANITY CHECK - Registry are running
id: validate_kind_install
run: |
docker pull busybox
docker tag busybox ${{ env.REGISTRY_LOCAL_ADDR }}/localbusybox
docker push ${{ env.REGISTRY_LOCAL_ADDR }}/localbusybox

# Format the image name to OCI compatable format
- name: INPUT PREP - image name formatting
id: image_name
run: |
IMAGE_NAME=${{ env.IMAGE_NAME }}
echo "IMAGE_NAME=${IMAGE_NAME,,}" >>${GITHUB_ENV}

# Extract metadata (tags, labels) the docker image build
- name: INPUT PREP - Extract Docker metadata from git repository
id: meta
uses: docker/metadata-action@8e5442c4ef9f78752691e2d8f8d19755c6f78e81 # v5.5.1
env:
VALIDATOR_IMAGE_DESCRIPTION: "CloudZero Agent Validator"
with:
# ONLY use the local registry address for the image until it is tested
images: ${{ env.REGISTRY_LOCAL_ADDR }}/${{ env.IMAGE_NAME }}
# Tag generation rules:
# 1. branch name (used for develop or main)
# 2. PR number (used for PRs)
# 3. version to match the semver pattern for the chart
tags: |
josephbarnett marked this conversation as resolved.
Show resolved Hide resolved
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
labels: |
maintainer=CloudZero
org.opencontainers.image.description=${{ env.VALIDATOR_IMAGE_DESCRIPTION }}
org.opencontainers.image.vendor=CloudZero
image.name=${{ env.REGISTRY_PROD_ADDR }}/${{ env.IMAGE_NAME }}
# https://github.com/docker/metadata-action?tab=readme-ov-file#latest-tag
# should only occur whtn a semver or raw when we are on master
flavor: |
latest=auto

- name: INPUT PREP - Set build time revision
run: |
REVISION=$(git rev-parse --short HEAD)
TAG=$(echo "${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}")
BUILD_TIME=$(date -u +'%Y-%m-%dT%H:%M:%SZ')
echo "REVISION=${REVISION}" >>${GITHUB_ENV}
echo "TAG=${TAG}" >>${GITHUB_ENV}
echo "BUILD_TIME=${BUILD_TIME}" >>${GITHUB_ENV}

- name: TEST - Build image
id: build_image
uses: docker/build-push-action@1a162644f9a7e87d8f4b053101d1d9a712edc18c # v6.3.0
env:
PLATFORMS: "linux/amd64,linux/arm64"
VALIDATOR_DOCKERFILE: docker/Dockerfile
VALIDATOR_CONTEXT: .
with:
push: true
context: ${{ env.VALIDATOR_CONTEXT }}
file: ${{ env.VALIDATOR_DOCKERFILE }}
tags: ${{ steps.meta.outputs.tags }}
josephbarnett marked this conversation as resolved.
Show resolved Hide resolved
labels: ${{ steps.meta.outputs.labels }}
platforms: ${{ env.PLATFORMS }}
cache-from: type=gha
cache-to: type=gha,mode=max
build-args: |
BUILD_TIME=${{ env.BUILD_TIME }}
REVISION=${{ env.REVISION }}
TAG=${{ env.TAG }}
josephbarnett marked this conversation as resolved.
Show resolved Hide resolved
- name: SECURITY - Grype Docker Image Scan
uses: anchore/scan-action@v3
with:
image: ${{ env.REGISTRY_LOCAL_ADDR }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }}
fail-build: true
severity-cutoff: high

- name: SECURITY - Trivy Docker Image Scan
uses: aquasecurity/trivy-action@0.23.0
with:
image-ref: ${{ env.REGISTRY_LOCAL_ADDR }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }}
format: 'table'
exit-code: '1'
ignore-unfixed: true
vuln-type: 'os,library'
severity: 'CRITICAL,HIGH'

###########################################################################
# PRODUCTION ONLY STEPS BEYOND THIS POINT
#
# install regctl for registry management operations
- name: PRODUCTION STEP - Install Regctl for registry management
if: github.event_name == 'release' || github.ref == 'refs/heads/develop' || github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
id: install_regctl
uses: iarekylew00t/regctl-installer@v1

# Login to product docker registry
- name: PRODUCTION STEP - login to container registry
if: github.event_name == 'release' || github.ref == 'refs/heads/develop' || github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
id: prod_registry_login
run: |
echo "${{ secrets.GITHUB_TOKEN }}" | \
regctl registry login ${{ env.REGISTRY_PROD_ADDR }} \
--user "${{ github.actor }}" \
--pass-stdin

# Copy the image from the local registry
# to the production registry (retagging at the same time)
# only allow on main, develop branches, or a version tag
- name: PRODUCTION STEP - Publish Image to Production
if: github.event_name == 'release' || github.ref == 'refs/heads/develop' || github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v')
id: prod_publish_image
run: |
regctl registry set --tls=disabled ${{ env.REGISTRY_LOCAL_ADDR }}
regctl image copy \
${{ env.REGISTRY_LOCAL_ADDR }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }} \
josephbarnett marked this conversation as resolved.
Show resolved Hide resolved
${{ env.REGISTRY_PROD_ADDR }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }}

if [[ ${{ steps.meta.outputs.version }} =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
regctl image copy \
${{ env.REGISTRY_LOCAL_ADDR }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }} \
${{ env.REGISTRY_PROD_ADDR }}/${{ env.IMAGE_NAME }}:latest
fi
6 changes: 6 additions & 0 deletions .github/workflows/events/develop-push-event.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"ref": "refs/heads/develop",
"repository": {
"full_name": "josephbarnett/cloudzero-agent-validator"
}
}
6 changes: 6 additions & 0 deletions .github/workflows/events/main-push-event.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"ref": "refs/heads/main",
"repository": {
"full_name": "cloudzero/cloudzero-agent-validator"
}
}
6 changes: 6 additions & 0 deletions .github/workflows/events/release-event.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"ref": "refs/tags/v0.9.0",
"repository": {
"full_name": "cloudzero/cloudzero-agent-validator"
}
}
21 changes: 21 additions & 0 deletions .github/workflows/golang-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: GoTest

on: [push]

jobs:
build:

runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Setup Golang
uses: actions/setup-go@v5
with:
go-version: '1.22.x'
cache-dependency-path: go.sum
- name: Install dependencies
run: |
go mod download
- name: Run go tests
run: |
go test -timeout 30s -race -cover ./...
Loading