Skip to content

Latest commit

 

History

History
310 lines (215 loc) · 13.1 KB

DEVELOPMENT.md

File metadata and controls

310 lines (215 loc) · 13.1 KB
Raw

Development

This doc explains the development workflow so you can get started contributing to Skaffold!

Requirements

You must install these tools:

  1. go: The language skaffold is built in (version >= go 1.19)
  2. git: For source control
  3. make: For building skaffold.

Getting started

First you will need to setup your GitHub account and create a fork:

  1. Create a GitHub account
  2. Setup GitHub access via SSH
  3. Create and checkout a repo fork

Once you have those, you can iterate on skaffold:

  1. Build your dev version of skaffold
  2. Verify changes locally
  3. Run skaffold tests
  4. Build docs if you are making doc changes

When you're ready, you can create a PR!

Checkout your fork

To make local changes to skaffold and eventually submit a pull request to the repo, we recommend creating your own fork.

Once you've done this, clone your fork to your local machine:

git clone git@github.com:${YOUR_GITHUB_USERNAME}/skaffold.git
cd skaffold
git remote add upstream git@github.com:GoogleContainerTools/skaffold.git
git remote set-url --push upstream no_push

Adding the upstream remote sets you up nicely for regularly syncing your fork.

Creating a PR

When you have changes you would like to propose to skaffold, you will need to:

  1. Ensure the commit message(s) follow the conventional commits guidelines. The Skaffold repo has a bot to check that commit messages follow this style.
  2. Ensure the PR description describes what issue you are fixing and how you are fixing it (include references to issue numbers if appropriate)
  3. Add unit tests. Unit test coverage should increase or stay the same with every PR.
  4. Add integration test if applicable
  5. Create a pull request

Please follow our small Pull Requests guidelines for quicker response time.

Reviews

Each PR must be reviewed by a maintainer. This maintainer will add the kokoro:run label to a PR to kick off the integration tests, which must pass for the PR to be submitted.

Making a config change

Some changes to the skaffold code require a change to the skaffold config. These changes require a few extra steps:

  • Open the latest Config at pkg/skaffold/schema/latest/config.go and inspect the comment at L28

  • If the line mentions the config version is not released, proceed making your changes.

    // This config version is not yet released, it is SAFE TO MODIFY the structs in this file.

  • If the line mentions the config version is released then,

    // !!! WARNING !!! This config version is already released, please DO NOT MODIFY the structs in this file.

    • Run ./hack/new-version.sh to create a new version.

    • Run make test to verify changes.

    • Commit these generated changes, and submit a PR.

Once you've done this, merge or rebase your development branch with config changes, including the new config change. Any new config changes belong in pkg/skaffold/schema/latest/config.go. Do not edit the older config versions.

  • Be sure and update the documentation in pkg/skaffold/schema/<previous_config_version>/upgrade.go with any additions, removals, or updates you make to the config.

  • In case of backwards compatibility issues, update the Upgrade() method from the previous config version appropriately. This is usually required when a previously existing field in the config is changed, but not when a new field is added.

Note: the Upgrade() method is called by skaffold automatically for older config versions. This can also be done manually by users by running skaffold fix.

  • Finally, before committing your final changes and opening your pull request, be sure and run make test locally. This will regenerate the JSON schemas for the skaffold config with your new changes. Commit the resulting changes autogenerated by the scripts.

For more details behind the logic of config changes see the Skaffold config management doc.

Making changes to the Skaffold API

We build the API directly through gRPC, which gets translated into REST API through a reverse proxy gateway library. When adding new message types, make changes to proto/v2/skaffold.proto, and when adding new enum types, make changes to proto/enums/enums.proto. When changing either of these files, you can run ./hack/generate-proto.sh to generate the equivalent Go code.

Adding actionable error messages to code.

Skaffold has a built-in framework to provide actionable error messages for user to help bootstrap skaffold errors.

Also, v1.19.0 onwards, skaffold is collecting failure error codes to help the team get more insights into common failure scenarios.

To take advantage of this framework, contributors can simply use the ErrDef struct to throw meaningful actionable error messages and improve user experience.

e.g In this example PR,

  1. The contributor created distinct error codes in enums.proto 
     // The Kptfile cannot be created via `kpt pkg init`.
 RENDER_KPTFILE_INIT_ERR = 1501;
 // The Kptfile is not a valid yaml file
 RENDER_KPTFILE_INVALID_YAML_ERR = 1401;
 // The Kptfile is not a valid API schema
 RENDER_KPTFILE_INVALID_SCHEMA_ERR = 1402;
    The INIT in this case stands for skaffold INIT phase which includes, parsing of skaffold config and creating a skaffold runner. The other valid phases are BUILD, DEPLOY, STATUSCHECK. Complete list here
  2. Run hack/generate-proto.sh. These will generate go code and structs for the newly added proto fields. 
     git status
    modified:   docs-v1/content/en/api/skaffold.swagger.json
    modified:   docs-v1/content/en/docs/references/api/grpc.md
    modified:   proto/enums/enums.pb.go
    modified:   proto/enums/enums.proto
  3. The contributor then used these error codes when creating an error in their proposed code change. They used the constructor sErrors.NewErrorWithStatusCode in pkg/skaffold/errors to instantiate an object of struct ErrDef. ErrDef implements the golang error interface. 
     err :=  sErrors.NewErrorWithStatusCode(
 proto.ActionableErr{
   Message: fmt.Sprintf("unsupported validator %q", c.Name),
   ErrCode: proto.StatusCode_CONFIG_UNKNOWN_VALIDATOR,
   Suggestions: []*proto.Suggestion{
     {
        SuggestionCode: proto.SuggestionCode_CONFIG_ALLOWLIST_VALIDATORS,
        Action: fmt.Sprintf(
 	  	"please only use the following validators in skaffold-managed mode: %v. "+
 		 "to use custom validators, please use kpt-managed mode.", AllowlistedValidators),
    },
   },
 })

With above two changes, skaffold will now show a meaning full error message when this error condition is met.

skaffold dev 
unsupported validator "foo" please only use the following validators in skaffold-managed mode: [kubeval].
To use custom validators, please use kpt-managed mode.

Building skaffold

To build with your local changes you have two options:

  1. Build the skaffold binary:

    make
./out/skaffold version

    You can then run this binary directly, or copy/symlink it into your path.

  2. Build and install the skaffold binary:

    make install
skaffold version

    This will install skaffold via go install (note that if you have manually downloaded and installed skaffold to /usr/local/bin, this will probably take precedence in your path over your $GOPATH/bin).

    If you are unsure if you are running a released or locally built version of skaffold, you can run skaffold version - output which includes dirty indicates you have built the binary locally.

Verifying local changes

If you are iterating on skaffold and want to see your changes in action, you can:

  1. Build skaffold
  2. Use the quickstart example

Testing skaffold

skaffold has both unit tests and integration tests.

Unit Tests

The unit tests live with the code they test and can be run with:

make quicktest

You can also run a larger suite of tests/checks (unit tests, docs check, linters check) using:

make test

In case you see a linter error such as:

make test
RUN hack/linter.sh
ERRO Running error: no such linter "gocritic"

re-run the hack/install-golint.sh script to upgrade golangci-lint.

Integration tests

The integration tests live in integration. They can be run with:

make integration

These tests require a Docker daemon, a Kubernetes cluster and all the tools used by every Builder and Deployer, such as kubectl, bazel, java, kustomize...

A way to run the integration tests without installing those tools and without depending on a Kubernetes cluster is to install kind and run:

make integration-in-kind

Running a subset of integration tests

You can select specific integration tests to run via the INTEGRATION_TEST_ARGS env var:

INTEGRATION_TEST_ARGS="-run=TestDev/" make integration

Running GCP specific integration tests

Another set of the integration tests require a GCP project because they will push to a GCR registry or use Cloud Build to build artifacts. Those tests can be run with:

GCP_ONLY=true make integration

These tests will be kicked off by reviewers for submitted PRs.

Building skaffold docs

The latest version of the skaffold site is based on the Hugo theme of the github.com/google/docsy template.

Testing docs locally

Before creating a PR with doc changes, we recommend that you locally verify the generated docs with:

make preview-docs

Once PRs with doc changes are merged, they will get automatically published to the docs for the latest build to https://skaffold-latest.firebaseapp.com. which at release time will be published with the latest release to https://skaffold.dev.

Previewing the docs on the PR

Mark your PR with docs-modifications label. Our PR review process will answer in comments in ~5 minutes with the URL of your preview and will remove the label.

Testing the Skaffold release image building process

Skaffold's release image build process works with Google Cloud Build within our own project k8s-skaffold.

In order to be able to iterate/fix the release process you can pass in your own project as a parameter to the build.

We continuously release edge images under gcr.io/k8s-skaffold/skaffold:edge. This is done by triggering cloudbuild.yaml on every push to main.

To run a build on your own project:

gcloud builds submit --config deploy/cloudbuild.yaml --substitutions=COMMIT_SHA=$(git rev-parse HEAD) --project <personal_project>

We build the latest stable release image under gcr.io/k8s-skaffold/skaffold:latest. This is done by triggering cloudbuild-release.yaml on every new tag in our Github repo.

To test a release on your own project:

gcloud builds submit --config deploy/cloudbuild-release.yaml --substitutions=TAG_NAME=<release_tag> --project <personal_project>

To just run a release image build without Google Cloud Build only using your local Docker daemon, you can run:

make -j release GCP_PROJECT=<personalproject>