Welcome! We gladly accept contributions on new conformance tests, example CNFs, updates to documentation, enhancements, bug reports and more.
CNF Conformance is Apache 2.0 licensed and accepts contributions via GitHub pull requests. Please read the following guidelines carefully to make it easier to get your contribution accepted.
Support channels include:
Before starting work on a major feature, please reach out to us via GitHub Issues or Slack. We will make sure no one else is already working on it and ask you to open a GitHub issue.
- Small patches and bug fixes don't need prior communication.
Issues are used as the primary method for tracking items in the CNF Conformance initiative. Please self-assign an issue to yourself when you start to work on it so we don't duplicate work :)
1. New Features: To request an enhancement, please create a new issue using the Feature Request Template
2. Report Bugs: To report a bug, please create a new issue using the Bug Report Template. Check out How to Report Bugs Effectively.
NOTE: you can enable higher logging level output via the command line or env var. to help with debugging
# cmd line
./cnf-conformance -l debug test
# make sure to use -- if running from source
crystal src/cnf-conformance.cr -- -l debug test
# env var
LOGLEVEL=DEBUG ./cnf-conformance test
Also setting the verbose option for many tasks will add extra output to help with debugging
crystal src/cnf-conformance.cr test_name verbose
Check usage documentation for more info about invoking commands and loggin
3. New Conformance Tests:
- To request a new workload test, please create a new issue using the New Workload Test Template
- To request a new platform test, please create a new issue using the New Platform Test Template
4. New CNF Example: To suggest a new CNF, please create a GitHub issue using the New Example CNF template.
To install the CNF Conformance test suite and run a CNF, follow instructions at:
The test framework and tests (using upstream tools) are written in the human readable, compiled language, Crystal. Common capabilities like dependencies between tests and categories are supported.
Outline of what a contributor's workflow looks like:
- Fork it (https://github.com/cncf/cnf-conformance/fork)
- Create a branch from where you want to base your work (usually master). Example
git checkout -b my-new-feature) - Read the [INSTALL.md](install for build and test instructions)
- Make your changes and arrange them in readable commits.
- Commit your changes (Ex. `git commit -am 'Add some feature'``)
- Make sure your commit messages are in the proper format (see below).
- Push to the branch (Ex.
git push origin my-new-feature) - Make sure branch is up to date with upstream base branch (eg.
master) - Make sure all tests pass, and add any new tests as appropriate.
- Create a new Pull Request (PR)
Once you have implemented the feature or bug fix in your branch, you will open a PR to the upstream cnf-conformance repo. Before opening the PR ensure you rebased on the latest upstream, have added spec tests, if needed, all spec tests are passing.
In order to open a pull request (PR) it is required to be up to date with the latest changes upstream. If other commits are pushed upstream before your PR is merged, you will also need to rebase again before it will be merged.
Using the automated pull request template, please note a description of the changes, the type of change, the issue(s) related to the PR, how the changes have been tested and if updates are needed in the documentation.
For general advice on how to submit a pull request, please see Creating a pull request.
The CNF Conformance team meets once a week on Thursdays at 14:15-15:00 UTC.
- Meeting minutes are here
Thank you for your contributions. We appreciate your help and look forward to collaborating with you!