command-bot provides interfaces for executing pre-defined commands on GitLab CI.
Before starting to work on this project, we recommend reading the Implementation section.
command-bot executes arbitrary commands on GitLab CI from commands in pull request comments (the GitHub App has to be installed in the repository) and from API requests.
Comment in a pull request:
bot [command] [bot-args] $ [args]
In [bot-args] are optional, you can provide the following options
-v/--var(optional): defines environment variables for the CI job which runs the command. You can specify this option multiple times for multiple variables.
bot help
Bot responds with an actual list of commands generated from pipeline.
In the pull request where you previously ran bot queue, comment:
bot cancel
The API provides an alternative interface for executing commands directly without having to go through pull request comments.
Use a Master Token for queueing a command through POST /api/queue.
curl \
-H "X-Auth: $token" \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-X POST http://command-bot/api/queue \
-d '{
"configuration": "bench-bot",
"args": ["runtime", "westend-dev", "pallet_balances"],
"variables": {
"RUST_LOG": "info"
},
"gitRef": {
"contributor": {
"owner": "user",
"repo": "substrate",
"branch": "benchmarking-test"
},
"upstream": {
"owner": "paritytech",
"repo": "substrate"
}
}
}'For bench-bot you can optionally specify a string in gitRef.upstream.branch
which is the name of the branch to be merged into the contributor branch
before benchmarks.
POST /api/queue will return a { task: { id: string } } response. The
task.id can used for cancelling an ongoing command through DELETE /api/task/:task_id.
curl \
-H "X-Auth: $token" \
-H "Content-Type: application/json" \
-X DELETE http://command-bot/api/task/${TASK_ID}The GitHub App is necessary for the application to receive webhook events and access the GitHub API properly.
Follow the instructions of https://gitlab.parity.io/groups/parity/opstooling/-/wikis/Bots/Development/Create-a-new-GitHub-App for creating a new GitHub App.
After creating the app, you should configure and install it (make sure the environment is properly set up before using it).
Configuration is done at https://github.com/settings/apps/${APP}/permissions.
- Issues: Read-only
- Allows for interacting with the comments API
- Pull Requests: Read & write
- Allows for posting comments on pull requests
- Contents: Read-only
- Allows for cloning repositories before running commands
- Members: Read-only
- Related to
$ALLOWED_ORGANIZATIONS: this permission enables the bot to request the organization membership of the command's requester even if their membership is private
- Related to
- Issue comment
- Allows for receiving events for pull request comments
Having created and configured the
GitHub App, install it in a repository through
https://github.com/settings/apps/${APP}/installations.
- Node.js for running the application
yarnfor installing packages and running scripts- If it's not already be bundled with Node.js, install with
npm install -g yarn
- If it's not already be bundled with Node.js, install with
gitfor cloning branches before executing try-runtime-cli- RocksDB's build requirements
- Please check the generateDockerfile for the relevant packages
All environment variables are documented in the
.env.example.cjs file. For development you're welcome to
copy that file to .env.cjs so that all values will be loaded automatically
once the application starts.
-
Set up the command-bot application
During development it's handy to use a smee.io proxy, through the
WEBHOOK_PROXY_URLenvironment variable, for receiving GitHub Webhook Events in your local server instance. -
Install the GitHub app in a GitHub repository
-
Create a repository in GitLab and set up
GITLAB_PUSH_NAMESPACEalong withGITLAB_ACCESS_TOKENto run the commands for the GitHub repository (Step 3).Note: The GitLab repository name should match how the repository is named on GitHub.
-
Run
yarnto install the dependencies -
Run
yarn devto start a development server oryarn watchfor a development server which automatically restarts when you make changes to the source files -
Trigger the commands in the repositories where you've installed the GitHub App (Step 3) and check if it works
The
sampleconfiguration is available for debugging purposes.bot sample $ hiwill runecho hiin a GitLab job (GitLab repository from Step 4).
See https://gitlab.parity.io/groups/parity/opstooling/-/wikis
When you push a deployment tag to GitHub, it will be mirrored to GitLab and then its CI pipeline will be run for deploying the app.
The application can be deployed to the following environments:
-
Production
To build and deploy: Either push a tag with the pattern
/^v-[0-9]+\.[0-9]+.*$/or trigger a new pipeline withBUILDset toproduction.To only deploy (an existing tag): trigger a new pipeline with
DEPLOYset toproduction.
Step 1: Create a Task for a command request
A request is turned into a
task
either
via API or
through a pull request Webhook event
(which are delivered from GitHub
as HTTP POST requests).
Step 2: Queue the task
The task is saved to the database so that it will be retried later in case it can't be finished (e.g. due to a container restart or crash).
TODO: Update this to reflect command-bot refactor.
Step 3: Get the result
Take the result from the command's execution and post it as a pull request comment if it originated from a pull request comment or send it to server logs if it originated from an API request.