From 5351af840ef1aaf0db08193e5f15804190964d5a Mon Sep 17 00:00:00 2001 From: Mario Constanti Date: Wed, 20 Nov 2024 10:51:37 +0100 Subject: [PATCH] docs: document operator update path... (#216) ... and compatibility matrix --------- Signed-off-by: Mario Constanti --- README.md | 21 +-- docs/adrs/github_default_endpoint.md | 48 ++++++ docs/architectural-decision-records.md | 8 +- .../kube-state-metrics-config.md | 4 +- docs/operator_update.md | 161 ++++++++++++++++++ docs/readme.md | 9 + 6 files changed, 236 insertions(+), 15 deletions(-) create mode 100644 docs/adrs/github_default_endpoint.md create mode 100644 docs/operator_update.md diff --git a/README.md b/README.md index 568b4ccd..e68632cc 100644 --- a/README.md +++ b/README.md @@ -33,16 +33,17 @@ ### Garm Version -garm-operator uses [`garm-api-client`](https://github.com/cloudbase/garm/tree/main/client) to talk -with `garm` servers. The supported `garm` server version is determined by `garm-api-client`. +`garm-operator` is tightly coupled with the `garm` server. Therefore, the compatibility between `garm-operator` and `garm` is crucial. -> [!WARNING] -> `garm-operator` is currently tested with `garm` server version `v0.1.4`. Newer versions are not guaranteed to work. -> -> `garm` server version `>v0.1.4` introduces a few breaking changes. -> [provider]() and [credential]() configuration has moved away from being defined in the `garm` server configuration. -> Instead, they are now defined via the `garm` API itself. -> This feature set isn't implemented in the `garm-operator` yet. +| garm-operator | garm | compatible | +|---------------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `0.4.x` | `0.1.5` | :white_check_mark: | +| `0.4.x` | `0.1.4` | :x: | +| `0.3.x` | `0.1.4` | :white_check_mark: | +| `0.3.x` | `0.1.5` | :warning: It is not recommended to operate during a longer time period. This state mostly exists when you update the `garm-operator` from `0.3.x` to `0.4.x` | + +> [!NOTE] +> please also read the [operator update guide](docs/operator_update.md) when you want to update the `garm-operator`. ### Kubernetes Version @@ -92,7 +93,7 @@ For local development, please read the [development guide](DEVELOPMENT.md). To make some assumptions and corresponding decisions transparent, we use ADRs (Architecture Decision Records) to document them. -All ADRs can be found in the [here](docs/architectural-decision-records.md). +All ADRs can be found [here](docs/architectural-decision-records.md). ## Contributing diff --git a/docs/adrs/github_default_endpoint.md b/docs/adrs/github_default_endpoint.md new file mode 100644 index 00000000..875a91e0 --- /dev/null +++ b/docs/adrs/github_default_endpoint.md @@ -0,0 +1,48 @@ + + +--- +date: 2024-11-20 +desc: Reflection of the default GitHub endpoint +state: accepted +--- + + + +# Reflection of the default GitHub endpoint + +## Context and Problem Statement + +With `garm` in version `0.1.5`, the GitHub API endpoints aren't defined in the configuration file anymore. The GitHub API endpoints are now defined via the `garm` API. + +`garm` also ships with a default immutable GitHub API endpoint and points to the public GitHub API. + +With the new API version `v1beta1` for all `garm-operator` related resources, we have to make use of the new `GitHubEndpoint` object to define the connection to the GitHub API. +The `GitHubEndpoint` is referenced in the `GitHubCredential` object. + +## Decision Drivers + +* The default GitHub API endpoint is immutable. +* The GitHub API endpoint must be configured via a new `GitHubEndpoint` object which then got referenced in the `GitHubCredential` object. + +## Considered Options + +* Reflecting the default GitHub API endpoint back into the kubernetes cluster by creating a `GitHubEndpoint` object with the default values. + But as the default GitHub API endpoint is immutable, we have to catch this case in the operator and forbid any update/delete operations on this reflected `GitHubEndpoint`. + +## Pros and Cons of the Options + +#### Pros: +* The code must not handle the immutability of the default GitHub API endpoint. + +#### Cons: +* Everyone who wants to use the `garm-operator` with the default GitHub API endpoint has to create a `GitHubEndpoint` object. + +## Decision Outcome + +The validation rules in the `CRD` blocks the creation of a `GitHubEndpoint` object with the name `github.com`. \ No newline at end of file diff --git a/docs/architectural-decision-records.md b/docs/architectural-decision-records.md index 57422559..729e6abe 100644 --- a/docs/architectural-decision-records.md +++ b/docs/architectural-decision-records.md @@ -4,6 +4,8 @@ In this file we are collecting all our DHC CaaS architectural decision records. -| Date | Description | -|:-----------|:----------------------------------------------------| -| 2023-07-24 | [reduce login API calls](adrs/jwt_token_caching.md) | \ No newline at end of file +| Date | Description | +|:-----------|:-----------------------------------------------------------------------------| +| 2023-07-24 | [reduce login API calls](adrs/jwt_token_caching.md) | +| 2023-12-04 | [Garm Server Initialization](adrs/garm_server_initialization.md) | +| 2024-11-20 | [Reflection of the default GitHub endpoint](adrs/github_default_endpoint.md) | \ No newline at end of file diff --git a/docs/kube-state-metrics/kube-state-metrics-config.md b/docs/kube-state-metrics/kube-state-metrics-config.md index 197e4ad6..bec66b12 100644 --- a/docs/kube-state-metrics/kube-state-metrics-config.md +++ b/docs/kube-state-metrics/kube-state-metrics-config.md @@ -1,8 +1,8 @@ -# Kube State Metrics Configuration +# kube-state-metrics Configuration -[Here](../../config/kube-state-metrics/configmap.yaml) you will find a sample configuration for a kube-state-metrics agent to expose metrics of `garm-operators` custom resources. +[Here](../../config/kube-state-metrics/configmap.yaml) you will find a sample configuration for [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics) to expose metrics of `garm-operators` custom resources. If you are using the official [helm chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-state-metrics) you can place the contents of `.data.config.yaml` into your helm-charts `values.yaml` file like so: ```yaml diff --git a/docs/operator_update.md b/docs/operator_update.md new file mode 100644 index 00000000..360885c7 --- /dev/null +++ b/docs/operator_update.md @@ -0,0 +1,161 @@ + + +# compatibility + + +- [update to operator 0.4.x](#update-to-operator-04x) + - [1. update garm to version >=0.1.5](#1-update-garm-to-version-015) + - [2. update garm-operator to version 0.4.x](#2-update-garm-operator-to-version-04x) + - [3. create new CustomResources](#3-create-new-customresources) + + +## update to operator 0.4.x + +> [!WARNING] +> `garm-operator` in version `0.4.x` is not compatible with `garm` in version `0.1.4`. +> If you connect `garm-operator` in version `0.4.x` to `garm` in version `0.1.4`, +> the operator will stop working as we check the `garm` version in the operator. + +### 1. update `garm` to version `>=0.1.5` + +`garm-operator` in version `0.3.x` is compatible with `garm` in version `0.1.5`, if the `garm` server got updated from `0.1.4` to `0.1.5`. +This is because `garm` is doing some migration steps by moving parts of the configuration from the `garm` server to the `garm` API. +These newly introduced API endpoints are not by `garm-operator` in version `0.3.x`. + +Therefore it's possible to still work with the existing `CustomResources`. + +### 2. update `garm-operator` to version `0.4.x` + +Once `garm` got updated to version `0.1.5`, you can update the `garm-operator` to version `0.4.x`. + +### 3. create new `CustomResources` + +As `garm` moved some configuration parts to the API, you have to create a few new `CustomResources`. +Without these new `CustomResources`, `garm-operator` is not able to reconcile `Enterprises`, `Organizations` and `Repository` objects. + +#### 3.1 create a `GarmServerConfig` object + +It's now possible to define the `callbackUrl`, `metadataUrl` and `webhookUrl` via a `GarmServerConfig` object. Therefore it's not needed anymore to restart the `garm` server to apply these changes. + +```yaml +apiVersion: garm-operator.mercedes-benz.com/v1beta1 +kind: GarmServerConfig +metadata: + name: garm-server-config + namespace: garm-operator-system +spec: + callbackUrl: http:///api/v1/callbacks + metadataUrl: http:///api/v1/metadata + webhookUrl: http:///api/v1/webhook +``` + +By running `kubectl get garmserverconfig` you should see the newly created `GarmServerConfig` object. + +```bash +NAME ID VERSION AGE +garm-server-config dd2524b9-0789-499d-ba7d-3dba65cf9d3f v0.1.5 16h +``` + +More information about the defined Urls can be found [in the garm documentation](https://github.com/cloudbase/garm/blob/main/doc/using_garm.md#controller-operations). + +#### 3.2 create `GitHubEndpoint` objects + +The `GitHubEndpoint` objects are used to define the connection to the GitHub API. It's possible to define multiple `GitHubEndpoint` objects to connect to different GitHub instances. + +```yaml +apiVersion: garm-operator.mercedes-benz.com/v1beta1 +kind: GitHubEndpoint +metadata: + name: github-enterprise + namespace: garm-operator-system +spec: + description: "github" + apiBaseUrl: "https://github.com" + uploadBaseUrl: "https://uploads.github.com" + baseUrl: "https://api.github.com/" +``` + +By running `kubectl get githubendpoint` you should see the newly created `GitHubEndpoint` object. + +```bash +NAME URL READY AGE +github https://github.com True 3m2s +``` + +More information about the github endpoints can be found [in the garm documentation](https://github.com/cloudbase/garm/blob/main/doc/using_garm.md#github-endpoints) + +> [!NOTE] +> `garm` already ships a default `Github` object. But as this object is immutable, and we do not wanted to reflect the object into the `garm-operator`, we decided to create a new `CustomResource` for this. See the [Reflection of the default GitHub endpoint ADR](docs/adrs/github_default_endpoint.md) for more information. + +#### 3.2 create `GitHubCredential` objects + +With the new `v1beta1` of `Enterprises`, `Organizations` or `Repositories`, the reference to the Github credential has changed. +In the previous `v1alpha1`, the used credential was referenced by the `.spec.credentialsName` field, which then pointed to a credential object, specified in the `garm` server configuration. + +With the new `v1beta1`, the reference to the credential is done by the `.spec.credentialRef` field, which points to a `GitHubCredential` object. + +> [!NOTE] +> As long as `GitHubCredential` is not applied in `v1beta1`, the `garm-operator` will quit reconciling `Enterprises`, `Organizations` or `Repositories`. +> The current assumption is, that a `GitHubCredential`, which was set in `.spec.credentialsName` in `v1alpha1`, will be created as `GitHubCredential` in `v1beta1` with the same name. +> If this is not the case, the `garm-operator` will not be able to reconcile the `Enterprises`, `Organizations` or `Repositories` objects. +> This state can be seen in the `status.conditions` field of these objects. +> +> ```yaml +> status: +> conditions: +> - lastTransitionTime: "2024-11-14T20:01:27Z" +> message: GitHubCredential.garm-operator.mercedes-benz.com "github-pat" not found +> reason: CredentialsRefFailed +> status: "False" +> type: Ready +> - lastTransitionTime: "2024-11-14T20:01:27Z" +> message: GitHubCredential.garm-operator.mercedes-benz.com "github-pat" not found +> reason: CredentialsRefFailed +> status: "False" +> type: CredentialsRef +> [...] +> ``` + +Create the `GitHubCredential` object and a corresponding `Secret` object with the PAT token. + +```yaml +apiVersion: garm-operator.mercedes-benz.com/v1beta1 +kind: GitHubCredential +metadata: + name: github-pat + namespace: garm-operator-system +spec: + description: PAT for github + endpointRef: + apiGroup: garm-operator.mercedes-benz.com + kind: GitHubEndpoint + name: github + authType: pat + secretRef: + name: github-pat + key: token +--- +apiVersion: v1 +kind: Secret +metadata: + name: github-pat + namespace: garm-operator-system +data: + token: +``` + +More information about the github credentials can be found [in the garm documentation](https://github.com/cloudbase/garm/blob/main/doc/using_garm.md#github-credentials) + +By running `kubectl get githubcredential` you should see the newly created `GitHubCredential` object. + +```bash +NAME ID READY AUTHTYPE GITHUBENDPOINT AGE +github-pat 1 True pat github 3m51s +``` + +By running `kubectl get githubcredential -o wide` you see all the `Enterprise`, `Organization` or `Repository` objects which are using this `GitHubCredential`. + +```bash +NAME ID READY ERROR AUTHTYPE GITHUBENDPOINT REPOSITORIES ORGANIZATIONS ENTERPRISES AGE +github-pat 1 True pat github ["my-org"] 4m46s +``` \ No newline at end of file diff --git a/docs/readme.md b/docs/readme.md index f7746d6b..5143156c 100644 --- a/docs/readme.md +++ b/docs/readme.md @@ -4,15 +4,20 @@ - [Quickstart](#quickstart) +- [Architecture Decision Records](#architecture-decision-records) - [how to](#how-to) - [scale runners](#scale-runners) - [pause reconciliation](#pause-reconciliation) + - [configure the operator](#configure-the-operator) + - [monitor operator CRs](#monitor-operator-crs) ## Quickstart [Quickstart](quickstart.md) contains a walk-through of how to get started with `garm-operator`. +## [Architecture Decision Records](architectural-decision-records.md) + ## how to ### scale runners @@ -97,3 +102,7 @@ $ kubectl annotate organization developer-org garm-operator.mercedes-benz.com/pa organization.garm-operator.mercedes-benz.com/developer-org annotate ``` + +### [configure the operator](config/configuration-parsing.md) + +### [monitor operator CRs](kube-state-metrics/kube-state-metrics-config.md) \ No newline at end of file