Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ResourceGroup is a declarative API for generating a group of Kubernetes objects based on a matrix of input values and a set of templated resources.
The ResourceGroup API offers a high-level abstraction for defining and managing Flux resources and related Kubernetes objects as a single unit. It is designed to reduce the complexity of Kustomize overlays by providing a compact way
of defining different configurations for a set of workloads per tenant and/or environment.
Use cases:
Example
The following example shows a
ResourceGroup
that generates an application instance consisting of a FluxHelmRelease
andOCIRepository
for each tenant with a specific version and replica count.Writing a ResourceGroup spec
As with all other Kubernetes config, a ResourceGroup needs
apiVersion
,kind
, andmetadata
fields. The name of a ResourceGroup object must be a valid DNS subdomain name.A ResourceGroup also needs a
.spec
section.Inputs configuration
The
.spec.inputs
field is optional and specifies a list of input valuesto be used in the resources templates.
Example inputs:
An input value is a key-value pair of strings, where the key is the input name
which can be referenced in the resource templates using the
<< inputs.name >>
syntax.Resources configuration
The
.spec.resources
field is optional and specifies the list of Kubernetes resourceto be generated and reconciled on the cluster.
Example of plain resources without any templating:
Templating resources
The resources can be templated using the
<< inputs.name >>
syntax. The templating engineis based on Go text template. The
<< >>
delimiters are used instead of{{ }}
to avoidconflicts with Helm templating and allow ResourceGroups to be included in Helm charts.
Example of templated resources:
The above example will generate a
Namespace
,ServiceAccount
andRoleBinding
for each tenantwith the specified role.
Templating functions
The templating engine supports slim-sprig functions.
It is recommended to use the
quote
function when templating strings to avoid issues withspecial characters e.g.
<< inputs.version | quote >>
.When templating integers, use the
int
function to convert the string to an integere.g.
<< inputs.replicas | int >>
.When templating booleans, use the
bool
function to convert the string to a booleane.g.
<< inputs.enabled | bool >>
.When using integer or boolean inputs as metadata label values, use the
quote
function to convertthe value to a string e.g.
<< inputs.enabled | quote >>
.When using multi-line strings containing YAML, use the
nindent
function to properly format the stringe.g.:
Resource deduplication
The flux-operator deduplicates resources based on the
apiVersion
,kind
,metadata.name
andmetadata.namespace
fields.This allows defining shared resources that are applied only once, regardless of the number of inputs.
Example of a shared Flux source:
In the above example, the
OCIRepository
resource is created only onceand referred by all
HelmRelease
resources.Common metadata
The
.spec.commonMetadata
field is optional and specifies common metadata to be applied to all resources.It has two optional fields:
labels
: A map used for setting labelson an object. Any existing label will be overridden if it matches with a key in
this map.
annotations
: A map used for setting annotationson an object. Any existing annotation will be overridden if it matches with a key
in this map.
Example common metadata:
In the above example, all resources generated by the ResourceGroup
will not be pruned by the garbage collection process as the
fluxcd.controlplane.io/prune
annotation is set to
disabled
.Dependency management
.spec.dependsOn
is an optional list used to refer to Kubernetesobjects that the ResourceGroup depends on. If specified, then the ResourceGroup
is reconciled after the referred objects exist in the cluster.
A dependency is a reference to a Kubernetes object with the following fields:
apiVersion
: The API version of the referred object (required).kind
: The kind of the referred object (required).name
: The name of the referred object (required).namespace
: The namespace of the referred object (optional).ready
: A boolean indicating if the referred object must have theReady
status condition set toTrue
(optional, default isfalse
).Example of conditional reconciliation based on the existence of CustomResourceDefinitions
and the readiness of a ResourceGroup:
Note that is recommended to define dependencies on CustomResourceDefinitions if the ResourceGroup
deploys Flux HelmReleases which contain custom resources.
When the dependencies are not met, the flux-operator will reevaluate the requirements
every five seconds and reconcile the ResourceGroup when the dependencies are satisfied.
Failed dependencies are reported in the ResourceGroup
Ready
status condition,in log messages and Kubernetes events.
Reconciliation configuration
The reconciliation of behaviour of a ResourceGroup can be configured using the following annotations:
fluxcd.controlplane.io/reconcile
: Enable or disable the reconciliation loop. Default isenabled
, set todisabled
to pause the reconciliation.fluxcd.controlplane.io/reconcileEvery
: Set the reconciliation interval used for drift detection and correction. Default is1h
.fluxcd.controlplane.io/reconcileTimeout
: Set the reconciliation timeout including health checks. Default is5m
.Health check configuration
The
.spec.wait
field is optional and instructs the flux-operator to performa health check on all applied resources and waits for them to become ready. The health
check is enabled by default and can be disabled by setting the
.spec.wait
field tofalse
.The health check is performed for the following resources types:
PersistentVolumeClaim, Service, Ingress, CustomResourceDefinition.
By default, the wait timeout is
5m
and can be changed with thefluxcd.controlplane.io/reconcileTimeout
annotation, set on the ResourceGroup object.Role-based access control
The
.spec.serviceAccountName
field is optional and specifies the name of theKubernetes ServiceAccount used by the flux-operator to reconcile the ResourceGroup.
The ServiceAccount must exist in the same namespace as the ResourceGroup
and must have the necessary permissions to create, update and delete
the resources defined in the ResourceGroup.
On multi-tenant clusters, it is recommended to use a dedicated ServiceAccount per tenant namespace
with the minimum required permissions. To enforce a ServiceAccount for all ResourceGroups,
the
--default-service-account=flux-operator
flag can be set in the flux-operator container arguments.With this flag set, only the ResourceGroups created in the same namespace as the flux-operator
will run with cluster-admin permissions.