Skip to content

Commit

Permalink
Spread out backfill jobs to keep resource usage more stable
Browse files Browse the repository at this point in the history
  • Loading branch information
bastjan committed Jan 30, 2024
1 parent bdedcff commit 71640ab
Show file tree
Hide file tree
Showing 9 changed files with 558 additions and 558 deletions.
3 changes: 0 additions & 3 deletions class/defaults.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,6 @@ parameters:
repository: appuio/appuio-reporting
tag: v0.2.1

schedules:
backfill: '15 * * * *'

schedules_suspend:
backfill: false

Expand Down
2 changes: 1 addition & 1 deletion component/main.jsonnet
Original file line number Diff line number Diff line change
Expand Up @@ -183,10 +183,10 @@ local backfillCJ = function(rule, product, index)
},
// Keeping infinite jobs is not possible. Keep at least one month worth of jobs.
failedJobsHistoryLimit: 24 * 32,
schedule: '%s * * * *' % (index % 41+10) # spread out the jobs over the hour, from 10 to 50 minutes
},
};


// Define outputs below
{
'00_namespace': kube.Namespace(params.namespace) {
Expand Down
31 changes: 17 additions & 14 deletions docs/modules/ROOT/pages/references/parameters.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,36 +11,38 @@ default:: `${_instance}`
The namespace in which to deploy this component.


== `images`
== `namespaceMetadata`

[horizontal]
type:: dictionary
default:: https://github.com/appuio/component-appuio-reporting/blob/master/class/defaults.yml[See `class/defaults.yml`].
default::
+
[source,yaml]
----
namespaceMetadata:
labels:
openshift.io/cluster-monitoring: 'true'
----

Dictionary containing the container images used by this component.
The namespace metadata to apply to the namespace created by this component.


=== `images.reporting`
== `images`

[horizontal]
type:: dictionary
default:: https://github.com/appuio/component-appuio-reporting/blob/master/class/defaults.yml[See `class/defaults.yml`].

The image used for managing data in the reporting database.

Dictionary containing the container images used by this component.


== `schedules`
=== `images.reporting`

[horizontal]
type:: dictionary
default::
+
[source,yaml]
----
backfill: '15 * * * *'
----

Dictionary managing the schedules of the periodic jobs (implemented as Kubernetes CronJobs).
The image used for managing data in the reporting database.


== `development_mode`

Expand Down Expand Up @@ -239,6 +241,7 @@ The jsonnet parameter takes precedence over the pattern parameter if both are sp

Note that there is no jsonnet variant for the `query_pattern` parameter.

CronJobs are spread across the hour to allow for a more even load on the cluster and Prometheus.


== `network_policies.target_namespaces`
Expand Down
Loading

0 comments on commit 71640ab

Please sign in to comment.