Skip to content

Commit

Permalink
feat: default resources for Operator and Integration Pods
Browse files Browse the repository at this point in the history 
Closes #5601
  • Loading branch information
@squakez
squakez committed Jun 28, 2024
1 parent 39bfbb7 commit 2fb8632
Show file tree
Hide file tree
Showing 19 changed files with 190 additions and 93 deletions.
34 changes: 25 additions & 9 deletions docs/modules/ROOT/pages/installation/advanced/resources.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,25 +34,41 @@ The option accept a value in the following format `Key[=Value]:Effect[:Seconds]`
While installing the Camel K operator, you can also specify the resources requests and limits to assign to the operator `Pod` with `--operator-resources` option. The option will expect the configuration as required by https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/[Kubernetes Resource Management].

```
kamel install --operator-resources requests.cpu=256m --operator-resources limits.cpu=500m --operator-resources limits.memory=512Mi ...
kamel install --operator-resources limits.cpu=8 --operator-resources limits.memory=32Gi ...
```

The value expected by the option are in the form `requestType.requestResource=value` where `requestType` must be either `requests` or `limits`, `requestResource` must be either `cpu` or `memory` and `value` expressed in the numeric value as expected by the resource. You can specify more than one `operator-resources`.

NOTE: if you specify a limit, but does not specify a request, Kubernetes automatically assigns a request that matches the limit.

=== Suggested configuration
=== Default Operator Pod configuration

The main contributor to compute resources consumption is likely to be the Maven builds that are performed in the operator `Pod` (assuming you use the default build strategy). So the resource requirements should be defined accordingly. The following requirements are sensible defaults that should work in most cases:
The main Camel K Operator Pod contributor resources consumption is likely to be the number of parallel builds that are performed in the operator `Pod`. So the resource requirements should be defined accordingly. The following requirements are sensible defaults that should work in most cases:

```
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: 500m
memory: 512Mi
memory: "4Gi"
cpu: "1"
limits:
memory: "16Gi"
cpu: "4"
```

Note that if you plan to perform **native builds**, then the memory requirements must be increased significantly. Also, the CPU requirements are rather "soft", in the sense that it won't break the operator, but it'll perform slower in general.
Note that if you plan to perform **native builds**, then the memory requirements may be increased significantly. Also, the CPU requirements are rather "soft", in the sense that it won't break the operator, but it'll perform slower in general.

=== Default Integration Pod configuration

The resource set on the container here is highly dependant on what your application is doing. You can control this behavior by setting opportunely the resources on the Integration via container trait.

Be aware that the default are actually the following:

```
resources:
requests:
memory: "256Mi"
cpu: "125m"
limits:
memory: "1Gi"
cpu: "500m"
```
8 changes: 4 additions & 4 deletions docs/modules/ROOT/partials/apis/camel-k-crds.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6473,28 +6473,28 @@ string
|
The minimum amount of CPU required.
The minimum amount of CPU required (default 125 millicores).
|`requestMemory` +
string
|
The minimum amount of memory required.
The minimum amount of memory required (default 128 Mi).
|`limitCPU` +
string
|
The maximum amount of CPU required.
The maximum amount of CPU to be provided (default 500 millicores).
|`limitMemory` +
string
|
The maximum amount of memory required.
The maximum amount of memory to be provided (default 512 Mi).
|`expose` +
bool
Expand Down
8 changes: 4 additions & 4 deletions docs/modules/traits/pages/container.adoc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,19 +37,19 @@ The following configuration options are available:

| container.request-cpu
| string
| The minimum amount of CPU required.
| The minimum amount of CPU required (default 125 millicores).

| container.request-memory
| string
| The minimum amount of memory required.
| The minimum amount of memory required (default 128 Mi).

| container.limit-cpu
| string
| The maximum amount of CPU required.
| The maximum amount of CPU to be provided (default 500 millicores).

| container.limit-memory
| string
| The maximum amount of memory required.
| The maximum amount of memory to be provided (default 512 Mi).

| container.expose
| bool
Expand Down
16 changes: 2 additions & 14 deletions e2e/advanced/integration_profile_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ func TestIntegrationProfile(t *testing.T) {
integrationProfile.SetOperatorID(operatorID)
integrationProfile.Spec.Traits.Container = &traitv1.ContainerTrait{
Name: "ck-integration-global",
LimitCPU: "0.2",
LimitCPU: "0.3",
}

g.Expect(CreateIntegrationProfile(t, ctx, &integrationProfile)).To(Succeed())
Expand All @@ -58,7 +58,7 @@ func TestIntegrationProfile(t *testing.T) {
integrationProfile := v1.NewIntegrationProfile(ns1, "ipr-local")
integrationProfile.SetOperatorID(operatorID)
integrationProfile.Spec.Traits.Container = &traitv1.ContainerTrait{
LimitCPU: "0.1",
LimitCPU: "0.2",
}
g.Expect(CreateIntegrationProfile(t, ctx, &integrationProfile)).To(Succeed())
g.Eventually(SelectedIntegrationProfilePhase(t, ctx, ns1, "ipr-local"), TestTimeoutMedium).Should(Equal(v1.IntegrationProfilePhaseReady))
Expand Down Expand Up @@ -106,18 +106,6 @@ func TestIntegrationProfile(t *testing.T) {
g.Expect(Kamel(t, ctx, "delete", "limited", "-n", ns1).Execute()).To(Succeed())
})

t.Run("Run integration without integration profile", func(t *testing.T) {
g.Expect(KamelRunWithID(t, ctx, operatorID, ns1, "--name", "normal", "files/yaml.yaml").Execute()).To(Succeed())
g.Eventually(IntegrationPod(t, ctx, ns1, "normal"), TestTimeoutShort).Should(Not(BeNil()))
g.Eventually(IntegrationPodHas(t, ctx, ns1, "normal", func(pod *corev1.Pod) bool {
if len(pod.Spec.Containers) != 1 {
return false
}
cpuLimits := pod.Spec.Containers[0].Resources.Limits.Cpu()
return cpuLimits == nil || cpuLimits.IsZero()
}), TestTimeoutShort).Should(BeTrue())
})

// Clean up
g.Expect(Kamel(t, ctx, "delete", "--all", "-n", ns1).Execute()).To(Succeed())
})
Expand Down
2 changes: 1 addition & 1 deletion e2e/advanced/local_platform_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ func TestLocalPlatform(t *testing.T) {
pl.SetOperatorID(operatorID)

pl.Spec.Traits.Container = &traitv1.ContainerTrait{
LimitCPU: "0.1",
LimitCPU: "0.2",
}
})).To(Succeed())

Expand Down
24 changes: 16 additions & 8 deletions helm/camel-k/crds/crd-integration-platform.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -734,10 +734,12 @@ spec:
- IfNotPresent
type: string
limitCPU:
description: The maximum amount of CPU required.
description: The maximum amount of CPU to be provided (default
500 millicores).
type: string
limitMemory:
description: The maximum amount of memory required.
description: The maximum amount of memory to be provided (default
512 Mi).
type: string
name:
description: The main container name. It's named `integration`
Expand All @@ -753,10 +755,12 @@ spec:
the `expose` parameter is true.
type: string
requestCPU:
description: The minimum amount of CPU required.
description: The minimum amount of CPU required (default 125
millicores).
type: string
requestMemory:
description: The minimum amount of memory required.
description: The minimum amount of memory required (default
128 Mi).
type: string
runAsNonRoot:
description: Security Context RunAsNonRoot configuration (default
Expand Down Expand Up @@ -2766,10 +2770,12 @@ spec:
- IfNotPresent
type: string
limitCPU:
description: The maximum amount of CPU required.
description: The maximum amount of CPU to be provided (default
500 millicores).
type: string
limitMemory:
description: The maximum amount of memory required.
description: The maximum amount of memory to be provided (default
512 Mi).
type: string
name:
description: The main container name. It's named `integration`
Expand All @@ -2785,10 +2791,12 @@ spec:
the `expose` parameter is true.
type: string
requestCPU:
description: The minimum amount of CPU required.
description: The minimum amount of CPU required (default 125
millicores).
type: string
requestMemory:
description: The minimum amount of memory required.
description: The minimum amount of memory required (default
128 Mi).
type: string
runAsNonRoot:
description: Security Context RunAsNonRoot configuration (default
Expand Down
24 changes: 16 additions & 8 deletions helm/camel-k/crds/crd-integration-profile.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -611,10 +611,12 @@ spec:
- IfNotPresent
type: string
limitCPU:
description: The maximum amount of CPU required.
description: The maximum amount of CPU to be provided (default
500 millicores).
type: string
limitMemory:
description: The maximum amount of memory required.
description: The maximum amount of memory to be provided (default
512 Mi).
type: string
name:
description: The main container name. It's named `integration`
Expand All @@ -630,10 +632,12 @@ spec:
the `expose` parameter is true.
type: string
requestCPU:
description: The minimum amount of CPU required.
description: The minimum amount of CPU required (default 125
millicores).
type: string
requestMemory:
description: The minimum amount of memory required.
description: The minimum amount of memory required (default
128 Mi).
type: string
runAsNonRoot:
description: Security Context RunAsNonRoot configuration (default
Expand Down Expand Up @@ -2526,10 +2530,12 @@ spec:
- IfNotPresent
type: string
limitCPU:
description: The maximum amount of CPU required.
description: The maximum amount of CPU to be provided (default
500 millicores).
type: string
limitMemory:
description: The maximum amount of memory required.
description: The maximum amount of memory to be provided (default
512 Mi).
type: string
name:
description: The main container name. It's named `integration`
Expand All @@ -2545,10 +2551,12 @@ spec:
the `expose` parameter is true.
type: string
requestCPU:
description: The minimum amount of CPU required.
description: The minimum amount of CPU required (default 125
millicores).
type: string
requestMemory:
description: The minimum amount of memory required.
description: The minimum amount of memory required (default
128 Mi).
type: string
runAsNonRoot:
description: Security Context RunAsNonRoot configuration (default
Expand Down
12 changes: 8 additions & 4 deletions helm/camel-k/crds/crd-integration.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6675,10 +6675,12 @@ spec:
- IfNotPresent
type: string
limitCPU:
description: The maximum amount of CPU required.
description: The maximum amount of CPU to be provided (default
500 millicores).
type: string
limitMemory:
description: The maximum amount of memory required.
description: The maximum amount of memory to be provided (default
512 Mi).
type: string
name:
description: The main container name. It's named `integration`
Expand All @@ -6694,10 +6696,12 @@ spec:
the `expose` parameter is true.
type: string
requestCPU:
description: The minimum amount of CPU required.
description: The minimum amount of CPU required (default 125
millicores).
type: string
requestMemory:
description: The minimum amount of memory required.
description: The minimum amount of memory required (default
128 Mi).
type: string
runAsNonRoot:
description: Security Context RunAsNonRoot configuration (default
Expand Down
12 changes: 8 additions & 4 deletions helm/camel-k/crds/crd-kamelet-binding.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6957,10 +6957,12 @@ spec:
- IfNotPresent
type: string
limitCPU:
description: The maximum amount of CPU required.
description: The maximum amount of CPU to be provided
(default 500 millicores).
type: string
limitMemory:
description: The maximum amount of memory required.
description: The maximum amount of memory to be provided
(default 512 Mi).
type: string
name:
description: The main container name. It's named `integration`
Expand All @@ -6976,10 +6978,12 @@ spec:
only when the `expose` parameter is true.
type: string
requestCPU:
description: The minimum amount of CPU required.
description: The minimum amount of CPU required (default
125 millicores).
type: string
requestMemory:
description: The minimum amount of memory required.
description: The minimum amount of memory required (default
128 Mi).
type: string
runAsNonRoot:
description: Security Context RunAsNonRoot configuration
Expand Down
12 changes: 8 additions & 4 deletions helm/camel-k/crds/crd-pipe.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6955,10 +6955,12 @@ spec:
- IfNotPresent
type: string
limitCPU:
description: The maximum amount of CPU required.
description: The maximum amount of CPU to be provided
(default 500 millicores).
type: string
limitMemory:
description: The maximum amount of memory required.
description: The maximum amount of memory to be provided
(default 512 Mi).
type: string
name:
description: The main container name. It's named `integration`
Expand All @@ -6974,10 +6976,12 @@ spec:
only when the `expose` parameter is true.
type: string
requestCPU:
description: The minimum amount of CPU required.
description: The minimum amount of CPU required (default
125 millicores).
type: string
requestMemory:
description: The minimum amount of memory required.
description: The minimum amount of memory required (default
128 Mi).
type: string
runAsNonRoot:
description: Security Context RunAsNonRoot configuration
Expand Down
8 changes: 4 additions & 4 deletions pkg/apis/camel/v1/trait/container.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,13 @@ type ContainerTrait struct {
PlatformBaseTrait `property:",squash" json:",inline"`
// To automatically enable the trait
Auto *bool `property:"auto" json:"auto,omitempty"`
// The minimum amount of CPU required.
// The minimum amount of CPU required (default 125 millicores).
RequestCPU string `property:"request-cpu" json:"requestCPU,omitempty"`
// The minimum amount of memory required.
// The minimum amount of memory required (default 128 Mi).
RequestMemory string `property:"request-memory" json:"requestMemory,omitempty"`
// The maximum amount of CPU required.
// The maximum amount of CPU to be provided (default 500 millicores).
LimitCPU string `property:"limit-cpu" json:"limitCPU,omitempty"`
// The maximum amount of memory required.
// The maximum amount of memory to be provided (default 512 Mi).
LimitMemory string `property:"limit-memory" json:"limitMemory,omitempty"`
// Can be used to enable/disable exposure via kubernetes Service.
Expose *bool `property:"expose" json:"expose,omitempty"`
Expand Down
Loading

0 comments on commit 2fb8632

Please sign in to comment.