docs: explain KEDA/no KEDA scaling more #6406
Conversation
Signed-off-by: Calum Murray <cmurray@redhat.com>
✅ Deploy Preview for knative ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: Cali0707 The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
knative-prow
bot
added
the
approved
|
/cc @matzew |
knative-prow
bot
added
the
size/L
|
/cherry-pick release-1.19 |
|
@Cali0707: once the present PR merges, I will cherry-pick it on top of In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/cherry-pick release-1.18 |
|
@Cali0707: once the present PR merges, I will cherry-pick it on top of In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
| name: config-kafka-autoscaler | ||
| namespace: knative-eventing | ||
| data: | ||
| # What autoscaling class should be used. Can either be keda.autoscaling.knative.dev or dispabled. |
There was a problem hiding this comment.
| # What autoscaling class should be used. Can either be keda.autoscaling.knative.dev or dispabled. | |
| # What autoscaling class should be used. Can either be keda.autoscaling.knative.dev or deactivated. |
There was a problem hiding this comment.
(typo + inclusive language)
There was a problem hiding this comment.
disabled is the key we are using in the code afaik - I just took it from the repo
| class: keda.autoscaling.knative.dev | ||
| # The period in seconds the autoscaler waits until it scales down | ||
| cooldown-period: "30" | ||
| # The lag that is used for scaling (1<->N) |
There was a problem hiding this comment.
This should probably have a unit?
| cooldown-period: "30" | ||
| # The lag that is used for scaling (1<->N) | ||
| lag-threshold: "100" | ||
| # The maximum number of replicas to scale up to |
There was a problem hiding this comment.
This says "replicas", but the comment above talks about "Kafka consumers" (which I assume means threads reading from a shared consumer group)?
| KEDA uses different lag thresholds to determine when to scale your Kafka components: | ||
| ### Scaling from 0 to 1 (Activation) | ||
| - **Trigger**: When consumer lag exceeds the `activation-lag-threshold` (default: 0) |
There was a problem hiding this comment.
activation-lag-threshold doesn't seem to be in the above ConfigMap. Is this another key?
| dispatcher_pods = ceil(total_consumers_for_resource_type / POD_CAPACITY) | ||
| ``` | ||
|
|
||
| Where `POD_CAPACITY` defaults to 20, meaning each dispatcher pod can handle up to 20 Kafka consumers. |
There was a problem hiding this comment.
In the example above, max-scale was set to 50, which seems like it would mean 3 dispatcher pods (the same as if the value were 60). Is there a practical difference between 41 and 60 here given the default value of POD_CAPACITY?
I ask because we had a few samples for Serving autoscaling where the end result was that a single request coming in for a scaled-down pod would sometimes cause 2 pods to start, because the pending request + the "spare capacity" factor ended up crossing the one-pod capacity threshold (i.e. the "spare capacity" number was exactly equal to "pod capacity", so ceil(200 + 1 / 200) => 2).
|
|
||
| ```bash | ||
| # List all consumer groups in your workload namespace | ||
| kubectl get consumergroups -n <your-namespace> |
There was a problem hiding this comment.
can we use the FQN of the resource, @Cali0707 ?
| ``` | ||
|
|
||
| For example: | ||
| - 2 Triggers each with `min-scale: "40"` = 80 total consumers |
There was a problem hiding this comment.
Not rendered as a list. You probably need a blank line between the bulletpoint and the "For example:" header.
| ## Configure Autoscaling for a Resource | ||
|
|
||
| If you want to customize how KEDA scales a KafkaSource, trigger, or subscription you can set annotations on the resource: | ||
| If you want to customize how KEDA scales a KafkaSource, trigger, or subscription you can set annotations on the resource. |
There was a problem hiding this comment.
nit: here trigger, or subscription have a small first letter, but here a capital first letter :)
Resolves #6402
This PR adds much more explanation for how the EKB scaling works with and without KEDA