Generate rule on pod creation triggers twice

[marevers]

Kyverno Version

1.9

Kubernetes Version

1.29

Kubernetes Platform

AKS

Description

I've got the following Kyverno policy.

apiVersion: kyverno.io/v1
kind: Policy
metadata:
  name: create-user
  namespace: monitoring-package
spec:
  mutateExistingOnPolicyUpdate: false
  rules:
  - name: create-user
    match:
      any:
      - resources:
          kinds:
          - Pod
          selector:
            matchLabels:
              app.kubernetes.io/name: grafana
    preconditions:
      all:
      - key: "{{request.operation}}"
        operator: Equals
        value: CREATE
    generate:
      apiVersion: batch/v1
      kind: Job
      name: "create-user-{{request.object.metadata.resourceVersion}}"
      namespace: monitoring-package
      data:
        kind: Job
        spec:
          template:
            ...

The policy, in itself, works. It creates a Job that creates a user in Grafana when Grafana restarts. However, when I delete Grafana pod and a new one gets created, the policy triggers twice and two jobs are generated. In the Kyverno logs, it appears as follows:

I0528 13:59:26.880366       1 generate.go:100] background "msg"="start processing UR" "apiVersion"="v1" "kind"="Pod" "name"="prometheus-grafana-fb5f8779c-gt5hr" "namespace"="monitoring-package" "policy"="monitoring-package/create-user" "resourceVersion"="297591842" "ur"="ur-d9v7f"
I0528 13:59:28.023777       1 generate.go:523] background "msg"="created generate target resource" "apiVersion"="v1" "genAPIVersion"="batch/v1" "genKind"="Job" "genName"="create-user-297591846" "genNamespace"="monitoring-package" "kind"="Pod" "name"="prometheus-grafana-fb5f8779c-gt5hr" "namespace"="monitoring-package" "policy"="monitoring-package/create-user"
I0528 13:59:28.045630       1 generate.go:100] background "msg"="start processing UR" "apiVersion"="v1" "kind"="Pod" "name"="prometheus-grafana-fb5f8779c-gt5hr" "namespace"="monitoring-package" "policy"="monitoring-package/create-user" "resourceVersion"="297591848" "ur"="ur-d9v7f"
I0528 13:59:29.151266       1 generate.go:523] background "msg"="created generate target resource" "apiVersion"="v1" "genAPIVersion"="batch/v1" "genKind"="Job" "genName"="create-user-297591886" "genNamespace"="monitoring-package" "kind"="Pod" "name"="prometheus-grafana-fb5f8779c-gt5hr" "namespace"="monitoring-package" "policy"="monitoring-package/create-user"

So indeed we have two URs, with two resource versions. I would guess somehow there are two CREATE requests for the same pod because of that.

Is there something that I can add to the precondition in order to prevent that?

Steps to reproduce

1. Use a generate policy like the above.
2. For testing adjust the label matcher to match a pod you have to test e.g.

spec:
  rules:
    - name: generate-on-pod-creation
      match:
        all:
          - resources:
              kinds:
                - Pod
              selector:
                matchLabels:
                  foo: bar

3. Delete the pod in question.
4. See that the generate rule is triggered twice.

Expected behavior

The generate rule should only trigger once instead of twice.

Screenshots

No response

Kyverno logs

No response

Slack discussion

No response

Troubleshooting

• I have read and followed the documentation AND the troubleshooting guide.
• I have searched other issues in this repository and mine is not recorded.

[chipzoller]

You could use a context variable to do an API call, but I'm not 100% certain that works all the time in the 1.9 version.

Moving to discussion.

[marevers]

Hello @chipzoller , thank you for your reply. In my precondition/match I am already triggering on the CREATE API call on the one pod. I have looked at this documentation but so far I do not really know how that could help avoiding the double generation.

As noted currently I am using 1.9 but an upgrade is possible if that is required for this to work.

[chipzoller]

https://kyverno.io/docs/writing-policies/external-data-sources/#variables-from-kubernetes-api-server-calls

[chipzoller]

Does this resolve your question?

[marevers]

Thank you for the pointer to the correct docs. However, I am not sure in what way this could prevent the double job creation. Are you suggesting there should be a condition that gets current running jobs from Kubernetes to see if a running job matching a name already exists?

[chipzoller]

Yes

[marevers]

Okay, I will give that at try.