PipelineRun webhook defaulting skips default-service-account when another MutatingWebhookConfiguration intercepts CREATE first

[waveywaves]

Expected Behavior

When a PipelineRun is created without an explicit serviceAccountName, the Tekton webhook should apply the value from config-defaults ConfigMap's default-service-account field (e.g., pipeline), regardless of whether other mutating webhooks run before or after the Tekton webhook.

Actual Behavior

When another MutatingWebhookConfiguration (e.g., tekton-kueue's tekton-kueue-mutating-webhook-configuration) is registered and intercepts PipelineRun CREATE requests before the Tekton webhook, the Tekton webhook's SetDefaults() does not apply the default-service-account value. The PipelineRun ends up with serviceAccountName: "default" instead of the configured value ("pipeline").

The Tekton webhook log shows PatchBytes: null for the PipelineRun CREATE, meaning SetDefaults ran but produced no SA patch.

TaskRun creation is NOT affected — TaskRuns correctly receive the configured default SA even with the same additional webhook present.

Steps to Reproduce the Problem

1. Install OpenShift Pipelines (or Tekton Pipelines) with config-defaults ConfigMap set to default-service-account: pipeline
2. Verify PipelineRun defaulting works:

   kubectl create --dry-run=server -o jsonpath='{.spec.taskRunTemplate.serviceAccountName}' -f - <<'EOF'
   apiVersion: tekton.dev/v1
   kind: PipelineRun
   metadata:
     generateName: test-
   spec:
     pipelineSpec:
       tasks:
       - name: echo
         taskSpec:
           steps:
           - image: busybox
             script: echo hello
   EOF
   # Returns: "pipeline" ✅

3. Install any component that registers a MutatingWebhookConfiguration intercepting PipelineRun CREATE (e.g., tekton-kueue/scheduler)
4. Repeat the same dry-run test:

   # Returns: "default" ❌

5. Delete the additional MutatingWebhookConfiguration:

   kubectl delete mutatingwebhookconfiguration <name>

6. Repeat the dry-run test:

   # Returns: "pipeline" ✅ (works again)

Additional Info

• Kubernetes version: OpenShift 4.20.16
• Tekton Pipeline version: 1.22.0
• The issue is 100% reproducible: MWC present → SA defaults to "default", MWC absent → SA defaults to "pipeline"
• TaskRun defaulting is NOT affected (always gets the correct SA)
• The Tekton webhook has reinvocationPolicy: IfNeeded and does get reinvoked, but returns no SA patch
• The additional webhook (tekton-kueue) does NOT modify serviceAccountName — it only sets spec.status and labels
• Webhook execution order is alphabetical: the additional webhook runs before webhook.pipeline.tekton.dev
• Restarting the Tekton webhook pod does not fix the issue
• This may be related to how controller-runtime serializes typed Go structs back to JSON after mutation, potentially introducing zero-value fields that prevent the Tekton webhook from applying its defaults

[waveywaves]

Related downstream issue: konflux-ci/tekton-kueue#319

The tekton-kueue webhook is the specific mutating webhook that triggers this behavior. The fix needs to be in both places — Tekton's webhook should be resilient to other webhooks, and tekton-kueue should avoid leaking zero-value fields.

[vdemeester]

The PatchBytes: null on the CREATE event is the key clue — it means SetDefaults ran but found nothing to change. Since ServiceAccountName should be "" on a fresh PipelineRun, the only way line 65 of pipelinerun_defaults.go skips the assignment is if defaultSA is also "":

if prs.TaskRunTemplate.ServiceAccountName == "" && defaultSA != "" {

This could happen if config.FromContextOrDefaults(ctx) is returning a Defaults struct where DefaultServiceAccount is empty rather than the ConfigMap-configured value. Worth checking whether the webhook's config store is properly initialized when handling reinvocation requests, or whether the additional webhook's presence somehow affects the context passed to SetDefaults.

The fact that TaskRun defaulting works correctly suggests the issue may be specific to PipelineRun's defaulting path — possibly a difference in how the webhook admission handler constructs the context for PipelineRun vs TaskRun, or a difference in the admission registration (e.g., which resources/operations each webhook configuration matches).

A good debugging step would be to add a temporary log line in PipelineRunSpec.SetDefaults to print cfg.Defaults.DefaultServiceAccount and prs.TaskRunTemplate.ServiceAccountName to confirm which side of the guard is causing the skip.