Skip to content

Conversation

@everpeace
Copy link
Contributor

@everpeace everpeace commented Oct 15, 2025

Purpose of this PR

Proposed changes:

  • Introduce SparkApplication.Spec.{Driver,Executor}.SparkApplicationLabelMutation field which provides users to control which labels in SparkApplication are mutated to the Spark pods
kind: SparkApplication
metadata:
  name: spark-app
  labels:
    keyA: valueA
    keyB: valueB
spec:
  driver:
    sparkApplicationLabelsMutation:
      labelKeysMatches:
      - fixed: keyA
  executor:
    sparkApplicationLabelsMutation:
      labelKeysMatches:
      - regex: '^.*'
----
#
# Above SparkApplication generates Spark driver/executor pod like this:
#
kind: Pod
metadata:
  labels:
    # SparkApplication's label
    # keyB is excluded because "keyB" doesn't match "fixed: keyA"
    keyA: valueA

    # managed labels should not be controlled by users
    spark-role: driver
    sparkoperator.k8s.io/launched-by-spark-operator: "true"
    sparkoperator.k8s.io/app-name: spark-app

---
kind: Pod
metadata:
  labels:
    # SparkApplication's label
    # all the labels are mutated because any label key matches "regex: '^.*'"
    keyA: valueA
    keyB: valueB

    # managed labels should not be controlled by users
    spark-role: executor
    sparkoperator.k8s.io/launched-by-spark-operator: "true"
    sparkoperator.k8s.io/app-name: spark-app

Change Category

  • Bugfix (non-breaking change which fixes an issue)
  • Feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that could affect existing functionality)
  • Documentation update

Rationale

Fixes #2669

Checklist

  • I have conducted a self-review of my own code.
  • I have updated documentation accordingly.
  • I have added tests that prove my changes are effective or that my feature works.
  • Existing unit tests pass locally with my changes.

Additional Notes

@google-oss-prow
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mwielgus for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…belMutation" field to control which labels in SparkApplication are mutated to the Spark pods.

Signed-off-by: Shingo Omura <everpeace@gmail.com>
@ChenYi015
Copy link
Member

It is a bit compliated for users to use a field like sparkApplicationLabelsMutation when integrating with Kueue. Let us use the controller-reference way first and see wether the community has a request for this feature.
/hold for review

@everpeace
Copy link
Contributor Author

ref: #2669 (comment)

As discussed in the community meeting on 2025-10-17, we plan to introduce a simpler solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

FeatureRequest: Control which SparkApplication labels are mutated spark pods

2 participants