Skip to content

Support controlling SparkApplication labels mutation to spark pods #2668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Support controlling SparkApplication labels mutation to spark pods #2668

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions api/v1beta2/sparkapplication_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -525,6 +525,38 @@ type SparkPodSpec struct {
// ShareProcessNamespace settings for the pod, following the Kubernetes specifications.
// +optional
ShareProcessNamespace *bool `json:"shareProcessNamespace,omitempty"`
// SparkApplicationLabelsMutation defines how to mutate the labels of the SparkApplication to the Spark pod.
// If not specified, any labels in SparkApplication are mutated to the Spark pod for backward compatibility.
// +optional
SparkApplicationLabelsMutation *SparkApplicationLabelsMutationSpec `json:"sparkApplicationLabelsMutation,omitempty"`
}

// SparkApplicationLabelsMutationSpec defines how to mutate the labels of the SparkApplication to the Spark pod.
type SparkApplicationLabelsMutationSpec struct {
// LabelKeyMatches is a list of match conditions for label keys in SparkApplication to be mutated to the Spark pod or not.
// If multiple conditions are specified, conditions are OR-ed.
// If empty list specified, any labels will not match the condition. i.e., no labels are mutated.
// +optional
// +kubebuilder:validation:MaxItems=32
LabelKeyMatches []MutatingLabelKeyMatchCondition `json:"labelKeyMatches,omitempty"`
}

// MutatingLabelKeyMatchCondition defines a condition for how to match a label key
type MutatingLabelKeyMatchCondition struct {
// Fixed is a fixed string to be matched against a label key.
// Fixed and Regex are mutually exclusive, and at least one of them must be specified.
// +optional
// +kubebuilder:validation:MinLength=1
Fixed *string `json:"fixed,omitempty"`

// Regex is a condition matched to the regex pattern of a label key.
// Fixed and Regex are mutually exclusive, and at least one of them must be specified.
// +optional
// +kubebuilder:validation:MinLength=1
Regex *string `json:"regex,omitempty"`

// Invert indicates whether to invert the match result.
Invert bool `json:"invert,omitempty"`
}

// DriverSpec is specification of the driver.
Expand Down
52 changes: 52 additions & 0 deletions api/v1beta2/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

68 changes: 68 additions & 0 deletions charts/spark-operator-chart/crds/sparkoperator.k8s.io_scheduledsparkapplications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5079,6 +5079,40 @@ spec:
- name
type: object
type: array
sparkApplicationLabelsMutation:
description: |-
SparkApplicationLabelsMutation defines how to mutate the labels of the SparkApplication to the Spark pod.
If not specified, any labels in SparkApplication are mutated to the Spark pod for backward compatibility.
properties:
labelKeyMatches:
description: |-
LabelKeyMatches is a list of match conditions for label keys in SparkApplication to be mutated to the Spark pod or not.
If multiple conditions are specified, conditions are OR-ed.
If empty list specified, any labels will not match the condition. i.e., no labels are mutated.
items:
description: MutatingLabelKeyMatchCondition defines
a condition for how to match a label key
properties:
fixed:
description: |-
Fixed is a fixed string to be matched against a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
invert:
description: Invert indicates whether to invert
the match result.
type: boolean
regex:
description: |-
Regex is a condition matched to the regex pattern of a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
type: object
maxItems: 32
type: array
type: object
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Expand Down Expand Up @@ -10192,6 +10226,40 @@ spec:
- name
type: object
type: array
sparkApplicationLabelsMutation:
description: |-
SparkApplicationLabelsMutation defines how to mutate the labels of the SparkApplication to the Spark pod.
If not specified, any labels in SparkApplication are mutated to the Spark pod for backward compatibility.
properties:
labelKeyMatches:
description: |-
LabelKeyMatches is a list of match conditions for label keys in SparkApplication to be mutated to the Spark pod or not.
If multiple conditions are specified, conditions are OR-ed.
If empty list specified, any labels will not match the condition. i.e., no labels are mutated.
items:
description: MutatingLabelKeyMatchCondition defines
a condition for how to match a label key
properties:
fixed:
description: |-
Fixed is a fixed string to be matched against a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
invert:
description: Invert indicates whether to invert
the match result.
type: boolean
regex:
description: |-
Regex is a condition matched to the regex pattern of a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
type: object
maxItems: 32
type: array
type: object
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Expand Down
68 changes: 68 additions & 0 deletions charts/spark-operator-chart/crds/sparkoperator.k8s.io_sparkapplications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5024,6 +5024,40 @@ spec:
- name
type: object
type: array
sparkApplicationLabelsMutation:
description: |-
SparkApplicationLabelsMutation defines how to mutate the labels of the SparkApplication to the Spark pod.
If not specified, any labels in SparkApplication are mutated to the Spark pod for backward compatibility.
properties:
labelKeyMatches:
description: |-
LabelKeyMatches is a list of match conditions for label keys in SparkApplication to be mutated to the Spark pod or not.
If multiple conditions are specified, conditions are OR-ed.
If empty list specified, any labels will not match the condition. i.e., no labels are mutated.
items:
description: MutatingLabelKeyMatchCondition defines a condition
for how to match a label key
properties:
fixed:
description: |-
Fixed is a fixed string to be matched against a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
invert:
description: Invert indicates whether to invert the
match result.
type: boolean
regex:
description: |-
Regex is a condition matched to the regex pattern of a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
type: object
maxItems: 32
type: array
type: object
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Expand Down Expand Up @@ -10113,6 +10147,40 @@ spec:
- name
type: object
type: array
sparkApplicationLabelsMutation:
description: |-
SparkApplicationLabelsMutation defines how to mutate the labels of the SparkApplication to the Spark pod.
If not specified, any labels in SparkApplication are mutated to the Spark pod for backward compatibility.
properties:
labelKeyMatches:
description: |-
LabelKeyMatches is a list of match conditions for label keys in SparkApplication to be mutated to the Spark pod or not.
If multiple conditions are specified, conditions are OR-ed.
If empty list specified, any labels will not match the condition. i.e., no labels are mutated.
items:
description: MutatingLabelKeyMatchCondition defines a condition
for how to match a label key
properties:
fixed:
description: |-
Fixed is a fixed string to be matched against a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
invert:
description: Invert indicates whether to invert the
match result.
type: boolean
regex:
description: |-
Regex is a condition matched to the regex pattern of a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
type: object
maxItems: 32
type: array
type: object
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Expand Down
68 changes: 68 additions & 0 deletions config/crd/bases/sparkoperator.k8s.io_scheduledsparkapplications.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5079,6 +5079,40 @@ spec:
- name
type: object
type: array
sparkApplicationLabelsMutation:
description: |-
SparkApplicationLabelsMutation defines how to mutate the labels of the SparkApplication to the Spark pod.
If not specified, any labels in SparkApplication are mutated to the Spark pod for backward compatibility.
properties:
labelKeyMatches:
description: |-
LabelKeyMatches is a list of match conditions for label keys in SparkApplication to be mutated to the Spark pod or not.
If multiple conditions are specified, conditions are OR-ed.
If empty list specified, any labels will not match the condition. i.e., no labels are mutated.
items:
description: MutatingLabelKeyMatchCondition defines
a condition for how to match a label key
properties:
fixed:
description: |-
Fixed is a fixed string to be matched against a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
invert:
description: Invert indicates whether to invert
the match result.
type: boolean
regex:
description: |-
Regex is a condition matched to the regex pattern of a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
type: object
maxItems: 32
type: array
type: object
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Expand Down Expand Up @@ -10192,6 +10226,40 @@ spec:
- name
type: object
type: array
sparkApplicationLabelsMutation:
description: |-
SparkApplicationLabelsMutation defines how to mutate the labels of the SparkApplication to the Spark pod.
If not specified, any labels in SparkApplication are mutated to the Spark pod for backward compatibility.
properties:
labelKeyMatches:
description: |-
LabelKeyMatches is a list of match conditions for label keys in SparkApplication to be mutated to the Spark pod or not.
If multiple conditions are specified, conditions are OR-ed.
If empty list specified, any labels will not match the condition. i.e., no labels are mutated.
items:
description: MutatingLabelKeyMatchCondition defines
a condition for how to match a label key
properties:
fixed:
description: |-
Fixed is a fixed string to be matched against a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
invert:
description: Invert indicates whether to invert
the match result.
type: boolean
regex:
description: |-
Regex is a condition matched to the regex pattern of a label key.
Fixed and Regex are mutually exclusive, and at least one of them must be specified.
minLength: 1
type: string
type: object
maxItems: 32
type: array
type: object
template:
description: |-
Template is a pod template that can be used to define the driver or executor pod configurations that Spark configurations do not support.
Expand Down
Loading