Skip to content

Commit

Permalink
feat: Support node taints per nodepool and control plane (#909)
Browse files Browse the repository at this point in the history 
**What problem does this PR solve?**:

**Which issue(s) this PR fixes**:
Fixes https://jira.nutanix.com/browse/NCN-102652

**How Has This Been Tested?**:
<!--
Please describe the tests that you ran to verify your changes.
Provide output from the tests and any manual steps needed to replicate
the tests.
-->

**Special notes for your reviewer**:
<!--
Use this to provide any additional information to the reviewers.
This may include:
- Best way to review the PR.
- Where the author wants the most review attention on.
- etc.
-->
  • Loading branch information
@jimmidyson
jimmidyson authored Sep 25, 2024
1 parent 3106b2f commit a85f32f
Show file tree
Hide file tree
Showing 22 changed files with 945 additions and 29 deletions.
28 changes: 28 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_awsclusterconfigs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -343,6 +343,34 @@ spec:
default: m5.xlarge
type: string
type: object
taints:
description: Taints specifies the taints the Node API object should be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
encryptionAtRest:
description: |-
Expand Down
29 changes: 29 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_awsworkernodeconfigs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,35 @@ spec:
description: The AWS instance type to use for the cluster Machines.
type: string
type: object
taints:
description: Taints specifies the taints the Node API object should
be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
type: object
served: true
Expand Down
28 changes: 28 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_dockerclusterconfigs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,34 @@ spec:
pattern: ^((?:[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*|\[(?:[a-fA-F0-9:]+)\])(:[0-9]+)?/)?[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*(/[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*)*(:[\w][\w.-]{0,127})?(@[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][0-9A-Fa-f]{32,})?$
type: string
type: object
taints:
description: Taints specifies the taints the Node API object should be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
docker:
type: object
Expand Down
29 changes: 29 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_dockernodeconfigs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,35 @@ spec:
pattern: ^((?:[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*|\[(?:[a-fA-F0-9:]+)\])(:[0-9]+)?/)?[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*(/[a-z0-9]+((?:[._]|__|[-]+)[a-z0-9]+)*)*(:[\w][\w.-]{0,127})?(@[A-Za-z][A-Za-z0-9]*(?:[-_+.][A-Za-z][A-Za-z0-9]*)*[:][0-9A-Fa-f]{32,})?$
type: string
type: object
taints:
description: Taints specifies the taints the Node API object should
be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
type: object
served: true
Expand Down
28 changes: 28 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_nutanixclusterconfigs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,34 @@ spec:
required:
- machineDetails
type: object
taints:
description: Taints specifies the taints the Node API object should be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
encryptionAtRest:
description: |-
Expand Down
28 changes: 28 additions & 0 deletions api/v1alpha1/crds/caren.nutanix.com_nutanixnodeconfigs.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,34 @@ spec:
required:
- machineDetails
type: object
taints:
description: Taints specifies the taints the Node API object should be registered with.
items:
description: |-
The node this Taint is attached to has the "effect" on
any pod that does not tolerate the Taint.
properties:
effect:
default: NoSchedule
description: |-
The effect of the taint on pods that do not tolerate the taint.
Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
enum:
- NoSchedule
- PreferNoSchedule
- NoExecute
type: string
key:
description: The taint key to be applied to a node.
type: string
value:
description: The taint value corresponding to the taint key.
type: string
required:
- effect
- key
type: object
type: array
type: object
type: object
served: true
Expand Down
52 changes: 52 additions & 0 deletions api/v1alpha1/nodeconfig_types.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ func (s AWSWorkerNodeConfig) VariableSchema() clusterv1.VariableSchema { //nolin
type AWSWorkerNodeConfigSpec struct {
// +kubebuilder:validation:Optional
AWS *AWSWorkerNodeSpec `json:"aws,omitempty"`

GenericNodeSpec `json:",inline"`
}

// AWSControlPlaneConfigSpec defines the desired state of AWSNodeConfig.
Expand All @@ -58,6 +60,8 @@ type AWSWorkerNodeConfigSpec struct {
type AWSControlPlaneNodeConfigSpec struct {
// +kubebuilder:validation:Optional
AWS *AWSControlPlaneNodeSpec `json:"aws,omitempty"`

GenericNodeSpec `json:",inline"`
}

// +kubebuilder:object:root=true
Expand All @@ -79,6 +83,8 @@ func (s DockerNodeConfig) VariableSchema() clusterv1.VariableSchema { //nolint:g
type DockerNodeConfigSpec struct {
// +kubebuilder:validation:Optional
Docker *DockerNodeSpec `json:"docker,omitempty"`

GenericNodeSpec `json:",inline"`
}

// +kubebuilder:object:root=true
Expand All @@ -100,8 +106,54 @@ func (s NutanixNodeConfig) VariableSchema() clusterv1.VariableSchema { //nolint:
type NutanixNodeConfigSpec struct {
// +kubebuilder:validation:Optional
Nutanix *NutanixNodeSpec `json:"nutanix,omitempty"`

GenericNodeSpec `json:",inline"`
}

type GenericNodeSpec struct {
// Taints specifies the taints the Node API object should be registered with.
// +kubebuilder:validation:Optional
Taints []Taint `json:"taints,omitempty"`
}

// The node this Taint is attached to has the "effect" on
// any pod that does not tolerate the Taint.
type Taint struct {
// The taint key to be applied to a node.
// +kubebuilder:validation:Required
Key string `json:"key"`

// The taint value corresponding to the taint key.
// +kubebuilder:validation:Optional
Value string `json:"value,omitempty"`

// The effect of the taint on pods that do not tolerate the taint.
// Valid effects are NoSchedule, PreferNoSchedule and NoExecute.
// +kubebuilder:validation:Required
// +kubebuilder:default=NoSchedule
// +kubebuilder:validation:Enum:=NoSchedule;PreferNoSchedule;NoExecute
Effect TaintEffect `json:"effect"`
}

type TaintEffect string

const (
// Do not allow new pods to schedule onto the node unless they tolerate the taint,
// but allow all pods submitted to Kubelet without going through the scheduler
// to start, and allow all already-running pods to continue running.
// Enforced by the scheduler.
TaintEffectNoSchedule TaintEffect = "NoSchedule"

// Like TaintEffectNoSchedule, but the scheduler tries not to schedule
// new pods onto the node, rather than prohibiting new pods from scheduling
// onto the node entirely. Enforced by the scheduler.
TaintEffectPreferNoSchedule TaintEffect = "PreferNoSchedule"

// Evict any already-running pods that do not tolerate the taint.
// Currently enforced by NodeController.
TaintEffectNoExecute TaintEffect = "NoExecute"
)

//nolint:gochecknoinits // Idiomatic to use init functions to register APIs with scheme.
func init() {
SchemeBuilder.Register(&AWSWorkerNodeConfig{}, &DockerNodeConfig{}, &NutanixNodeConfig{})
Expand Down
39 changes: 39 additions & 0 deletions api/v1alpha1/zz_generated.deepcopy.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions docs/content/addons/serviceloadbalancer.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
+++
title = "Service LoadBalancer"
icon = "fa-solid fa-arrows-turn-to-dots"
+++

When an application running in a cluster needs to be exposed outside of the cluster, one option is
Expand Down
Loading

0 comments on commit a85f32f

Please sign in to comment.