Add gp3 storageclass #2941
Add gp3 storageclass #2941
Conversation
We add this unconditionally to all clusters for simplification, so we can set storageClass: gp3 for new clusters that come up on AWS without issue. This doesn't change the default, and does not change the storageclass in existing clusters. In addition to using gp3, it also sets reclaimPolicy to Retain, so if the PVC is deleted, it does not delete the PV or the underlying EBS volume. Ref 2i2c-org#2906 Ref 2i2c-org#2717
|
Merging this PR will trigger the following deployment actions. Support and Staging deployments
Production deploymentsNo production hub upgrades will be triggered |
| @@ -0,0 +1,18 @@ | |||
| apiVersion: storage.k8s.io/v1 | |||
There was a problem hiding this comment.
| apiVersion: storage.k8s.io/v1 | |
| # By default EKS clusters doesn't provide a StorageClass for gp3 based storage, that is | |
| # reported to be more cost efficient than gp2 based storage. Due to that we install a gp3 | |
| # StorageClass that can be explicitly referenced in AWS clusters. | |
| apiVersion: storage.k8s.io/v1 |
| kind: StorageClass | ||
| metadata: | ||
| name: gp3 | ||
| # Use the AWS EBS CSI provisioner |
There was a problem hiding this comment.
| # Use the AWS EBS CSI provisioner |
| apiVersion: storage.k8s.io/v1 | ||
| kind: StorageClass | ||
| metadata: | ||
| name: gp3 | ||
| # Use the AWS EBS CSI provisioner | ||
| provisioner: ebs.csi.aws.com | ||
| parameters: | ||
| # gp3 is cheaper than gp2 | ||
| type: gp3 | ||
| fsType: ext4 | ||
| # Do not delete the underlying EBS volume whenever this PV & PVC | ||
| # are deleted! This allows us recovery in case of accidental deletion | ||
| reclaimPolicy: Retain | ||
| # Create the EBS volume only when the pod consuming it tries to use it. | ||
| # This ensures the EBS volume is created in the correct *zone*, so we | ||
| # don't end up with issues where the volume is in a different zone (but | ||
| # same region) as our pods. | ||
| volumeBindingMode: WaitForFirstConsumer |
There was a problem hiding this comment.
Provisioner value?
This is from a gp2 StorageClass resource in a recent AWS cluster (catalystproject-africa), and it looked the same in the older 2i2c-aws-us cluster.
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
storageclass.kubernetes.io/is-default-class: "true"
name: gp2
parameters:
fsType: ext4
type: gp2
provisioner: kubernetes.io/aws-ebs
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumerNote that provisioner is set to something else than proposed in this PR. Should we go for whats observed in our clusters?
Default storage class?
Reading up about StorageClass defaults, it seems that the annotation controls it, and having two with the same annotation is not good. It would be good if we could have this a default storage class, but it seems out of control for the Helm chart to make it so.
I guess we can't make it the default without also having a pre-setup step for new clusters to make the gp2 StorageClass no longer the default storage class. So, let's not think about this now.
Installable in GKE/AKS?
Is it okay to create a storageclass that references a provisioner that isn't available? Yes maybe, but its also a bit confusing that we create a resource of no relevance.
Can we make this install only in AWS/EKS based clusters?
There was a problem hiding this comment.
provisioner: kubernetes.io/aws-ebs is deprecated since 1.17 - see https://aws.amazon.com/blogs/containers/amazon-ebs-csi-driver-is-now-generally-available-in-amazon-eks-add-ons/, search for the word 'deprecated'. I'm annoyed EKS still ships with that.
I agree that we should not make this the default, don't want to fiddle with that.
Let me think about the optionality question. We have been shipping a GKE SSD provisioner (with a deprecated provisioner even!) for all our clusters apparently.
There was a problem hiding this comment.
Yeah, making this not be present by default but explicitly be set is the right thing to do! I'll do that.
There was a problem hiding this comment.
Yeah, making this not be present by default but explicitly be set is the right thing to do! I'll do that.
I think making the choice to install a StorageClass resource explicit is a great!
|
Going to close this in favor of opening something along the lines of #3112 later for AWS, once I get the GCP stuff done. |
We add this unconditionally to all clusters for simplification, so we can set storageClass: gp3 for new clusters that come up on AWS without issue. This doesn't change the default, and does not change the storageclass in existing clusters. In addition to using gp3, it also sets reclaimPolicy to Retain, so if the PVC is deleted, it does not delete the PV or the underlying EBS volume.
Ref #2906
Ref #2717