Skip to content

Commit

Permalink
Merge pull request kubernetes-sigs#640 from jonathanrainer/more-contr…
Browse files Browse the repository at this point in the history 
…ol-over-subpath-pattern

Allow more control of the name of the directory created by the Access Point under Dynamic Provisioning
  • Loading branch information
@k8s-ci-robot
k8s-ci-robot authored Aug 15, 2023
2 parents a6c4391 + 7f460f7 commit a07b66b
Show file tree
Hide file tree
Showing 8 changed files with 895 additions and 131 deletions.
2 changes: 2 additions & 0 deletions charts/aws-efs-csi-driver/values.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -178,5 +178,7 @@ storageClasses: []
# gidRangeStart: "1000"
# gidRangeEnd: "2000"
# basePath: "/dynamic_provisioning"
# subPathPattern: "/subPath"
# ensureUniqueDirectory: true
# reclaimPolicy: Delete
# volumeBindingMode: Immediate
44 changes: 23 additions & 21 deletions docs/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,19 +26,21 @@ The following CSI interfaces are implemented:
* Identity Service: GetPluginInfo, GetPluginCapabilities, Probe

### Storage Class Parameters for Dynamic Provisioning
| Parameters | Values | Default | Optional | Description |
|---------------------|--------|---------|-----------|-------------|
| provisioningMode | efs-ap | | false | Type of volume provisioned by efs. Currently, Access Points are supported. |
| fileSystemId | | | false | File System under which access points are created. |
| directoryPerms | | | false | Directory permissions for [Access Point root directory](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-root-directory-access-point) creation. |
| uid | | | true | POSIX user Id to be applied for [Access Point root directory](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-root-directory-access-point) creation and for [user identity enforcement](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-identity-access-points). |
| gid | | | true | POSIX group Id to be applied for [Access Point root directory](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-root-directory-access-point) creation and for [user identity enforcement](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-identity-access-points). |
| gidRangeStart | | 50000 | true | Start range of the POSIX group Id to be applied for [Access Point root directory](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-root-directory-access-point) creation and for [user identity enforcement](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-identity-access-points). Not used if uid/gid is set. For user identity enforcement, this value will be applied as both the uid and the gid. |
| gidRangeEnd | | 7000000 | true | End range of the POSIX group Id. Not used if uid/gid is set. |
| basePath | | | true | Path under which access points for dynamic provisioning is created. If this parameter is not specified, access points are created under the root directory of the file system |
| az | | "" | true | Used for cross-account mount. `az` under storage class parameter is optional. If specified, mount target associated with the az will be used for cross-account mount. If not specified, a random mount target will be picked for cross account mount |

**Note**
| Parameters | Values | Default | Optional | Description |
|-----------------------|--------|-----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| provisioningMode | efs-ap | | false | Type of volume provisioned by efs. Currently, Access Points are supported. |
| fileSystemId | | | false | File System under which access points are created. |
| directoryPerms | | | false | Directory permissions for [Access Point root directory](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-root-directory-access-point) creation. |
| uid | | | true | POSIX user Id to be applied for [Access Point root directory](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-root-directory-access-point) creation. |
| gid | | | true | POSIX group Id to be applied for [Access Point root directory](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-root-directory-access-point) creation. |
| gidRangeStart | | 50000 | true | Start range of the POSIX group Id to be applied for [Access Point root directory](https://docs.aws.amazon.com/efs/latest/ug/efs-access-points.html#enforce-root-directory-access-point) creation. Not used if uid/gid is set. |
| gidRangeEnd | | 7000000 | true | End range of the POSIX group Id. Not used if uid/gid is set. |
| basePath | | | true | Path under which access points for dynamic provisioning is created. If this parameter is not specified, access points are created under the root directory of the file system |
| subPathPattern | | `/${.PV.name}` | true | The template used to construct the subPath under which each of the access points created under Dynamic Provisioning. Can be made up of fixed strings and limited variables, is akin to the 'subPathPattern' variable on the [nfs-subdir-external-provisioner](https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner) chart. Supports `.PVC.name`,`.PVC.namespace` and `.PV.name` |
| ensureUniqueDirectory | | true | true | **NOTE: Only set this to false if you're sure this is the behaviour you want**.<br/> Used when dynamic provisioning is enabled, if set to true, appends the a UID to the pattern specified in `subPathPattern` to ensure that access points will not accidentally point at the same directory. |
| az | | "" | true | Used for cross-account mount. `az` under storage class parameter is optional. If specified, mount target associated with the az will be used for cross-account mount. If not specified, a random mount target will be picked for cross account mount |

**Note**
* Custom Posix group Id range for Access Point root directory must include both `gidRangeStart` and `gidRangeEnd` parameters. These parameters are optional only if both are omitted. If you specify one, the other becomes mandatory.
* When using a custom Posix group ID range, there is a possibility for the driver to run out of available POSIX group Ids. We suggest ensuring custom group ID range is large enough or create a new storage class with a new file system to provision additional volumes.
* `az` under storage class parameter is not be confused with efs-utils mount option `az`. The `az` mount option is used for cross-az mount or efs one zone file system mount within the same aws account as the cluster.
Expand Down Expand Up @@ -179,13 +181,13 @@ This procedure requires Helm V3 or later. To install or upgrade Helm, see [Using
helm repo add aws-efs-csi-driver https://kubernetes-sigs.github.io/aws-efs-csi-driver/
```

1. Update the repo.
2. Update the repo.

```sh
helm repo update aws-efs-csi-driver
```

1. Install a release of the driver using the Helm chart.
3. Install a release of the driver using the Helm chart.

```sh
helm upgrade --install aws-efs-csi-driver --namespace kube-system aws-efs-csi-driver/aws-efs-csi-driver
Expand Down Expand Up @@ -232,19 +234,19 @@ If you want to download the image with a manifest, we recommend first trying the
**Note**
If you encounter an issue that you aren't able to resolve by adding IAM permissions, try the [Manifest \(public registry\)](#-manifest-public-registry-) steps instead.
1. In the following command, replace `region-code` with the AWS Region that your cluster is in. Then run the modified command to replace `us-west-2` in the file with your AWS Region.
2. In the following command, replace `region-code` with the AWS Region that your cluster is in. Then run the modified command to replace `us-west-2` in the file with your AWS Region.
```sh
sed -i.bak -e 's|us-west-2|region-code|' private-ecr-driver.yaml
```
1. Replace `account` in the following command with the account from [Amazon container image registries](add-ons-images.md) for the AWS Region that your cluster is in and then run the modified command to replace `602401143452` in the file.
3. Replace `account` in the following command with the account from [Amazon container image registries](add-ons-images.md) for the AWS Region that your cluster is in and then run the modified command to replace `602401143452` in the file.
```sh
sed -i.bak -e 's|602401143452|account|' private-ecr-driver.yaml
```
1. If you already created a service account by following [Create an IAM policy and role for Amazon EKS](./iam-policy-create.md), then edit the `private-ecr-driver.yaml` file. Remove the following lines that create a Kubernetes service account.
4. If you already created a service account by following [Create an IAM policy and role for Amazon EKS](./iam-policy-create.md), then edit the `private-ecr-driver.yaml` file. Remove the following lines that create a Kubernetes service account.
```
apiVersion: v1
Expand All @@ -257,7 +259,7 @@ If you want to download the image with a manifest, we recommend first trying the
---
```
1. Apply the manifest.
5. Apply the manifest.
```sh
kubectl apply -f private-ecr-driver.yaml
Expand All @@ -277,7 +279,7 @@ For some situations, you may not be able to add the necessary IAM permissions to
"github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/?ref=release-1.X" > public-ecr-driver.yaml
```
1. If you already created a service account by following [Create an IAM policy and role](./iam-policy-create.md), then edit the `private-ecr-driver.yaml` file. Remove the following lines that create a Kubernetes service account.
2. If you already created a service account by following [Create an IAM policy and role](./iam-policy-create.md), then edit the `private-ecr-driver.yaml` file. Remove the following lines that create a Kubernetes service account.
```sh
apiVersion: v1
Expand All @@ -290,7 +292,7 @@ For some situations, you may not be able to add the necessary IAM permissions to
---
```
1. Apply the manifest.
3. Apply the manifest.
```sh
kubectl apply -f public-ecr-driver.yaml
Expand Down
30 changes: 17 additions & 13 deletions examples/kubernetes/dynamic_provisioning/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,13 @@ This example requires Kubernetes 1.17 or later and a driver version of 1.2.0 or
fs-582a03f3
```

1. Download a `StorageClass` manifest for Amazon EFS.
2. Download a `StorageClass` manifest for Amazon EFS.

```sh
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/master/examples/kubernetes/dynamic_provisioning/specs/storageclass.yaml
```

1. Edit [the file](./specs/storageclass.yaml). Find the following line, and replace the value for `fileSystemId` with your file system ID.
3. Edit [the file](./specs/storageclass.yaml). Find the following line, and replace the value for `fileSystemId` with your file system ID.

```
fileSystemId: fs-582a03f3
Expand All @@ -39,29 +39,33 @@ This example requires Kubernetes 1.17 or later and a driver version of 1.2.0 or
* `gidRangeStart` (Optional) - The starting range of the Posix group ID to be applied onto the root directory of the access point. The default value is `50000`.
* `gidRangeEnd` (Optional) - The ending range of the Posix group ID. The default value is `7000000`.
* `basePath` (Optional) - The path on the file system under which the access point root directory is created. If the path isn't provided, the access points root directory is created under the root of the file system.
* `subPathPattern` (Optional) - A pattern that describes the subPath under which an access point should be created. So if the pattern were `${.PVC.namespace}/${PVC.name}`, the PVC namespace is `foo` and the PVC name is `pvc-123-456`, and the `basePath` is `/dynamic_provisioner` the access point would be
created at `/dynamic_provisioner/foo/pvc-123-456`.
* `ensureUniqueDirectories` (Optional) - A boolean that ensures that, if set, a UUID is appended to the final element of
any dynamically provisioned path, as in the above example. This can be turned off but this requires you as the
administrator to ensure that your storage classes are set up correctly. Otherwise, it's possible that 2 pods could
end up writing to the same directory by accident. **Please think very carefully before setting this to false!**

1. Deploy the storage class.
4. Deploy the storage class.

```sh
kubectl apply -f storageclass.yaml
```

1. Test automatic provisioning by deploying a Pod that makes use of the PVC:
2. Test automatic provisioning by deploying a Pod that makes use of the PVC:

1. Download a manifest that deploys a Pod and a PVC.

```sh
curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-efs-csi-driver/master/examples/kubernetes/dynamic_provisioning/specs/pod.yaml
```

1. Deploy the Pod with a sample app and the PVC used by the Pod.
2. Deploy the Pod with a sample app and the PVC used by the Pod.

```sh
kubectl apply -f pod.yaml
```
1. Determine the names of the Pods running the controller.
3. Determine the names of the Pods running the controller.
```sh
kubectl get pods -n kube-system | grep efs-csi-controller
```
Expand All @@ -73,7 +77,7 @@ This example requires Kubernetes 1.17 or later and a driver version of 1.2.0 or
efs-csi-controller-74ccf9f566-wswg9 3/3 Running 0 40m
```

1. After few seconds, you can observe the controller picking up the change \(edited for readability\). Replace `74ccf9f566-q5989` with a value from one of the Pods in your output from the previous command.
4. After few seconds, you can observe the controller picking up the change \(edited for readability\). Replace `74ccf9f566-q5989` with a value from one of the Pods in your output from the previous command.

```sh
kubectl logs efs-csi-controller-74ccf9f566-q5989 \
Expand All @@ -91,7 +95,7 @@ This example requires Kubernetes 1.17 or later and a driver version of 1.2.0 or

If you don't see the previous output, run the previous command using one of the other controller Pods.
1. Confirm that a persistent volume was created with a status of `Bound` to a `PersistentVolumeClaim`:
5. Confirm that a persistent volume was created with a status of `Bound` to a `PersistentVolumeClaim`:
```sh
kubectl get pv
Expand All @@ -104,7 +108,7 @@ This example requires Kubernetes 1.17 or later and a driver version of 1.2.0 or
pvc-5983ffec-96cf-40c1-9cd6-e5686ca84eca 20Gi RWX Delete Bound default/efs-claim efs-sc 7m57s
```
1. View details about the `PersistentVolumeClaim` that was created.
6. View details about the `PersistentVolumeClaim` that was created.
```sh
kubectl get pvc
Expand All @@ -117,7 +121,7 @@ This example requires Kubernetes 1.17 or later and a driver version of 1.2.0 or
efs-claim Bound pvc-5983ffec-96cf-40c1-9cd6-e5686ca84eca 20Gi RWX efs-sc 9m7s
```
1. View the sample app Pod's status until the `STATUS` becomes `Running`.
7. View the sample app Pod's status until the `STATUS` becomes `Running`.

```sh
kubectl get pods -o wide
Expand Down Expand Up @@ -149,7 +153,7 @@ If a Pod doesn't have an IP address listed, make sure that you added a mount tar
[...]
```

1. \(Optional\) Terminate the Amazon EKS node that your Pod is running on and wait for the Pod to be re\-scheduled. Alternately, you can delete the Pod and redeploy it. Complete the previous step again, confirming that the output includes the previous output.
2. \(Optional\) Terminate the Amazon EKS node that your Pod is running on and wait for the Pod to be re\-scheduled. Alternately, you can delete the Pod and redeploy it. Complete the previous step again, confirming that the output includes the previous output.

**Note**
When you want to delete an access point in a file system when deleting PVC, you should specify `elasticfilesystem:ClientRootAccess` to the file system access policy to provide the root permissions.
4 changes: 3 additions & 1 deletion examples/kubernetes/dynamic_provisioning/specs/storageclass.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,6 @@ parameters:
directoryPerms: "700"
gidRangeStart: "1000" # optional
gidRangeEnd: "2000" # optional
basePath: "/dynamic_provisioning" # optional
basePath: "/dynamic_provisioning" # optional
subPathPattern: "${.PVC.namespace}/${.PVC.name}" # optional
ensureUniqueDirectory: "true" # optional
2 changes: 1 addition & 1 deletion go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ require (
github.com/aws/aws-sdk-go v1.44.76
github.com/container-storage-interface/spec v1.6.0
github.com/golang/mock v1.6.0
github.com/google/uuid v1.3.0
github.com/kubernetes-csi/csi-test/v5 v5.0.0
github.com/mitchellh/go-ps v0.0.0-20170309133038-4fdf99ab2936
github.com/onsi/ginkgo/v2 v2.9.0
Expand Down Expand Up @@ -41,7 +42,6 @@ require (
github.com/google/go-cmp v0.5.9 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
github.com/imdario/mergo v0.3.6 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
Expand Down
Loading

0 comments on commit a07b66b

Please sign in to comment.