[ENH] ✨ implement s3instanceref and default and add

allowedNamespaces

README.md

@@ -74,25 +74,12 @@ The operator exposes a few parameters, meant to be set as arguments, though it's
 
 The parameters are summarized in the table below :
 
-| Flag name                       | Default          | Environment variable | Multiple values allowed | Description                                                                                                                                    |
-| ------------------------------- | ---------------- | -------------------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
-| `health-probe-bind-address`     | `:8081`          | -                    | no                      | The address the probe endpoint binds to. Comes from Operator SDK.                                                                              |
-| `leader-elect`                  | `false`          | -                    | no                      | Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager. Comes from Operator SDK. |
-| `metrics-bind-address`          | `:8080`          | -                    | no                      | The address the metric endpoint binds to. Comes from Operator SDK.                                                                             |
-| `region`                        | `us-east-1`      | -                    | no                      | The region to configure for the S3 client.                                                                                                     |
-| `s3-access-key`                 | -                | `S3_ACCESS_KEY`      | no                      | The access key used to interact with the S3 server.                                                                                            |
-| `s3-ca-certificate-base64`      | -                | -                    | yes                     | (Optional) Base64 encoded, PEM format CA certificate, for https requests to the S3 server.                                                     |
-| `s3-ca-certificate-bundle-path` | -                | -                    | no                      | (Optional) Path to a CA certificates bundle file, for https requests to the S3 server.                                                         |
-| `s3-endpoint-url`               | `localhost:9000` | -                    | no                      | Hostname (or hostname:port) of the S3 server.                                                                                                  |
-| `s3-provider`                   | `minio`          | -                    | no                      | S3 provider (possible values : `minio`, `mockedS3Provider`)                                                                                    |
-| `s3-secret-key`                 | -                | `S3_SECRET_KEY`      | no                      | The secret key used to interact with  the S3 server.                                                                                           |
-| `useSsl`                        | true             | -                    | no                      | Use of SSL/TLS to connect to the S3 server                                                                                                     |
-| `bucket-deletion`               | false            | -                    | no                      | Trigger bucket deletion on the S3 backend upon CR deletion. Will fail if bucket is not empty.                                                  |
-| `policy-deletion`               | false            | -                    | no                      | Trigger policy deletion on the S3 backend upon CR deletion                                                                                     |
-| `path-deletion`                 | false            | -                    | no                      | Trigger path deletion on the S3 backend upon CR deletion. Limited to deleting the `.keep` files used by the operator.                          |
-| `s3User-deletion`               | false            | -                    | no                      | Trigger S3User deletion on the S3 backend upon CR deletion.                                                                                    |
-| `override-existing-secret`      | false            | -                    | no                      | Update secret linked to s3User if already exist, else noop                                                                                     |
-| `s3LabelSelector`               | ""               | -                    | no                      | Filter resource that this instance will manage. If Empty all resource in the cluster will be manage                                            |
+| Flag name                   | Default | Environment variable | Multiple values allowed | Description                                                                                                                                    |
+| --------------------------- | ------- | -------------------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
+| `health-probe-bind-address` | `:8081` | -                    | no                      | The address the probe endpoint binds to. Comes from Operator SDK.                                                                              |
+| `leader-elect`              | `false` | -                    | no                      | Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager. Comes from Operator SDK. |
+| `metrics-bind-address`      | `:8080` | -                    | no                      | The address the metric endpoint binds to. Comes from Operator SDK.                                                                             |  |
+| `override-existing-secret`  | false   | -                    | no                      | Update secret linked to s3User if already exist, else noop                                                                                     |
 ## Minimal rights needed to work
 
 The Operator need at least this rights:
@@ -166,10 +153,15 @@ metadata:
   name: s3-default-instance                     # Name of the S3Instance
 spec:
   s3Provider: minio                             # Type of the Provider. Can be "mockedS3Provider" or "minio"
-  urlEndpoint: minio.example.com                # URL of the Provider
-  secretName: minio-credentials                 # Name of the secret containing 2 Keys S3_ACCESS_KEY and S3_SECRET_KEY
+  url: https://minio.example.com                # URL of the Provider
+  secretRef: minio-credentials                  # Name of the secret containing 2 Keys S3_ACCESS_KEY and S3_SECRET_KEY
+  caCertSecretRef: minio-certs                  # Name of the secret containing key ca.crt with cert of s3provider
   region: us-east-1                             # Region of the Provider
-  useSSL: true                                  # useSSL to query the Provider
+  allowedNamespaces: []                         # namespaces allowed to have buckets, policies, ... Wildcard prefix/suffix allowed. If empty only the same namespace as s3instance is allowed
+  bucketDeletionEnabled: true                   # Allowed bucket entity suppression on s3instance                             
+  policyDeletionEnabled: true                   # Allowed policy entity suppression on s3instance          
+  pathDeletionEnabled: true                     # Allowed path entity suppression on s3instance          
+  s3UserDeletionEnabled: true                   # Allowed s3User entity suppression on s3instance          
 ```
 
 ### Bucket example
@@ -307,6 +299,13 @@ spec:
 
 Each S3user is linked to a kubernetes secret which have the same name that the S3User. The secret contains 2 keys: `accessKey` and `secretKey`.
 
+### :info: How works s3InstanceRef
+
+S3InstanceRef can get the following values:
+- empty: In this case the s3instance use will be the default one configured at startup if the namespace is in the namespace allowed for this s3Instance
+- `s3InstanceName`: In this case the s3Instance use will be the s3Instance with the name `s3InstanceName` in the current namespace (if the current namespace is allowed)
+- `namespace/s3InstanceName`: In this case the s3Instance use will be the s3Instance with the name `s3InstanceName` in the namespace `namespace` (if the current namespace is allowed to use this s3Instance)
+
 ## Operator SDK generated guidelines
 
 <details>

api/v1alpha1/bucket_types.go

@@ -37,8 +37,12 @@ type BucketSpec struct {
 	Paths []string `json:"paths,omitempty"`
 
 	// s3InstanceRef where create the bucket
-	// +kubebuilder:validation:Optional
-	S3InstanceRef string `json:"s3InstanceRef,omitempty"`
+	// +kubebuilder:validation:Pattern=`^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?(/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?)?$`
+	// +kubebuilder:validation:MinLength=1
+	// +kubebuilder:validation:MaxLength=127
+	// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="s3InstanceRef is immutable"
+	// +kubebuilder:default=s3-operator/default
+	S3InstanceRef string `json:"s3InstanceRef"`
 
 	// Quota to apply to the bucket
 	// +kubebuilder:validation:Required

api/v1alpha1/path_types.go

@@ -37,7 +37,11 @@ type PathSpec struct {
 	Paths []string `json:"paths,omitempty"`
 
 	// s3InstanceRef where create the Paths
-	// +kubebuilder:validation:Optional
+	// +kubebuilder:default=s3-operator/default
+	// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="s3InstanceRef is immutable"
+	// +kubebuilder:validation:Pattern=`^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?(/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?)?$`
+	// +kubebuilder:validation:MinLength=1
+	// +kubebuilder:validation:MaxLength=127
 	S3InstanceRef string `json:"s3InstanceRef,omitempty"`
 }
 

api/v1alpha1/policy_types.go

@@ -37,7 +37,11 @@ type PolicySpec struct {
 	PolicyContent string `json:"policyContent"`
 
 	// s3InstanceRef where create the Policy
-	// +kubebuilder:validation:Optional
+	// +kubebuilder:default=s3-operator/default
+	// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="s3InstanceRef is immutable"
+	// +kubebuilder:validation:Pattern=`^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?(/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?)?$`
+	// +kubebuilder:validation:MinLength=1
+	// +kubebuilder:validation:MaxLength=127
 	S3InstanceRef string `json:"s3InstanceRef,omitempty"`
 }
 

api/v1alpha1/s3instance_types.go

@@ -28,27 +28,46 @@ type S3InstanceSpec struct {
 
 	// type of the S3Instance
 	// +kubebuilder:validation:Required
-	S3Provider string `json:"s3Provider"`
+	// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="S3Provider is immutable"
+	// +kubebuilder:default=minio
+	// +kubebuilder:validation:Enum=minio;mockedS3Provider
+	S3Provider string `json:"s3Provider,omitempty"`
 
 	// url of the S3Instance
 	// +kubebuilder:validation:Required
-	UrlEndpoint string `json:"urlEndpoint"`
+	Url string `json:"url"`
 
-	// SecretName associated to the S3Instance containing accessKey and secretKey
+	// Ref to Secret associated to the S3Instance containing accessKey and secretKey
 	// +kubebuilder:validation:Required
-	SecretName string `json:"secretName"`
+	SecretRef string `json:"secretRef"`
 
 	// region associated to the S3Instance
-	// +kubebuilder:validation:Required
-	Region string `json:"region"`
+	// +kubebuilder:validation:Optional
+	Region string `json:"region,omitempty"`
 
-	// useSSL when connecting to the S3Instance
+	// Secret containing key ca.crt with the certificate associated to the S3InstanceUrl
 	// +kubebuilder:validation:Optional
-	UseSSL bool `json:"useSSL,omitempty"`
+	CaCertSecretRef string `json:"caCertSecretRef,omitempty"`
 
-	// CaCertificatesBase64 associated to the S3InstanceUrl
+	// AllowedNamespaces to use this S3InstanceUrl if empty only the namespace of this instance url is allowed to use it
 	// +kubebuilder:validation:Optional
-	CaCertificatesBase64 []string `json:"caCertificateBase64,omitempty"`
+	AllowedNamespaces []string `json:"allowedNamespaces,omitempty"`
+
+	// BucketDeletionEnabled Trigger bucket deletion on the S3 backend upon CR deletion. Will fail if bucket is not empty.
+	// +kubebuilder:default=false
+	BucketDeletionEnabled bool `json:"bucketDeletionEnabled,omitempty"`
+
+	// PolicyDeletionEnabled Trigger policy deletion on the S3 backend upon CR deletion.
+	// +kubebuilder:default=false
+	PolicyDeletionEnabled bool `json:"policyDeletionEnabled,omitempty"`
+
+	// PathDeletionEnabled Trigger path deletion on the S3 backend upon CR deletion. Limited to deleting the `.keep` files used by the operator.
+	// +kubebuilder:default=false
+	PathDeletionEnabled bool `json:"pathDeletionEnabled,omitempty"`
+
+	// S3UserDeletionEnabled Trigger S3 deletion on the S3 backend upon CR deletion.
+	// +kubebuilder:default=false
+	S3UserDeletionEnabled bool `json:"s3UserDeletionEnabled,omitempty"`
 }
 
 // S3InstanceStatus defines the observed state of S3Instance

api/v1alpha1/s3user_types.go

@@ -39,7 +39,11 @@ type S3UserSpec struct {
 	SecretName string `json:"secretName"`
 
 	// s3InstanceRef where create the user
-	// +kubebuilder:validation:Optional
+	// +kubebuilder:default=s3-operator/default
+	// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="s3InstanceRef is immutable"
+	// +kubebuilder:validation:Pattern=`^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?(/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?)?$`
+	// +kubebuilder:validation:MinLength=1
+	// +kubebuilder:validation:MaxLength=127
 	S3InstanceRef string `json:"s3InstanceRef,omitempty"`
 }
 

api/v1alpha1/types.go

@@ -0,0 +1,16 @@
+package v1alpha1
+
+// Definitions to manage status condition types
+const (
+	// ConditionReconciled represents the status of the resource reconciliation
+	ConditionReconciled = "Reconciled"
+)
+
+// Definitions to manage status condition reasons
+const (
+	Reconciling     = "Reconciling"
+	Unreachable     = "Unreachable"
+	CreationFailure = "CreationFailure"
+	Reconciled      = "Reconciled"
+	DeletionFailure = "DeletionFailure"
+)

config/crd/bases/s3.onyxia.sh_buckets.yaml

@@ -58,11 +58,19 @@ spec:
                 - default
                 type: object
               s3InstanceRef:
+                default: s3-operator/default
                 description: s3InstanceRef where create the bucket
+                maxLength: 127
+                minLength: 1
+                pattern: ^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?(/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?)?$
                 type: string
+                x-kubernetes-validations:
+                - message: s3InstanceRef is immutable
+                  rule: self == oldSelf
             required:
             - name
             - quota
+            - s3InstanceRef
             type: object
           status:
             description: BucketStatus defines the observed state of Bucket

config/crd/bases/s3.onyxia.sh_paths.yaml

@@ -44,8 +44,15 @@ spec:
                   type: string
                 type: array
               s3InstanceRef:
+                default: s3-operator/default
                 description: s3InstanceRef where create the Paths
+                maxLength: 127
+                minLength: 1
+                pattern: ^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?(/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?)?$
                 type: string
+                x-kubernetes-validations:
+                - message: s3InstanceRef is immutable
+                  rule: self == oldSelf
             required:
             - bucketName
             type: object

config/crd/bases/s3.onyxia.sh_policies.yaml

@@ -42,8 +42,15 @@ spec:
                 description: Content of the policy (IAM JSON format)
                 type: string
               s3InstanceRef:
+                default: s3-operator/default
                 description: s3InstanceRef where create the Policy
+                maxLength: 127
+                minLength: 1
+                pattern: ^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?(/[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?)?$
                 type: string
+                x-kubernetes-validations:
+                - message: s3InstanceRef is immutable
+                  rule: self == oldSelf
             required:
             - name
             - policyContent

config/crd/bases/s3.onyxia.sh_s3instances.yaml

@@ -35,32 +35,65 @@ spec:
           spec:
             description: S3InstanceSpec defines the desired state of S3Instance
             properties:
-              caCertificateBase64:
-                description: CaCertificatesBase64 associated to the S3InstanceUrl
+              allowedNamespaces:
+                description: AllowedNamespaces to use this S3InstanceUrl if empty
+                  only the namespace of this instance url is allowed to use it
                 items:
                   type: string
                 type: array
+              bucketDeletionEnabled:
+                default: false
+                description: BucketDeletionEnabled Trigger bucket deletion on the
+                  S3 backend upon CR deletion. Will fail if bucket is not empty.
+                type: boolean
+              caCertSecretRef:
+                description: Secret containing key ca.crt with the certificate associated
+                  to the S3InstanceUrl
+                type: string
+              pathDeletionEnabled:
+                default: false
+                description: PathDeletionEnabled Trigger path deletion on the S3 backend
+                  upon CR deletion. Limited to deleting the `.keep` files used by
+                  the operator.
+                type: boolean
+              policyDeletionEnabled:
+                default: false
+                description: PolicyDeletionEnabled Trigger policy deletion on the
+                  S3 backend upon CR deletion.
+                type: boolean
               region:
                 description: region associated to the S3Instance
                 type: string
               s3Provider:
+                default: minio
                 description: type of the S3Instance
+                enum:
+                - minio
+                - mockedS3Provider
                 type: string
-              secretName:
-                description: SecretName associated to the S3Instance containing accessKey
-                  and secretKey
+                x-kubernetes-validations:
+                - message: S3Provider is immutable
+                  rule: self == oldSelf
+              s3UserDeletionEnabled:
+                default: false
+                description: S3UserDeletionEnabled Trigger S3 deletion on the S3 backend
+                  upon CR deletion.
+                type: boolean
+              secretRef:
+                description: Ref to Secret associated to the S3Instance containing
+                  accessKey and secretKey
                 type: string
-              urlEndpoint:
+              url:
                 description: url of the S3Instance
                 type: string
-              useSSL:
-                description: useSSL when connecting to the S3Instance
-                type: boolean
             required:
-            - region
+            - bucketDeletionEnabled
+            - pathDeletionEnabled
+            - policyDeletionEnabled
             - s3Provider
-            - secretName
-            - urlEndpoint
+            - s3UserDeletionEnabled
+            - secretRef
+            - url
             type: object
           status:
             description: S3InstanceStatus defines the observed state of S3Instance