Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syncing latest changes from upstream master for rook #690

Merged
merged 10 commits into from
Aug 1, 2024
2 changes: 1 addition & 1 deletion .github/workflows/auto-assign.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
assign:
permissions:
# write permissions are needed to assign the issue.
contents: write
issues: write
name: Run self assign job
runs-on: ubuntu-latest
steps:
Expand Down
116 changes: 66 additions & 50 deletions .mergify.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,49 +15,6 @@ pull_request_rules:
comment:
message: Hi @{{author}}, this pull request was opened against a release branch, is it expected? Normally patches should go in the master branch first and then be backported to release branches.

# release-1.10 branch
- name: automerge backport release-1.10
conditions:
- author=mergify[bot]
- base=release-1.10
- label!=do-not-merge
- "status-success=DCO"
- "check-success=canary"
- "check-success=unittests"
- "check-success=golangci-lint"
- "check-success=codegen"
- "check-success=codespell"
- "check-success=lint"
- "check-success=modcheck"
- "check-success=Shellcheck"
- "check-success=yaml-linter"
- "check-success=lint-test"
- "check-success=gen-rbac"
- "check-success=crds-gen"
- "check-success=pvc"
- "check-success=pvc-db"
- "check-success=pvc-db-wal"
- "check-success=encryption-pvc"
- "check-success=encryption-pvc-db"
- "check-success=encryption-pvc-db-wal"
- "check-success=encryption-pvc-kms-vault-token-auth"
- "check-success=encryption-pvc-kms-vault-k8s-auth"
- "check-success=lvm-pvc"
- "check-success=multi-cluster-mirroring"
- "check-success=rgw-multisite-testing"
- "check-success=TestCephSmokeSuite (v1.19.16)"
- "check-success=TestCephSmokeSuite (v1.25.0)"
- "check-success=TestCephHelmSuite (v1.19.16)"
- "check-success=TestCephHelmSuite (v1.25.0)"
- "check-success=TestCephMultiClusterDeploySuite (v1.25.0)"
- "check-success=TestCephUpgradeSuite (v1.19.16)"
- "check-success=TestCephUpgradeSuite (v1.25.0)"
actions:
merge:
method: merge
dismiss_reviews: {}
delete_head_branch: {}

# release-1.11 branch
- name: automerge backport release-1.11
conditions:
Expand Down Expand Up @@ -275,14 +232,64 @@ pull_request_rules:
dismiss_reviews: {}
delete_head_branch: {}

# release-1.10 branch
- actions:
backport:
branches:
- release-1.10
# release-1.15 branch
- name: automerge backport release-1.15
conditions:
- label=backport-release-1.10
name: backport release-1.10
- author=mergify[bot]
- base=release-1.15
- label!=do-not-merge
- "status-success=DCO"
- "check-success=linux-build-all (1.22)"
- "check-success=unittests"
- "check-success=golangci-lint"
- "check-success=codegen"
- "check-success=codespell"
- "check-success=lint"
- "check-success=modcheck"
- "check-success=Shellcheck"
- "check-success=yaml-linter"
- "check-success=lint-test"
- "check-success=gen-rbac"
- "check-success=crds-gen"
- "check-success=docs-check"
- "check-success=pylint"
- "check-success=canary-tests / canary (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / raw-disk-with-object (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / two-osds-in-device (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / osd-with-metadata-partition-device (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / osd-with-metadata-device (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / encryption (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / lvm (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / pvc (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / pvc-db (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / pvc-db-wal (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / encryption-pvc (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / encryption-pvc-db (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / encryption-pvc-db-wal (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / encryption-pvc-kms-vault-token-auth (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / encryption-pvc-kms-vault-k8s-auth (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / lvm-pvc (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / multi-cluster-mirroring (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / rgw-multisite-testing (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / encryption-pvc-kms-ibm-kp (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / multus-cluster-network (quay.io/ceph/ceph:v18)"
- "check-success=canary-tests / csi-hostnetwork-disabled (quay.io/ceph/ceph:v18)"
- "check-success=TestCephSmokeSuite (v1.25.16)"
- "check-success=TestCephSmokeSuite (v1.30.0)"
- "check-success=TestCephHelmSuite (v1.25.16)"
- "check-success=TestCephHelmSuite (v1.30.0)"
- "check-success=TestCephMultiClusterDeploySuite (v1.30.0)"
- "check-success=TestCephObjectSuite (v1.25.16)"
- "check-success=TestCephObjectSuite (v1.30.0)"
- "check-success=TestCephUpgradeSuite (v1.25.16)"
- "check-success=TestCephUpgradeSuite (v1.30.0)"
- "check-success=TestHelmUpgradeSuite (v1.25.16)"
- "check-success=TestHelmUpgradeSuite (v1.30.0)"
actions:
merge:
method: merge
dismiss_reviews: {}
delete_head_branch: {}

# release-1.11 branch
- actions:
Expand Down Expand Up @@ -319,3 +326,12 @@ pull_request_rules:
conditions:
- label=backport-release-1.14
name: backport release-1.14

# release-1.15 branch
- actions:
backport:
branches:
- release-1.15
conditions:
- label=backport-release-1.15
name: backport release-1.15
40 changes: 33 additions & 7 deletions Documentation/CRDs/Object-Storage/ceph-object-store-crd.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,11 @@ spec:
#zone:
#name: zone-a
#hosting:
# advertiseEndpoint:
# dnsName: "mystore.example.com"
# port: 80
# useTls: false
# dnsNames:
# - "mystore.example.com"
# - "mystore.example.org"
```

Expand Down Expand Up @@ -101,8 +104,7 @@ The gateway settings correspond to the RGW daemon settings.
* `sslCertificateRef`: If specified, this is the name of the Kubernetes secret(`opaque` or `tls`
type) that contains the TLS certificate to be used for secure connections to the object store.
If it is an opaque Kubernetes Secret, Rook will look in the secret provided at the `cert` key name. The value of the `cert` key must be
in the format expected by the [RGW
service](https://docs.ceph.com/docs/master/install/ceph-deploy/install-ceph-gateway/#using-ssl-with-civetweb):
in the format expected by the [RGW service](https://docs.ceph.com/docs/master/install/ceph-deploy/install-ceph-gateway/#using-ssl-with-civetweb):
"The server key, server certificate, and any other CA or intermediate certificates be supplied in
one file. Each of these items must be in PEM form." They are scenarios where the certificate DNS is set for a particular domain
that does not include the local Kubernetes DNS, namely the object store DNS service endpoint. If
Expand All @@ -115,7 +117,10 @@ The gateway settings correspond to the RGW daemon settings.
cluster. Rook will look in the secret provided at the `cabundle` key name.
* `hostNetwork`: Whether host networking is enabled for the rgw daemon. If not set, the network settings from the cluster CR will be applied.
* `port`: The port on which the Object service will be reachable. If host networking is enabled, the RGW daemons will also listen on that port. If running on SDN, the RGW daemon listening port will be 8080 internally.
* `securePort`: The secure port on which RGW pods will be listening. A TLS certificate must be specified either via `sslCerticateRef` or `service.annotations`
* `securePort`: The secure port on which RGW pods will be listening. A TLS certificate must be
specified either via `sslCerticateRef` or `service.annotations`. Refer to
[enabling TLS](../../Storage-Configuration/Object-Storage-RGW/object-storage.md#enabling-tls)
documentation for more details.
* `instances`: The number of pods that will be started to load balance this object store.
* `externalRgwEndpoints`: A list of IP addresses to connect to external existing Rados Gateways
(works with external mode). This setting will be ignored if the `CephCluster` does not have
Expand Down Expand Up @@ -155,9 +160,30 @@ The [zone](../../Storage-Configuration/Object-Storage-RGW/ceph-object-multisite.

## Hosting Settings

The hosting settings allow you to host buckets in the object store on a custom DNS name, enabling virtual-hosted-style access to buckets similar to AWS S3 (https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html).

* `dnsNames`: a list of DNS names to host buckets on. These names need to valid according RFC-1123. Otherwise it will fail. Each endpoint requires wildcard support like [ingress loadbalancer](https://kubernetes.io/docs/concepts/services-networking/ingress/#hostname-wildcards). Do not include the wildcard itself in the list of hostnames (e.g., use "mystore.example.com" instead of "*.mystore.example.com"). Add all the hostnames like openshift routes otherwise access will be denied, but if the hostname does not support wild card then virtual host style won't work those hostname. By default cephobjectstore service endpoint and custom endpoints from cephobjectzone is included. The feature is supported only for Ceph v18 and later versions.
`hosting` settings allow specifying object store endpoint configurations. These settings are only
supported for Ceph v18 and higher.

A common use case that requires configuring hosting is allowing
[virtual host-style](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html)
bucket access. This use case is discussed in more detail in
[Rook object storage docs](../../Storage-Configuration/Object-Storage-RGW/object-storage.md#virtual-host-style-bucket-access).

* `advertiseEndpoint`: By default, Rook advertises the most direct connection to RGWs to dependent
resources like CephObjectStoreUsers and ObjectBucketClaims. To advertise a different address
(e.g., a wildcard-enabled ingress), define the preferred endpoint here. Default behavior is
documented in more detail [here](../../Storage-Configuration/Object-Storage-RGW/object-storage.md#object-store-endpoint)
* `dnsName`: The valid RFC-1123 (sub)domain name of the endpoint.
* `port`: The nonzero port of the endpoint.
* `useTls`: Set to true if the endpoint is HTTPS. False if HTTP.
* `dnsNames`: When this or `advertiseEndpoint` is set, Ceph RGW will reject S3 client connections
who attempt to reach the object store via any unspecified DNS name. Add all DNS names that the
object store should accept here. These must be valid RFC-1123 (sub)domain names.
Rook automatically adds the known CephObjectStore service DNS name to this list, as well as
corresponding CephObjectZone `customEndpoints` (if applicable).

!!! Note
For DNS names that support wildcards, do not include wildcards.
E.g., use `mystore.example.com` instead of `*.mystore.example.com`.

## Runtime settings

Expand Down
94 changes: 87 additions & 7 deletions Documentation/CRDs/specification.md
Original file line number Diff line number Diff line change
Expand Up @@ -1915,7 +1915,9 @@ ObjectStoreHostingSpec
</td>
<td>
<em>(Optional)</em>
<p>Hosting settings for the object store</p>
<p>Hosting settings for the object store.
A common use case for hosting configuration is to inform Rook of endpoints that support DNS
wildcards, which in turn allows virtual host-style bucket addressing.</p>
</td>
</tr>
</table>
Expand Down Expand Up @@ -8977,6 +8979,60 @@ and prepares same OSD on that disk</p>
</tr>
</tbody>
</table>
<h3 id="ceph.rook.io/v1.ObjectEndpointSpec">ObjectEndpointSpec
</h3>
<p>
(<em>Appears on:</em><a href="#ceph.rook.io/v1.ObjectStoreHostingSpec">ObjectStoreHostingSpec</a>)
</p>
<div>
<p>ObjectEndpointSpec represents an object store endpoint</p>
</div>
<table>
<thead>
<tr>
<th>Field</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>dnsName</code><br/>
<em>
string
</em>
</td>
<td>
<p>DnsName is the DNS name (in RFC-1123 format) of the endpoint.
If the DNS name corresponds to an endpoint with DNS wildcard support, do not include the
wildcard itself in the list of hostnames.
E.g., use &ldquo;mystore.example.com&rdquo; instead of &ldquo;*.mystore.example.com&rdquo;.</p>
</td>
</tr>
<tr>
<td>
<code>port</code><br/>
<em>
int32
</em>
</td>
<td>
<p>Port is the port on which S3 connections can be made for this endpoint.</p>
</td>
</tr>
<tr>
<td>
<code>useTls</code><br/>
<em>
bool
</em>
</td>
<td>
<p>UseTls defines whether the endpoint uses TLS (HTTPS) or not (HTTP).</p>
</td>
</tr>
</tbody>
</table>
<h3 id="ceph.rook.io/v1.ObjectEndpoints">ObjectEndpoints
</h3>
<p>
Expand Down Expand Up @@ -9160,18 +9216,40 @@ bool
<tbody>
<tr>
<td>
<code>advertiseEndpoint</code><br/>
<em>
<a href="#ceph.rook.io/v1.ObjectEndpointSpec">
ObjectEndpointSpec
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>AdvertiseEndpoint is the default endpoint Rook will return for resources dependent on this
object store. This endpoint will be returned to CephObjectStoreUsers, Object Bucket Claims,
and COSI Buckets/Accesses.
By default, Rook returns the endpoint for the object store&rsquo;s Kubernetes service using HTTPS
with <code>gateway.securePort</code> if it is defined (otherwise, HTTP with <code>gateway.port</code>).</p>
</td>
</tr>
<tr>
<td>
<code>dnsNames</code><br/>
<em>
[]string
</em>
</td>
<td>
<em>(Optional)</em>
<p>A list of DNS names in which bucket can be accessed via virtual host path. These names need to valid according RFC-1123.
Each domain requires wildcard support like ingress loadbalancer.
Do not include the wildcard itself in the list of hostnames (e.g. use &ldquo;mystore.example.com&rdquo; instead of &ldquo;*.mystore.example.com&rdquo;).
Add all hostnames including user-created Kubernetes Service endpoints to the list.
CephObjectStore Service Endpoints and CephObjectZone customEndpoints are automatically added to the list.
<p>A list of DNS host names on which object store gateways will accept client S3 connections.
When specified, object store gateways will reject client S3 connections to hostnames that are
not present in this list, so include all endpoints.
The object store&rsquo;s advertiseEndpoint and Kubernetes service endpoint, plus CephObjectZone
<code>customEndpoints</code> are automatically added to the list but may be set here again if desired.
Each DNS name must be valid according RFC-1123.
If the DNS name corresponds to an endpoint with DNS wildcard support, do not include the
wildcard itself in the list of hostnames.
E.g., use &ldquo;mystore.example.com&rdquo; instead of &ldquo;*.mystore.example.com&rdquo;.
The feature is supported only for Ceph v18 and later versions.</p>
</td>
</tr>
Expand Down Expand Up @@ -9376,7 +9454,9 @@ ObjectStoreHostingSpec
</td>
<td>
<em>(Optional)</em>
<p>Hosting settings for the object store</p>
<p>Hosting settings for the object store.
A common use case for hosting configuration is to inform Rook of endpoints that support DNS
wildcards, which in turn allows virtual host-style bucket addressing.</p>
</td>
</tr>
</tbody>
Expand Down
Loading
Loading