Skip to content

Commit

Permalink
Merge pull request #576 from red-hat-storage/sync_us--master
Browse files Browse the repository at this point in the history
Syncing latest changes from upstream master for rook
  • Loading branch information
travisn authored Feb 20, 2024
2 parents 9047f88 + bcab869 commit b170ccf
Show file tree
Hide file tree
Showing 31 changed files with 197 additions and 182 deletions.
1 change: 1 addition & 0 deletions .github/workflows/daily-nightly-jobs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,7 @@ jobs:
path: /home/runner/work/rook/rook/tests/integration/_output/tests/

canary-tests:
if: github.repository == 'rook/rook'
uses: ./.github/workflows/canary-integration-test.yml
with:
ceph_images: '["quay.io/ceph/ceph:v18", "quay.io/ceph/daemon-base:latest-main-devel", "quay.io/ceph/daemon-base:latest-quincy-devel", "quay.io/ceph/daemon-base:latest-reef-devel"]'
Expand Down
4 changes: 3 additions & 1 deletion Documentation/CRDs/Block-Storage/ceph-block-pool-crd.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,8 +204,10 @@ stretched) then you will have 2 replicas per datacenter where each replica ends
* `name`: The name of Ceph pools is based on the `metadata.name` of the CephBlockPool CR. Some built-in Ceph pools
require names that are incompatible with K8s resource names. These special pools can be configured
by setting this `name` to override the name of the Ceph pool that is created instead of using the `metadata.name` for the pool.
Only the following pool names are supported: `device_health_metrics`, `.nfs`, and `.mgr`. See the example
Only the following pool names are supported: `.nfs`, `.mgr`, and `.rgw.root`. See the example
[builtin mgr pool](https://github.com/rook/rook/blob/master/deploy/examples/pool-builtin-mgr.yaml).
* `application`: The type of application set on the pool. By default, Ceph pools for CephBlockPools will be `rbd`,
CephObjectStore pools will be `rgw`, and CephFilesystem pools will be `cephfs`.

* `parameters`: Sets any [parameters](https://docs.ceph.com/docs/master/rados/operations/pools/#set-pool-values) listed to the given pool
* `target_size_ratio:` gives a hint (%) to Ceph in terms of expected consumption of the total cluster capacity of a given pool, for more info see the [ceph documentation](https://docs.ceph.com/docs/master/rados/operations/placement-groups/#specifying-expected-pool-size)
Expand Down
12 changes: 12 additions & 0 deletions Documentation/CRDs/specification.md
Original file line number Diff line number Diff line change
Expand Up @@ -10300,6 +10300,18 @@ QuotaSpec
<p>The quota settings</p>
</td>
</tr>
<tr>
<td>
<code>application</code><br/>
<em>
string
</em>
</td>
<td>
<em>(Optional)</em>
<p>The application name to set on the pool. Only expected to be set for rgw pools.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="ceph.rook.io/v1.PriorityClassNamesSpec">PriorityClassNamesSpec
Expand Down
4 changes: 2 additions & 2 deletions Documentation/Troubleshooting/ceph-csi-common-issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ to exist in the toolbox:

```console
$ ceph osd lspools
1 device_health_metrics
1 .mgr
2 replicapool
```

Expand All @@ -132,7 +132,7 @@ Now verify the `pool` mentioned in the `storageclass.yaml` exists, such as the e

```console
ceph osd lspools
1 device_health_metrics
1 .mgr
2 replicapool
3 myfs-metadata0
4 myfs-data0
Expand Down
13 changes: 0 additions & 13 deletions Documentation/Upgrade/ceph-upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,19 +39,6 @@ to Rook v1.13.
!!! warning
Ceph v17.2.2 has a blocking issue when running with Rook. Use v17.2.3 or newer when possible.

### Quincy Consideration

In Ceph Quincy (v17), the `device_health_metrics` pool was renamed to `.mgr`. Ceph will perform this
migration automatically. The pool rename will be automatically handled by Rook if the configuration
of the `device_health_metrics` pool is not customized via CephBlockPool.

If the configuration of the `device_health_metrics` pool is customized via CephBlockPool, two extra
steps are required after the Ceph upgrade is complete. Once upgrade is complete:

1. Create a new CephBlockPool to configure the `.mgr` built-in pool. For an example, see
[builtin mgr pool](https://github.com/rook/rook/blob/master/deploy/examples/pool-builtin-mgr.yaml).
2. Delete the old CephBlockPool that represents the `device_health_metrics` pool.

### CephNFS User Consideration

Ceph Quincy v17.2.1 has a potentially breaking regression with CephNFS. See the NFS documentation's
Expand Down
1 change: 1 addition & 0 deletions PendingReleaseNotes.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

- The removal of `CSI_ENABLE_READ_AFFINITY` option and its replacement with per-cluster
read affinity setting in cephCluster CR (CSIDriverOptions section) in [PR](https://github.com/rook/rook/pull/13665)
- Allow setting the Ceph `application` on a pool

## Features

Expand Down
8 changes: 4 additions & 4 deletions deploy/charts/rook-ceph-cluster/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -136,10 +136,10 @@ cephClusterSpec:
count: 2
allowMultiplePerNode: false
modules:
# Several modules should not need to be included in this list. The "dashboard" and "monitoring" modules
# are already enabled by other settings in the cluster CR.
- name: pg_autoscaler
enabled: true
# List of modules to optionally enable or disable.
# Note the "dashboard" and "monitoring" modules are already configured by other settings in the cluster CR.
# - name: rook
# enabled: true

# enable the ceph dashboard for viewing cluster status
dashboard:
Expand Down
23 changes: 22 additions & 1 deletion deploy/charts/rook-ceph/templates/resources.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,9 @@ spec:
spec:
description: NamedBlockPoolSpec allows a block pool to be created with a non-default name.
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -188,7 +191,7 @@ spec:
name:
description: The desired name of the pool if different from the CephBlockPool CR name.
enum:
- device_health_metrics
- .rgw.root
- .nfs
- .mgr
type: string
Expand Down Expand Up @@ -6773,6 +6776,9 @@ spec:
items:
description: NamedPoolSpec represents the named ceph pool spec
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -6941,6 +6947,9 @@ spec:
description: The metadata pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -10700,6 +10709,9 @@ spec:
description: The data pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -11823,6 +11835,9 @@ spec:
description: The metadata pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -12468,6 +12483,9 @@ spec:
description: The data pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -12631,6 +12649,9 @@ spec:
description: The metadata pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down
3 changes: 0 additions & 3 deletions deploy/examples/cluster-on-local-pvc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -179,9 +179,6 @@ spec:
continueUpgradeAfterChecksEvenIfNotHealthy: false
mgr:
count: 1
modules:
- name: pg_autoscaler
enabled: true
dashboard:
enabled: true
ssl: true
Expand Down
3 changes: 0 additions & 3 deletions deploy/examples/cluster-on-pvc-minikube.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -140,9 +140,6 @@ spec:
storage: 10Gi
mgr:
count: 1
modules:
- name: pg_autoscaler
enabled: true
dashboard:
enabled: true
ssl: false
Expand Down
6 changes: 2 additions & 4 deletions deploy/examples/cluster-on-pvc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,8 @@ spec:
skipUpgradeChecks: false
continueUpgradeAfterChecksEvenIfNotHealthy: false
mgr:
count: 1
modules:
- name: pg_autoscaler
enabled: true
count: 2
allowMultiplePerNode: false
dashboard:
enabled: true
ssl: true
Expand Down
23 changes: 22 additions & 1 deletion deploy/examples/crds.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,9 @@ spec:
spec:
description: NamedBlockPoolSpec allows a block pool to be created with a non-default name.
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -190,7 +193,7 @@ spec:
name:
description: The desired name of the pool if different from the CephBlockPool CR name.
enum:
- device_health_metrics
- .rgw.root
- .nfs
- .mgr
type: string
Expand Down Expand Up @@ -6768,6 +6771,9 @@ spec:
items:
description: NamedPoolSpec represents the named ceph pool spec
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -6936,6 +6942,9 @@ spec:
description: The metadata pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -10691,6 +10700,9 @@ spec:
description: The data pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -11814,6 +11826,9 @@ spec:
description: The metadata pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -12456,6 +12471,9 @@ spec:
description: The data pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down Expand Up @@ -12619,6 +12637,9 @@ spec:
description: The metadata pool settings
nullable: true
properties:
application:
description: The application name to set on the pool. Only expected to be set for rgw pools.
type: string
compressionMode:
description: 'DEPRECATED: use Parameters instead, e.g.'
enum:
Expand Down
2 changes: 1 addition & 1 deletion deploy/examples/import-external-cluster.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ ROOK_EXTERNAL_MONITOR_SECRET=mon-secret
OPERATOR_NAMESPACE=rook-ceph # default set to rook-ceph
CSI_DRIVER_NAME_PREFIX=${CSI_DRIVER_NAME_PREFIX:-$OPERATOR_NAMESPACE}
RBD_PROVISIONER=$CSI_DRIVER_NAME_PREFIX".rbd.csi.ceph.com" # csi-provisioner-name
CEPHFS_PROVISIONER=$CSI_DRIVER_NAME_PREFIX=".cephfs.csi.ceph.com" # csi-provisioner-name
CEPHFS_PROVISIONER=$CSI_DRIVER_NAME_PREFIX".cephfs.csi.ceph.com" # csi-provisioner-name
CLUSTER_ID_RBD=$NAMESPACE
CLUSTER_ID_CEPHFS=$NAMESPACE
: "${ROOK_EXTERNAL_ADMIN_SECRET:=admin-secret}"
Expand Down
1 change: 0 additions & 1 deletion deploy/examples/pool-builtin-mgr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ metadata:
spec:
# The required pool name with underscores cannot be specified as a K8s resource name, thus we override
# the pool name created in Ceph with this name property.
# The ".mgr" pool is called "device_health_metrics" in Ceph versions v16.x.y and below.
name: .mgr
failureDomain: host
replicated:
Expand Down
22 changes: 11 additions & 11 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,17 @@ replace github.com/rook/rook/pkg/apis => ./pkg/apis

require (
github.com/IBM/keyprotect-go-client v0.12.2
github.com/aws/aws-sdk-go v1.50.15
github.com/aws/aws-sdk-go v1.50.20
github.com/banzaicloud/k8s-objectmatcher v1.8.0
github.com/ceph/go-ceph v0.25.0
github.com/ceph/go-ceph v0.26.0
github.com/coreos/pkg v0.0.0-20230601102743-20bbbf26f4d8
github.com/csi-addons/kubernetes-csi-addons v0.8.0
github.com/gemalto/kmip-go v0.0.10
github.com/go-ini/ini v1.67.0
github.com/google/go-cmp v0.6.0
github.com/google/uuid v1.6.0
github.com/hashicorp/vault/api v1.12.0
github.com/k8snetworkplumbingwg/network-attachment-definition-client v1.5.0
github.com/k8snetworkplumbingwg/network-attachment-definition-client v1.6.0
github.com/kube-object-storage/lib-bucket-provisioner v0.0.0-20221122204822-d1a8c34382f1
github.com/libopenstorage/secrets v0.0.0-20231011182615-5f4b25ceede1
github.com/pkg/errors v0.9.1
Expand All @@ -32,14 +32,14 @@ require (
golang.org/x/sync v0.6.0
gopkg.in/ini.v1 v1.67.0
gopkg.in/yaml.v2 v2.4.0
k8s.io/api v0.29.1
k8s.io/apiextensions-apiserver v0.29.1
k8s.io/apimachinery v0.29.1
k8s.io/cli-runtime v0.29.1
k8s.io/client-go v0.29.1
k8s.io/cloud-provider v0.29.1
k8s.io/api v0.29.2
k8s.io/apiextensions-apiserver v0.29.2
k8s.io/apimachinery v0.29.2
k8s.io/cli-runtime v0.29.2
k8s.io/client-go v0.29.2
k8s.io/cloud-provider v0.29.2
k8s.io/utils v0.0.0-20231127182322-b307cd553661
sigs.k8s.io/controller-runtime v0.17.1
sigs.k8s.io/controller-runtime v0.17.2
sigs.k8s.io/mcs-api v0.1.0
sigs.k8s.io/yaml v1.4.0
)
Expand Down Expand Up @@ -134,7 +134,7 @@ require (
gopkg.in/evanphx/json-patch.v5 v5.7.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/component-base v0.29.1 // indirect
k8s.io/component-base v0.29.2 // indirect
k8s.io/klog/v2 v2.110.1 // indirect
k8s.io/kube-openapi v0.0.0-20231129212854-f0671cc7e66a // indirect
sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd // indirect
Expand Down
Loading

0 comments on commit b170ccf

Please sign in to comment.