From 966ce708bf4c692c0e056079440639b84c321829 Mon Sep 17 00:00:00 2001 From: Tesshu Flower Date: Thu, 17 Aug 2023 16:22:54 -0400 Subject: [PATCH 1/4] Doc updates for VolumePopulator Signed-off-by: Tesshu Flower --- CHANGELOG.md | 1 + docs/usage/index.rst | 6 + docs/usage/volume-populator/index.rst | 164 ++++++++++++++++++++++++++ 3 files changed, 171 insertions(+) create mode 100644 docs/usage/volume-populator/index.rst diff --git a/CHANGELOG.md b/CHANGELOG.md index 895c6d7cd..6767d4d6b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Restic - New option to run a restic unlock before the backup in the next sync. - Restic - Allow passing through of RCLONE_ env vars from the restic secret to the mover job. +- Volume Populator added for ReplicationDestinations. ### Changed diff --git a/docs/usage/index.rst b/docs/usage/index.rst index e72d60d34..d960e073f 100644 --- a/docs/usage/index.rst +++ b/docs/usage/index.rst @@ -14,6 +14,7 @@ Usage rsync-tls/index syncthing/index cli/index + volume-populator/index There are four different replication methods built into VolSync. Choose the method that best fits your use-case: @@ -54,3 +55,8 @@ Metrics VolSync :doc:`exposes a number of metrics ` that permit monitoring the status of replication relationships via Prometheus. +Volume Populator +================ + +VolSync provides a :doc:`Volume Populator ` to allow creation of PVCs that reference a +ReplicationDestination as a dataSourceRef. \ No newline at end of file diff --git a/docs/usage/volume-populator/index.rst b/docs/usage/volume-populator/index.rst new file mode 100644 index 000000000..430c09367 --- /dev/null +++ b/docs/usage/volume-populator/index.rst @@ -0,0 +1,164 @@ +======================================= +ReplicationDestination Volume Populator +======================================= + +.. toctree:: + :hidden: + + volumepopulator_example + +.. sidebar:: Contents + + .. contents:: ReplicationDestination Volume Populator + :local: + +VolSync includes a Volume Populator controller to allow creation of PVCs that use a dataSourceRef that refers to a +ReplicationDestination resource. + +.. note:: + The VolumePopulator feature of VolSync is available with kubernetes v1.22 and + above with the `AnyVolumeDataSource` feature gate enabled. The `AnyVolumeDataSource` feature gate is + enabled by default as of v1.24. + +When replicating or restoring a PVC via a ReplicationDestination using a VolumeSnapshot, +the end result is a VolumeSnapshot that contains the latestImage from the last successful syncrhonization. +Previously to create a PVC with the synchronized data, you needed to create a PVC with a dataSourceRef that points to +the VolumeSnapshot created by the ReplicationDestination. +The VolSync volume populator means you can instead point the dataSourceRef of a PVC to the VolSync +ReplicationDestination resource. VolSync will take care of finding the latestImage available in the +ReplicationDestination (or waiting for it to appear after replication completes) and then populating the PVC with +the contents of the VolumeSnapshot. + + +Configuring a PVC with ReplicationDestination Volume Populator +============================================================== + +This example will assume you have a working ReplicationDestination setup that is replicating/synchronizing data. +The example used here uses the Rclone mover, but other movers could be used. For more information about setting up the +replication itself, refer to the docs for the mover you are interested in. + +.. note:: + The ReplicationDestination used with the volume populator must use a copyMethod of `Snapshot`. + +Here's an example of a ReplicationDestination. + +.. code-block:: yaml + :caption: ReplicationDestination object + + --- + apiVersion: volsync.backube/v1alpha1 + kind: ReplicationDestination + metadata: + name: rclone-replicationdestination + namespace: dest + spec: + trigger: + # Every 6 minutes, offset by 3 minutes + schedule: "3,9,15,21,27,33,39,45,51,57 * * * *" + rclone: + rcloneConfigSection: "aws-s3-bucket" + rcloneDestPath: "volsync-test-bucket/mysql-pvc-claim" + rcloneConfig: "rclone-secret" + copyMethod: Snapshot + accessModes: [ReadWriteOnce] + capacity: 10Gi + storageClassName: my-sc + volumeSnapshotClassName: my-vsc + status: + lastSyncDuration: 30.038338887s + lastSyncTime: "2023-08-13T07:29:36Z" + latestImage: + apiGroup: snapshot.storage.k8s.io + kind: VolumeSnapshot + name: volsync-rclone-replicationdestination-dest-20230813072935 + +Now a PVC can be created that uses this ReplicationDestination. + +Here's an example PVC that will populate the volume from the snapshot created by the ReplicationDestination. + +.. code-block:: yaml + :caption: PVC object + + --- + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: restored-pvc + namespace: dest + spec: + accessModes: [ReadWriteOnce] + dataSourceRef: + kind: ReplicationDestination + apiGroup: volsync.backube + name: rclone-replicationdestination + resources: + requests: + storage: 10Gi + storageClassName: my-vsc + +The ``.spec.dataSourceRef.kind`` must be set to ``ReplicationDestination`` and the ``.spec.dataSourceRef.apiGroup`` +must be set to ``volsync.backube``. + +``.spec.dataSourceRef.name`` should be the name of your ReplicationDestination. + +The VolSync volume populator controller will start to populate the volume with the latest available snapshot +indicated in the ReplicationDestination at ``.status.latestImage``. In the above example ReplicationDestination this +would be: ``volsync-rclone-replicationdestination-dest-20230813072935``. + +.. note:: + If no latestImage exists yet, then the PVC will remain in pending state until the ReplicationDestination completes + and a snapshot is available. In this way you could create a ReplicationDestination and a PVC that uses the + ReplicationDestination at the same time. The PVC will only start the volume population process after the + ReplicationDestination has completed a replication and a snapshot is available (as seen in ``.status.latestImage``). + + Additionally, if the storage class used (``my-vsc`` in the example) has a ``volumeBindingMode`` of + ``WaitForFirstConsumer``, the volume populator will need to wait until there is a consumer of the PVC before it gets + populated. When a consumer does come along (for example a pod that wants to mount the PVC), then the volume will be + populated at that time. At this point the VolSync volume populator controller will take the latestImage from the + ReplicationDestination, which may have been updated if additional replications have occurred since the PVC was + created. + +Once the PVC has been populated, the status should be updated and it can be used. Here is an example of the PVC +after it has been populated. + +.. code-block:: yaml + :caption: PVC object after volume population complete + + --- + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + annotations: + pv.kubernetes.io/bind-completed: "yes" + pv.kubernetes.io/bound-by-controller: "yes" + volume.beta.kubernetes.io/storage-provisioner: hostpath.csi.k8s.io + volume.kubernetes.io/storage-provisioner: hostpath.csi.k8s.io + creationTimestamp: "2023-08-13T07:24:23Z" + finalizers: + - kubernetes.io/pvc-protection + name: restored-pvc + namespace: dest + resourceVersion: "4520" + uid: 55f748d8-538c-4457-b36f-ad1f956290d2 + spec: + accessModes: [ReadWriteOnce] + dataSource: + kind: ReplicationDestination + apiGroup: volsync.backube + name: rclone-replicationdestination + dataSourceRef: + kind: ReplicationDestination + apiGroup: volsync.backube + name: rclone-replicationdestination + resources: + requests: + storage: 10Gi + storageClassName: my-vsc + volumeMode: Filesystem + volumeName: pvc-dec29e3a-6f21-4eb0-84b2-b9d9d875f486 + status: + accessModes: + - ReadWriteOnce + capacity: + storage: 10Gi + phase: Bound From 19984c8b04ac070dc5447f5e1d35e7624553d530 Mon Sep 17 00:00:00 2001 From: Tesshu Flower Date: Thu, 17 Aug 2023 16:55:38 -0400 Subject: [PATCH 2/4] update rclone/rsync examples to use volumepopulator Signed-off-by: Tesshu Flower --- docs/usage/rclone/database_example.rst | 14 +++-------- docs/usage/restic/database_example.rst | 6 ++++- docs/usage/rsync/database_example.rst | 26 +++++++++++--------- docs/usage/volume-populator/index.rst | 7 +++--- examples/destination-database/mysql-pvc.yaml | 8 +++--- 5 files changed, 31 insertions(+), 30 deletions(-) diff --git a/docs/usage/rclone/database_example.rst b/docs/usage/rclone/database_example.rst index b083154e9..08dd2a700 100644 --- a/docs/usage/rclone/database_example.rst +++ b/docs/usage/rclone/database_example.rst @@ -88,20 +88,14 @@ destination side. At the end of the each successful iteration, the ``Replication updated with the latest snapshot image. Now deploy the MySQL database to the ``dest`` namespace which will use the data that has been replicated. -First we need to identify the latest snapshot from the ``ReplicationDestination`` object. Record the values of -the latest snapshot as it will be used to create a pvc. Then create the Deployment, Service, PVC, -and Secret. -Ensure that the next synchronization cycle does not start while the following -steps are being completed or VolSync may replace the existing snapshot with a -new one before the database starts. +The PVC uses the VolSync volume populator feature and sets the ReplicationDestination +as its dataSourceRef. This will populate the PVC with the latest snapshot contents from the ReplicationDestination. + +Create the Deployment, Service, PVC, and Secret. .. code:: console - # Get the latest snapshot name - $ kubectl get replicationdestination database-destination -n dest --template={{.status.latestImage.name}} - # Substitute that name into the database PVC template - $ sed -i 's/snapshotToReplace/volsync-dest-database-destination-20201203174504/g' examples/destination-database/mysql-pvc.yaml # Start the database $ kubectl create -n dest -f examples/destination-database/ diff --git a/docs/usage/restic/database_example.rst b/docs/usage/restic/database_example.rst index 97833dd6b..adaa979b2 100644 --- a/docs/usage/restic/database_example.rst +++ b/docs/usage/restic/database_example.rst @@ -242,9 +242,13 @@ Once the restore is complete, the ``.status.lastManualSync`` field will match To verify restore, deploy the MySQL database to the ``dest`` namespace which will use the data that has been restored from sourcePVC backup. +Create the Deployment, Service, and Secret. + .. code-block:: console - $ kubectl create -n dest -f examples/destination-database/ + $ kubectl create -n dest -f examples/destination-database/mysql-secret.yaml + $ kubectl create -n dest -f examples/destination-database/mysql-deployment.yaml + $ kubectl create -n dest -f examples/destination-database/mysql-service.yaml Validate that the mysql pod is running within the environment. diff --git a/docs/usage/rsync/database_example.rst b/docs/usage/rsync/database_example.rst index 240d1ac61..721b58dd1 100644 --- a/docs/usage/rsync/database_example.rst +++ b/docs/usage/rsync/database_example.rst @@ -113,14 +113,14 @@ lines: Status: Conditions: - Last Transition Time: 2020-12-03T16:07:35Z + Last Transition Time: 2023-08-03T16:07:35Z Message: Reconcile complete Reason: ReconcileComplete Status: True Type: Reconciled Last Sync Duration: 4.511334577s - Last Sync Time: 2020-12-03T16:09:04Z - Next Sync Time: 2020-12-03T16:10:00Z + Last Sync Time: 2023-08-03T16:09:04Z + Next Sync Time: 2023-08-03T16:10:00Z We will modify the source database by creating an additional database in the mysql pod running in the source namespace. @@ -151,18 +151,22 @@ the destination. Now the mysql database will be deployed to the destination namespace which will use the data that has been replicated. -First we need to identify the latest snapshot from the ReplicationDestination -object. Record the values of the latest snapshot as it will be used to create a -pvc. Then create the Deployment, Service, PVC, and Secret. Ensure that the above -steps are completed before a new replication cycle starts or the latest snapshot -may be replaced before it can be used. +First we need to wait for the next synchronization iteration to complete so the +changes made above to add a new database will be replicated to the destination. .. code:: console - $ kubectl get replicationdestination database-destination -n dest --template={{.status.latestImage.name}} - volsync-dest-database-destination-20201203174504 + $ kubectl get replicationdestination database-destination -n dest --template={{.status.lastSyncTime}} + 2023-08-03T16:29:01Z + +When the above has been updated to a newer time as in the example above, then we can proceed to create the +Deployment, Service, PVC, and Secret. + +The PVC uses the VolSync volume populator feature and sets the ReplicationDestination +as its dataSourceRef. This will populate the PVC with the latest snapshot contents from the ReplicationDestination. + +.. code:: console - $ sed -i 's/snapshotToReplace/volsync-dest-database-destination-20201203174504/g' examples/destination-database/mysql-pvc.yaml $ kubectl create -n dest -f examples/destination-database/ Validate that the mysql pod is running within the environment. diff --git a/docs/usage/volume-populator/index.rst b/docs/usage/volume-populator/index.rst index 430c09367..33e963fb4 100644 --- a/docs/usage/volume-populator/index.rst +++ b/docs/usage/volume-populator/index.rst @@ -21,7 +21,7 @@ ReplicationDestination resource. enabled by default as of v1.24. When replicating or restoring a PVC via a ReplicationDestination using a VolumeSnapshot, -the end result is a VolumeSnapshot that contains the latestImage from the last successful syncrhonization. +the end result is a VolumeSnapshot that contains the latestImage from the last successful synchronization. Previously to create a PVC with the synchronized data, you needed to create a PVC with a dataSourceRef that points to the VolumeSnapshot created by the ReplicationDestination. The VolSync volume populator means you can instead point the dataSourceRef of a PVC to the VolSync @@ -72,9 +72,8 @@ Here's an example of a ReplicationDestination. kind: VolumeSnapshot name: volsync-rclone-replicationdestination-dest-20230813072935 -Now a PVC can be created that uses this ReplicationDestination. - -Here's an example PVC that will populate the volume from the snapshot created by the ReplicationDestination. +Now a PVC can be created that uses this ReplicationDestination. This PVC will be populated with the contents of the +VolumeSnapshot indicated in the ``status.latestImage`` of the ReplicationDestination. .. code-block:: yaml :caption: PVC object diff --git a/examples/destination-database/mysql-pvc.yaml b/examples/destination-database/mysql-pvc.yaml index 4f36fb96c..4faba4988 100644 --- a/examples/destination-database/mysql-pvc.yaml +++ b/examples/destination-database/mysql-pvc.yaml @@ -6,10 +6,10 @@ metadata: spec: accessModes: - ReadWriteOnce - dataSource: - kind: VolumeSnapshot - apiGroup: snapshot.storage.k8s.io - name: snapshotToReplace + dataSourceRef: + kind: ReplicationDestination + apiGroup: volsync.backube + name: database-destination # Name of the ReplicationDestination resources: requests: storage: 2Gi From d39520b528e06240c5f9db74a5897ddce7544ed2 Mon Sep 17 00:00:00 2001 From: Tesshu Flower Date: Mon, 21 Aug 2023 13:25:37 -0400 Subject: [PATCH 3/4] Update rclone/restic exmaples for privileged Signed-off-by: Tesshu Flower --- docs/usage/rclone/database_example.rst | 38 +++++++++++++++++++ docs/usage/restic/database_example.rst | 16 +++++++- docs/usage/volume-populator/index.rst | 2 - examples/rclone/rclone.conf | 2 +- ...lsync_v1alpha1_replicationdestination.yaml | 2 +- .../volsync_v1alpha1_replicationsource.yaml | 2 +- 6 files changed, 56 insertions(+), 6 deletions(-) diff --git a/docs/usage/rclone/database_example.rst b/docs/usage/rclone/database_example.rst index 08dd2a700..faf1e6460 100644 --- a/docs/usage/rclone/database_example.rst +++ b/docs/usage/rclone/database_example.rst @@ -9,6 +9,19 @@ First, create the source namespace and deploy the source MySQL database. .. code:: console $ kubectl create ns source + $ kubectl annotate namespace source volsync.backube/privileged-movers="true" + +.. note:: + The second command to annotate the namespace is used to enable the rclone data mover to run in privileged mode. + This is because this simple example runs MySQL as root. For your own applications, you can run unprivileged by + setting the ``moverSecurityContext`` in your ReplicationSource/ReplicationDestination to match that of your + application in which case the namespace annotation will not be required. See the + :doc:`permission model documentation ` for more details. + +Deploy the source MySQL database. + +.. code:: console + $ kubectl create -f examples/source-database/ -n source Verify the database is running. @@ -41,6 +54,30 @@ Add a new database. > exit $ exit + +Now edit ``examples/rclone/rclone.conf`` with your rclone configuration, or you can deploy minio as object-storage to use +with the examples. + +To start minio in your cluster, run: + +.. code:: console + + $ hack/run-minio.sh + +If using minio then you can edit ``examples/rclone/rclone.conf`` to match the following: + +.. code-block:: none + :caption: rclone.conf for use with local minio + + [rclone-bucket] + type = s3 + provider = Minio + env_auth = false + access_key_id = access + secret_access_key = password + region = us-east-1 + endpoint = http://minio.minio.svc.cluster.local:9000 + Now, deploy the ``rclone-secret`` followed by ``ReplicationSource`` configuration. .. code:: console @@ -78,6 +115,7 @@ on the destination. .. code:: console $ kubectl create ns dest + $ kubectl annotate namespace dest volsync.backube/privileged-movers="true" $ kubectl create secret generic rclone-secret --from-file=rclone.conf=./examples/rclone/rclone.conf -n dest $ kubectl create -f examples/rclone/volsync_v1alpha1_replicationdestination.yaml -n dest diff --git a/docs/usage/restic/database_example.rst b/docs/usage/restic/database_example.rst index adaa979b2..a18d714dc 100644 --- a/docs/usage/restic/database_example.rst +++ b/docs/usage/restic/database_example.rst @@ -13,11 +13,24 @@ A MySQL database will be used as the example application. Creating source PVC to be backed up ----------------------------------- -Create a namespace called ``source``, and deploy the source MySQL database. +Create a namespace called ``source`` .. code-block:: console $ kubectl create ns source + $ kubectl annotate namespace source volsync.backube/privileged-movers="true" + +.. note:: + The second command to annotate the namespace is used to enable the restic data mover to run in privileged mode. + This is because this simple example runs MySQL as root. For your own applications, you can run unprivileged by + setting the ``moverSecurityContext`` in your ReplicationSource/ReplicationDestination to match that of your + application in which case the namespace annotation will not be required. See the + :doc:`permission model documentation ` for more details. + +Deploy the source MySQL database. + +.. code:: console + $ kubectl -n source create -f examples/source-database/ Verify the database is running: @@ -205,6 +218,7 @@ To restore from the backup, create a destination, deploy ``restic-config`` and .. code-block:: console $ kubectl create ns dest + $ kubectl annotate namespace dest volsync.backube/privileged-movers="true" $ kubectl -n dest create -f examples/restic/source-restic/ To start the restore, create a empty PVC for the data: diff --git a/docs/usage/volume-populator/index.rst b/docs/usage/volume-populator/index.rst index 33e963fb4..059c151e9 100644 --- a/docs/usage/volume-populator/index.rst +++ b/docs/usage/volume-populator/index.rst @@ -5,8 +5,6 @@ ReplicationDestination Volume Populator .. toctree:: :hidden: - volumepopulator_example - .. sidebar:: Contents .. contents:: ReplicationDestination Volume Populator diff --git a/examples/rclone/rclone.conf b/examples/rclone/rclone.conf index 150d25022..32ca59585 100644 --- a/examples/rclone/rclone.conf +++ b/examples/rclone/rclone.conf @@ -1,4 +1,4 @@ -[aws-s3-bucket] +[rclone-bucket] type = s3 provider = AWS env_auth = false diff --git a/examples/rclone/volsync_v1alpha1_replicationdestination.yaml b/examples/rclone/volsync_v1alpha1_replicationdestination.yaml index 3186415e7..023d60eee 100644 --- a/examples/rclone/volsync_v1alpha1_replicationdestination.yaml +++ b/examples/rclone/volsync_v1alpha1_replicationdestination.yaml @@ -8,7 +8,7 @@ spec: trigger: schedule: "*/5 * * * *" rclone: - rcloneConfigSection: "aws-s3-bucket" + rcloneConfigSection: "rclone-bucket" rcloneDestPath: "volsync-test-bucket/mysql-pv-claim" rcloneConfig: "rclone-secret" copyMethod: Snapshot diff --git a/examples/rclone/volsync_v1alpha1_replicationsource.yaml b/examples/rclone/volsync_v1alpha1_replicationsource.yaml index b9f8e7587..e595315c2 100644 --- a/examples/rclone/volsync_v1alpha1_replicationsource.yaml +++ b/examples/rclone/volsync_v1alpha1_replicationsource.yaml @@ -9,7 +9,7 @@ spec: trigger: schedule: "*/10 * * * *" rclone: - rcloneConfigSection: "aws-s3-bucket" + rcloneConfigSection: "rclone-bucket" rcloneDestPath: "volsync-test-bucket/mysql-pv-claim" rcloneConfig: "rclone-secret" copyMethod: Snapshot From 0d5baef56748e8d12e14cee0c90c2f0983a22adf Mon Sep 17 00:00:00 2001 From: Tesshu Flower Date: Fri, 25 Aug 2023 14:20:41 -0400 Subject: [PATCH 4/4] Update docs/usage/volume-populator/index.rst Co-authored-by: John Strunk Signed-off-by: Tesshu Flower --- docs/usage/volume-populator/index.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/usage/volume-populator/index.rst b/docs/usage/volume-populator/index.rst index 059c151e9..cf2d652dc 100644 --- a/docs/usage/volume-populator/index.rst +++ b/docs/usage/volume-populator/index.rst @@ -10,8 +10,7 @@ ReplicationDestination Volume Populator .. contents:: ReplicationDestination Volume Populator :local: -VolSync includes a Volume Populator controller to allow creation of PVCs that use a dataSourceRef that refers to a -ReplicationDestination resource. +When a PVC is created that directly references a ReplicationDestination object, VolSync's Volume Populator controller will automatically fill the PVC with the most recent replicated data, alleviating the need to manually specify the name of the Snapshot. .. note:: The VolumePopulator feature of VolSync is available with kubernetes v1.22 and