Add configurable selinux options for datamover backup:

[sseago]

Thank you for contributing to Velero!

Please add a summary of your change

Current podified datamover fails on backup in clusters with SELinux enabled because the default relabeling behavior on volume mount fails for readonly volumes, and then SELinux prevents the pod user from accessing the unlabeled_t files.

This provides user-configurable options via a new installer and node-agent server flag --selinux-datamover to configure the behavior, since there is no one-size-fits-all solution. The three valid non-empty values for this flag are:

• none: no changes (won't work in SELinux environments). Default value if field is empty.
• no-relabeling: Sets securityContext.SELinuxOptions.Type=spc_t. Preferred configuration for SELinux with some performance improvements for volumes with many files, but won't work with Restricted pods.
• no-readonly: Removes volumes.volumeSource.PVC.ReadOnly=true. For SELinux environments where spc_t is not allowed, but may break shallow copy.

Does your change fix a particular issue?

Fixes #(issue)

Please indicate you've done the following:

• [x ] Accepted the DCO. Commits without the DCO will delay acceptance.
• [x ] Created a changelog file (make new-changelog) or comment /kind changelog-not-required on this PR.
• [x ] Updated the corresponding documentation in site/content/docs/main.

[sseago]

Converted to draft as I haven't tested this yet in a real cluster.

[codecov]

Codecov Report

Attention: Patch coverage is 13.15789% with 33 lines in your changes missing coverage. Please review.

Project coverage is 59.15%. Comparing base (42de654) to head (dfb6e69).

Files with missing lines	Patch %	Lines
pkg/cmd/cli/install/install.go	0.00%	9 Missing ⚠️
pkg/cmd/cli/nodeagent/server.go	0.00%	9 Missing ⚠️
pkg/exposer/types.go	0.00%	5 Missing ⚠️
pkg/exposer/csi_snapshot.go	50.00%	2 Missing and 1 partial ⚠️
pkg/install/deployment.go	0.00%	3 Missing ⚠️
pkg/install/daemonset.go	0.00%	1 Missing and 1 partial ⚠️
pkg/install/resources.go	0.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8255      +/-   ##
==========================================
- Coverage   59.21%   59.15%   -0.06%     
==========================================
  Files         367      368       +1     
  Lines       30841    30878      +37     
==========================================
+ Hits        18262    18266       +4     
- Misses      11119    11149      +30     
- Partials     1460     1463       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sseago]

No problems with either of the selinux options in backing up an OpenShift workload, although I have not yet verified either of the options with shallow copy/ceph.
Passing in "none" (or leaving blank), the DU fails as expected with permission denied.

[sseago]

In doing some testing with this PR and ceph, I created my sample app with storageclass ocs-storagecluster-cephfs and configured readOnly backup volumes with the following node agent config:

data:
  backup-pvc.json: |+
    {
        "backupPVC": {
            "ocs-storagecluster-cephfs": {
                "readOnly": true
            }
        }
    }

Testing with the 3 options provided in this PR (verifying in all 3 cases that the PVC was created with ROX access mode):

1. --selinux-datamover=none: Failed with permission denied error as expected, since we're not enabling either of the SELinux fixes implemented here.
2. --selinux-datamover=no-relabeling: This was successful, as expected. Pod has spc_t set, relabeling is disabled, and the pod is able to access the mounted volume.
3. --selinux-noreadonly: This failed for me. I was concerned that this would fail in some way. However, I was not expecting this particular error. Unlike the "none" case above (where relabeling failed but the pod still came up -- just without volume access), here I got a relabeling error during ContainerCreating, so the pod never came up:

    State:          Waiting
      Reason:       CreateContainerError
...
  Warning  Failed                  2s (x5 over 27s)  kubelet                  Error: relabel failed /var/lib/kubelet/pods/90c105b4-a518-42b3-b4f5-48f0c56e6aeb/volumes/kubernetes.io~csi/pvc-51a34281-af1e-48da-8d1e-30e382689820/mount: lsetxattr /var/lib/kubelet/pods/90c105b4-a518-42b3-b4f5-48f0c56e6aeb/volumes/kubernetes.io~csi/pvc-51a34281-af1e-48da-8d1e-30e382689820/mount/data: read-only file system

[sseago]

@msfrucht See my results above -- it looks like for the ceph/ROX use case we must use the spc_t option rather than removing the readonly attribute from pod.spec.volumes.