[BACKPORT][v1.5.1][BUG - 1.5.0 Upgrade] Instance Manager Pods CrashLoop After Upgrade From 1.4.0

[github-actions]

backport #6264

[longhorn-io-github-bot]

Pre Ready-For-Testing Checklist

• Where is the reproduce steps/test steps documented?
  The reproduce steps/test steps are at:

  • Create a PVC with storageClassName in annotation instead of spec

  apiVersion: v1
  kind: PersistentVolumeClaim
  metadata:
    name: test
    annotations:
      volume.beta.kubernetes.io/storage-class: longhorn
  spec:
    accessModes:
      - ReadWriteOnce
    # storageClassName: longhorn
    resources:
      requests:
        storage: 1Gi
  

  • Write some data into the Volume
  • Backup the Volume
  • describe the BackupVolume

  k describe lhbv -n longhorn-system
  

  • Can still get the storageClassName from BackupVolume CR status

• Is there a workaround for the issue? If so, where is it documented?
  The workaround is at:

• manually edit pvc to add "storageClassName" to pvc.Spec

• Have the backend code been merged (Manager, Engine, Instance Manager, BackupStore etc) (including backport-needed/*)?
  The PR is at

• fix(backup): pvc storage class could be in annotation (backport #2063) longhorn-manager#2064

[roger-ryao]

Verified on v1.5.x-head 20230712

• longhorn v1.5.x-head 3b7a875
• longhorn-manager v1.5.x-head longhorn/longhorn-manager@bef6044

The test steps

1. Set up a Longhorn v1.4.2 cluster with some orphan resources.
2. Create a Pod with PVC with storageClassName in annotation instead of spec
   Create a Pod with a PVC that has the storageClassName specified in the annotation instead of the spec.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: longhorn-volv-pvc
annotations:
  volume.beta.kubernetes.io/storage-class: longhorn
spec:
accessModes:
  - ReadWriteOnce
# storageClassName: longhorn
resources:
  requests:
    storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
name: volume-test
namespace: default
spec:
restartPolicy: Always
containers:
- name: volume-test
  image: nginx
  imagePullPolicy: IfNotPresent
  livenessProbe:
    exec:
      command:
        - ls
        - /data/lost+found
    initialDelaySeconds: 5
    periodSeconds: 5
  volumeMounts:
  - name: volv
    mountPath: /data
  ports:
  - containerPort: 80
volumes:
- name: volv
  persistentVolumeClaim:
    claimName: longhorn-volv-pvc

3. Write some data into the Volume
4. Backup the Volume
5. describe the BackupVolume

kubectl describe lhbv -n longhorn-system

6. Upgrade to v1.5.x-head
7. describe the BackupVolume after upgrade
   Verify that the storageClassName is present in the BackupVolume CR status.

Result Passed

1. The storageClassName was present in the BackupVolume CR status after upgraded.