Workload identity is not working for Azure provider #442

[vikrantoct7]

What steps did you take and what happened:
We are working on restricted environment where are unable to use service principal/storage SAS (disabled by policy at subscription level). Pod identity is another option but MS has obsoleted and replaced with workload identity. We are using workload managed identity for application deployment but can not use this for Valero.

We are using Azure plugin velero/velero-plugin-for-microsoft-azure:v1.6.0

We update velero helm chart for workload managed identity and valye.yaml with following
change #1
labels:
azure.workload.identity/use: "true"

change #2
podAnnotations:
azure.workload.identity/inject-proxy-sidecar: "true"
change #3
serviceAccount:
server:
create: true
name:
annotations:
azure.workload.identity/client-id: cc719c9e-3ded-4ec5-842a-be89c640bef0

after making these changes, velero helm successfully installed (helm generated manifest is attached.)
manifest.txt

Note:
i) Already Establish federated identity credential
ii) Contributor and Storage Blob Data Contributor are assigned to workload identity

What did you expect to happen:
Getting below error by running kubectl logs deployment/velero -n velero

I0321 13:42:22.246928 1 request.go:601] Waited for 1.046083552s due to client-side throttling, not priority and fairness, request: GET:https://10.0.0.1:443/apis/flowcontrol.apiserver.k8s.io/v1beta1?timeout=32s
time="2023-03-21T13:45:50Z" level=error msg="Error getting backup store for this location" backupLocation=velero/default controller=backup-sync error="rpc error: code = Unknown desc = azure.BearerAuthorizer#WithAuthorization: Failed to refresh the Token for request to https://management.azure.com/subscriptions/e80e99a4-f3d7-44f5-82e1-77ceaef31baf/resourceGroups/POC-RG/providers/Microsoft.Storage/storageAccounts/veleroc3ba26992a57/listKeys?%24expand=kerb&api-version=2019-06-01: StatusCode=404 -- Original Error: adal: Refresh request failed. Status Code = '404'. Response body: getting assigned identities for pod velero/velero-84d8455c9b-lklch in CREATED state failed after 16 attempts, retry duration [5]s, error: . Check MIC pod logs for identity assignment errors\n Endpoint http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fmanagement.azure.com%2F" error.file="/go/src/velero-plugin-for-microsoft-azure/velero-plugin-for-microsoft-azure/object_store.go:217" error.function=main.getStorageAccountKey logSource="pkg/controller/backup_sync_controller.go:100"

The following information will help us better understand what's going on:
It seems that Currently backupStorageLocation.config.serviceAccount is only supported with GCP provider. when we use service account in config setting, helm chart through exception that service account is not supported for azure provider.

Looking resolution to make velero work with azure workload managed identity.
this is critical blocker

if Azure workload managed identity is not supported by velero or velero/velero-plugin-for-microsoft-azure:v1.6.0 , i will convert this into feature. please confirm

[ywk253100]

Dup of #5116, please see the latest status about Workload Identity support in #5116 (comment).

I'm closing this one.