Backport 6065

[RainbowMango]

Which PR needs cherry-picks:

PR #6065

Which release branches need this patch:

• release-1.12
• release-1.11
• release-1.10

In addition, we hope to add a E2E test for the case fixed by #6065.

• E2E test case

How to cherry-pick PRs:

The hack/cherry_pick_pull.sh script can help you initiate a cherry-pick
automatically, please follow the instructions at this guideline.

The script will send the PR for you, please remember copy the release notes from
the original PR by to the new PR description part.

How to join or take the task:

Just reply on the issue with the message /assign in a separate line.

Then, the issue will be assigned to you.

Useful References:

• Release timeline: https://karmada.io/docs/releases
• How to cherry-pick PRs: https://karmada.io/docs/contributor/cherry-picks

Anything else we need to know:

[JimDevil]

/assign