One click upgrade index version without using reindex

[AmiStrn]

Is your feature request related to a problem? Please describe.
Upgrading OpenSearch versions is not possible if there are indices from the previous major version.
e.g.

• cluster contains indices with version ES 6.8 and then upgraded to OpenSearch 1.X
• now the cluster is version 1.X but indices are still on version 6.8
• can't upgrade to version 2.X until all indices are at version 1.X
• reindex all indices from version 6.8 to 1.X (takes ages and requires alias manipulation to be done seamlessly)

Describe the solution you'd like nb

• It would be great to have an API to in-place upgrade the indices version to the current cluster version in c v ne click
• have the cluster able to handle older versions of indices so the upgrade can happen ASAP and worry about index versions later

Describe alternatives you've considered
found no alternative to reindexing

Additional context
The reindex API is built for a variety of use cases, it is unintuitive that it should be required for upgrading. additionally, wrong index versions simply won't let the nodes start, so in the event that the cluster is in the middle of a rolling in-place upgrade, the upgrade would get stuck midway.
Upgrading should be easy in order to encourage people to use the next version, this is one of the big pain points in that.

relates to - #3757

[dblock]

Do #5451 and #5437 enable this, or is there more? I do think we should keep the issue open as is because it focus on the upgrade path which is likely to need more work like docs/faqs, rather than on the technical item of supporting old versions of indices.

[AmiStrn]

@dblock Those tickets are crucial for the remote indices' use case of auditing data.
As you mention this ticket has more to do with upgrading path.
Making it as smooth as possible for users to migrate to opensearch, and stay up to date without requiring reindexing or difficult processes.

[GitHub4Jyo]

I agree that requiring reindexing for upgrades is a significant pain point, especially for clusters with large volumes of data or indices where "_source": { "enabled": false } In such cases, reindexing becomes impractical and acts as a potential blocker for upgrades, given the time and resource costs. Additionally, it complicates rollback scenarios since reindexed data cannot easily revert to the previous state. Introducing an API for in-place index upgrades or backward compatibility for older index versions would greatly streamline the upgrade process and make it more user-friendly. This change would encourage adoption of newer versions while minimizing downtime and operational complexity.