[xcm/pallet_xcm] New XCM version change improvements

[bkontur]

Summary

All the xcm-builder and xcm-executor modules are implemented with xcm::latest::prelude::* or xcm::latest::*. This means that when we add a new XCM version to the xcm module and change latest to point to the new XCM version, it affects everything.

One of the affected aspects is the XCM version handling in the pallet_xcm, where we store operational data with XcmVersion (e.g., SupportedVersion). Currently, all data use xcm::latest::XCM_VERSION (e.g., destination XCM versions). The only way to migrate these data is to set CurrentMigration::<T>::put(VersionMigrationStage::default());, but this is not used anywhere, only in the old migration, which is not used.

One consequence is that the "notify subscribers when the XCM version changes" feature does not work without proper migration. Yes, we have force_xcm_version or fn wrap_version with fn note_unknown_version (we have one scenario for bridges where we don't use wrap_version + note_unknown_version, because SubscribeVersion does not work over bridge - issue), which could help and/or trigger negotiation from scratch, but then the "version notify" and SubscribeVersion become somewhat useless.

We didn't encounter this issue with XCMv3 because a migration was added to the runtimes here.

Example of problem/issue

We encountered this issue on BridgeHubRococo. However, when we upgrade all testnet runtimes to version 1.7.0, the problem will persist across all instances. In our case, we stored some XCM versions in SupportedVersion with XcmVersion=3. The BridgeHubRococo was subsequently upgraded with runtime versions featuring XCMv4, without any associated migration process. As a result, pallet_xcm now uses XCM_VERSION=4 as a key to operational data, but no data were migrated to XcmVersion=4. This has led to the malfunctioning of certain functionalities. Unfortunately, there is currently no mechanism to trigger the migration; it can only occur with another runtime upgrade that includes the necessary migration steps.

TODO/Improvements (minimal)

• Implement the fn try_state function in pallet_xcm to verify that all stored operational data has been migrated to the current XCM version, ensuring migration steps are not overlooked.
• Add reusable MigrateTo(XcmVersion) to the pallet_xcm - whenever the XCM version is updated, include the corresponding migration step MigrateTo(XcmVersion::4) in the runtime. After the upgrade, these migration steps can be removed.

Improvements (other possibilities)

(Maybe over-complicated and improvements (minimal) above are enough)

• Avoid direct usage of XCM_VERSION in pallet_xcm. Instead, create a new storage item or reuse pub const CurrentXcmVersion as pub storage CurrentXcmVersion. This approach ensures that any changes to XCM_VERSION won't require automatic migration by default, preventing potential disruptions. Migrations will only be necessary if a runtime chooses to upgrade to a higher version, at which point they can add the corresponding migration step or trigger an extrinsic. Note: This approach may result in additional storage reads.
  • Introduce an extrinsic that allows manual triggering to initiate CurrentMigration.
• Explore other sophisticated solutions for adding new XCM version to the runtime. (Maybe without need to add/remove MigrateTo(XcmVersion))
• Change storage version when upgrading to new XCM version? (That could force us to add migration)
• automatic update of SafeXcmVersion? Which can be done just by force_default_xcm_version?
• simplify version handling of pallet_xcm, now we use XCM_VERSION and LatestVersionedLocation, maybe we should not use Versioned* struct for storage and use just xcm::latest and do migrations, it could pretty simplify stuff
• AdvertisedXcmVersion vs CurrentXcmVersion vs SafeXcmVersion
• RemoteLockedFungibles remove XcmVersion [XCM] pallet_xcm internal data changes proposal #6188