[NEW] Rolling downgrade, forward compatibility

[zuiderkwast]

The problem/use-case that the feature addresses

After a rolling upgrade of a cluster, if some problem is found, such as CPU or memory usage or some bug, the administrator wants to be able to a rolling downgrade again and take some time to investigate the problem.

When adding new nodes to a cluster, the standard procedure is to add new nodes as replicas to existing nodes. For replication to work, we require that a replica's RDB version >= primary's RDB version. This works for rolling upgrade, but it doesn't work for downgrading if the RDB version has changed.

Some system that I've been made aware of is stuck on Redis 6 (RDB 9) because of the requirement to be able to do rolling downgrades.

The cluster bus is already backward and forward compatible AFAIC. It would be good to avoid breaking that. :)

Note that we're only talking about adding new nodes of an old version to a cluster. We don't need to start an old node with a config file from a new version such as nodes.conf, nor do we want to reuse a node-id when replacing nodes in the cluster.

Description of the feature

Possibility to generate RDB in an older version than the latest, if no keys require the newer RDB version.

An example is the bump from RDB 10 to 11 (that I'm guilty of by introducing listpack encoding for sets). If we configure the node to generate RDB 10, the node can easily be made to store this listpack as a RDB_TYPE_SET (which is just each element as a string) instead of RDB_TYPE_SET_LISTPACK (a listpack dump). The bump from 9 to 10 is similar.

Alternatives you've considered

#59 = Full sync without RDB preamble, although it could be combined with a REPLCONF to check if the replica supports the current RDB version and fallback to AOF without preamble. (RDB is a bit faster and smaller in size.)

Additional information

Redis never supported downgrades. This would be a Valkey feature that would make Valkey superior in terms of compatibility.

Valkey/Redis	RDB	Why bump
Redis 5.0	9
Redis 6.0	9
Redis 6.2	9
Redis 7.0	10	Change ziplist to listpack for hashes, sets, etc.
Valkey 7.2	11	Add listpack encoding to sets
Valkey 8.0	11

[madolson]

The major decisions:

1. Do we want to be able to support this downgrade.
2. How far back to we want to support restoring into older versions? (At least one version, maybe much further)
3. How will we document this.
4. What is the format we will use to support forward compatibility. One option is to have the primary send an older RDB compatible object. Another option is to send it in AOF format.

Consensus from core meeting seems to say yes for 1.

[madolson]

We also want to create a separate issue outlining how we can add guardrails to prevent usage of new features which would break forward compatibility. The discussed example was for hash field expiration, which must necessarily change the RDB format.