Proposed addition of --key-slots argument for pre-sharding. #200
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In an attempt to migrate an existing single instance Redis (Community) server to a 3x3 cluster (3 daemons per vm x 3 vms with 2 replicas each) there didn't seem to be any tools or path to restore the existing keys distributed correctly across the cluster.
Unfortunately, the redis-cli --pipe argument does not handle MOVED responses and thus generates errors when the key attempting to be loaded does not fall within the key slot (CRC16 % 16,384) served by the currently connected master node instance.
This is an open issue on the official Redis github here: redis/redis#6098
This new --key-slots argument allows the ability to pre-shard each .protocol file allowing for manually loading to the appropriate master node. In the example above 3 masters serve the following key slots:
So using the new --key-slots argument 3 individual .protocol files can be generated for individually loading the existing keys (from the single instance) to each of the cluster's 3 master nodes:
Then the dump-shard#.protocol files can be individually loaded to the node masters of the cluster (with the redis-cli --pipe argument):
I'm sure I have probably not written this addition 100% in line with the current code design. However, I wanted to put this
idea out to see if there would be any interest. I also realize that this code base has grown a bit old at this point and doesn't
support the more current version RDB files (I lucked out in my case being against an older Redis 5.0.14 RDB dump file).