Skip to content

support cluster configuration with dfly_bench #4493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
romange opened this issue Jan 22, 2025 · 1 comment · Fixed by #4761
Closed

support cluster configuration with dfly_bench #4493

romange opened this issue Jan 22, 2025 · 1 comment · Fixed by #4761
Assignees
@mkaruza
Labels
enhancement New feature or request fun Fun tasks

Comments

@romange
Copy link
Collaborator

romange commented Jan 22, 2025

dfly_bench covers lots of load-testing use-cases but it still does not support the cluster mode.

The goal of this task is to provide cluster support by sending cluster discovery request if the protocol flag FLAGS_P is RESP.

In that case we should create threads x c connections per shard and drive all of them to completion
@romange romange added enhancement New feature or request help wanted Extra attention is needed good first issue Good for newcomers fun Fun tasks interview labels Jan 22, 2025
@romange romange self-assigned this Feb 3, 2025
@adiholden adiholden removed help wanted Extra attention is needed interview labels Mar 9, 2025
@romange romange removed the good first issue Good for newcomers label Mar 9, 2025
@dragonflydb dragonflydb deleted a comment from akulabs8 Mar 9, 2025
@adiholden adiholden assigned adiholden and unassigned romange Mar 9, 2025
@romange
Copy link
Collaborator Author

romange commented Mar 9, 2025 •
edited
Loading

Most cluster support is done, what's missing is handling MOVED responses by moving slots ownership from one Driver instance to another. It is possible to simulate this locally using cluster_mgr.py
Driver::ParseRESP() is where we identify MOVED but currently drivers are not aware of each other, so they can not communicate between them. Note that each thread manages its own driver set via TLocalClient.

@adiholden adiholden assigned mkaruza and unassigned adiholden Mar 9, 2025
mkaruza added a commit that referenced this issue Mar 13, 2025
@mkaruza
feat(dfly_bench): Handle moved slots in cluster
95f7fff 
When cluster slot reconfiguration happens during dfly_bench execution we
will update slot information for each shard connection where this error
is received.

Added `--pool_size` argument that can be used to set number of threads
that are going to be running.

Resolves #4493

Signed-off-by: mkaruza <mario@dragonflydb.io>
@mkaruza mkaruza mentioned this issue Mar 13, 2025
Merged
mkaruza added a commit that referenced this issue Mar 13, 2025
@mkaruza
feat(dfly_bench): Handle moved slots in cluster
b703198 
When cluster slot reconfiguration happens during dfly_bench execution we
will update slot information for each shard connection where this error
is received.

Added `--pool_size` argument that can be used to set number of threads
that are going to be running.

Resolves #4493

Signed-off-by: mkaruza <mario@dragonflydb.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mkaruza mkaruza

Labels
enhancement New feature or request fun Fun tasks
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(dfly_bench): Handle moved slots in cluster dragonflydb/dragonfly
3 participants
@romange @mkaruza @adiholden