Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix pool management #382

Open
sylwiaszunejko opened this issue Oct 14, 2024 · 0 comments
Open

Fix pool management #382

sylwiaszunejko opened this issue Oct 14, 2024 · 0 comments

Comments

@sylwiaszunejko
Copy link
Collaborator

Pool/metadata management in the driver needs a refactor. It is currently a mess of thread pools, callbacks and random locks sprinkled in some places that are hard to understand. E.g. there is an assumption in the code on_add is called rarely enough that all the handling can be pretty much sequential. This is no longer the case, so the assumptions are violated. (See #317)

The solution would be to move to a model from Rust Driver (I think Java Driver does something very similar): there should be a thread (or asyncio task) that manages the cluster and pools. Driver communicates with it using messages. Then such task doesn't have to worry about concurrency because it is the only owner of the data. Apart from that each connection pool can have it's own task that manages the pool.

This model is easy to understand for developer that reads the code and nullifies most of the problem that we see here in Python Driver.

Refs: #317 #380

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant