Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization: remove lock from RandomEngine #4828

Open
Tracked by #4899
ray6080 opened this issue Feb 2, 2025 · 0 comments
Open
Tracked by #4899

Optimization: remove lock from RandomEngine #4828

ray6080 opened this issue Feb 2, 2025 · 0 comments

Comments

@ray6080
Copy link
Contributor

ray6080 commented Feb 2, 2025

Description

The lock is introduced in #4827 to address race conditions, however, it creates bottleneck in the pipeline.

The correct way to handle this is to introduce LocalState into ScalarFunction and each function instance (e.g., gen_random_uuid in each thread) would have its own RandomEngine with a unique seed generated from the randomEngine under ClientContext.

This requires rework of the interface and implementation of ScalarFunction and ExpressionEvaluator.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant