Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] any particular reason to set pypeln.utils.TIMEOUT to 0.0001? #96

Open
rudolfix opened this issue Sep 9, 2022 · 0 comments
Open
Labels
bug Something isn't working

Comments

@rudolfix
Copy link

rudolfix commented Sep 9, 2022

Describe the bug
~10 thread based workers saturates the cpu (Python8 / Ubuntu / pypeln 0.4.9) by polling for new items in the input queue in the loop.

Was is the reason to set the timeout to such low value? When I change that to 0.1 (my tasks are IO bound and take around a second to complete) the pipeline still works fine. Is it safe to lower it? Will other pipeline types (ie. task) be affected?

Also polling with 0.0001 timeout is probably below fidelity of OS system timer so that makes the call non blocking or to block for much longer (ie. on windows the effective minimum sleep is 16ms but maybe my knowledge is outdated)

@rudolfix rudolfix added the bug Something isn't working label Sep 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant