[Bug] any particular reason to set pypeln.utils.TIMEOUT
to 0.0001?
#96
Labels
bug
Something isn't working
pypeln.utils.TIMEOUT
to 0.0001?
#96
Describe the bug
~10 thread based workers saturates the cpu (Python8 / Ubuntu / pypeln 0.4.9) by polling for new items in the input queue in the loop.
Was is the reason to set the timeout to such low value? When I change that to 0.1 (my tasks are IO bound and take around a second to complete) the pipeline still works fine. Is it safe to lower it? Will other pipeline types (ie. task) be affected?
Also polling with 0.0001 timeout is probably below fidelity of OS system timer so that makes the call non blocking or to block for much longer (ie. on windows the effective minimum sleep is 16ms but maybe my knowledge is outdated)
The text was updated successfully, but these errors were encountered: