Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bulk mode that is mindful of event loop #700

Open
kibertoad opened this issue Sep 19, 2023 · 2 comments
Open

Bulk mode that is mindful of event loop #700

kibertoad opened this issue Sep 19, 2023 · 2 comments

Comments

@kibertoad
Copy link

kibertoad commented Sep 19, 2023

Stemming of a larger text can take long time (10 seconds and more). Considering that Node.js runs on event loop, blocking operations prevent anything else from being processed.

Would it be possible to introduce async method which would automatically chunk text that exceeds X size and run it on different ticks, and then recombine everything back after done, in order to let event loop to process other requests?

@kibertoad
Copy link
Author

@Hugo-ter-Doest I can try to come up with a PR if such change would be accepted.

@MukeshSinghBisht
Copy link
Contributor

MukeshSinghBisht commented Sep 21, 2023

@kibertoad idea is interesting,
Want to add more, as js typcially lack in its computation power, so along with your idea , we can give more flexibility to the user by providing aws lambda support(or any other service which can be helpful at this point) which is good at scaling in case data become extremely large.
Probably that will require configuration to be provided and accomodating those config in current implementation.
Fascilitating these popular platforms can be a good selling point for this library as well.
@Hugo-ter-Doest any thoughts on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants