Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crawl and limitTotalTasks #314

Open
mitra42 opened this issue Feb 13, 2020 · 0 comments
Open

Crawl and limitTotalTasks #314

mitra42 opened this issue Feb 13, 2020 · 0 comments
Labels
2SC/ON
Milestone
Bali deploy

Comments

@mitra42
Copy link
Collaborator

mitra42 commented Feb 13, 2020

Either crawl is ignoring limitTotalTasks or its saying it completed when it didn't .

To test .... try a large crawl (like bali-lontar-transcribed) which is ~12k tasks
It will report that it got all of them, and appears to have done so, even though default limit is 3k tasks.
@mitra42 mitra42 added the UpNext label Feb 13, 2020
@mitra42 mitra42 added this to the Bali deploy milestone Feb 13, 2020
@mitra42 mitra42 added the 2SC/ON label Mar 5, 2020
@mitra42 mitra42 removed the UpNext label Mar 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
2SC/ON
Projects
None yet
Milestone
Bali deploy
Development

No branches or pull requests
1 participant
@mitra42