Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Page limit from quotas is not always being set correctly. #2369

Closed
ikreymer opened this issue Feb 10, 2025 · 0 comments · Fixed by #2370
Closed

[Bug]: Page limit from quotas is not always being set correctly. #2369

ikreymer opened this issue Feb 10, 2025 · 0 comments · Fixed by #2370
Assignees
@ikreymer
Labels
bug Something isn't working

Comments

@ikreymer
Copy link
Member

Browsertrix Version

1.13.2

What did you expect to happen? What happened instead?

Currently, the page limit from org quotas is only set when a new workflow is created, but not when a workflow is edited.
Additionally, the crawl / workflow details pages report the overall cluster max pages, not the per org quotas, if any.

Reproduction instructions

  1. Start a crawl in an org with quotas.
  2. Observe the crawl and workflow details pages for the crawl.
    Also:
  3. Edit an existing workflow in an org with quotas.
  4. Check the page limit set on the crawl.

Screenshots / Video

No response

Environment

No response

Additional details

Fix forthcoming. The fix is to set the page limit on the crawl when created, and when serializing the workflow, and not store it in the config.
In case the quota is later updated, it would apply to all existing workflows.
@ikreymer ikreymer added the bug Something isn't working label Feb 10, 2025
@ikreymer ikreymer self-assigned this Feb 10, 2025
@ikreymer ikreymer moved this from Todo to Ready in Webrecorder Projects Feb 10, 2025
@ikreymer ikreymer moved this from Triage to Todo in Webrecorder Projects Feb 10, 2025
@ikreymer ikreymer moved this from Ready to In Review in Webrecorder Projects Feb 10, 2025
ikreymer added a commit that referenced this issue Feb 10, 2025
@ikreymer
fix page quotas:
e1ba768 
- add ensure_page_limit_quotas() which sets the limit to the quota
- set the page limit when: creating new crawl, creating configmap, returning crawl workflow
- don't set the quota page limit on new or existing crawl workflows (remove setting it on new workflows) to allow updated quotas to take affect for next crawl
- fixes #2369
@ikreymer ikreymer mentioned this issue Feb 10, 2025
Merged
SuaYoo pushed a commit that referenced this issue Feb 11, 2025
@ikreymer @SuaYoo
fix page quotas:
cc69668 
- add ensure_page_limit_quotas() which sets the limit to the quota
- set the page limit when: creating new crawl, creating configmap, returning crawl workflow
- don't set the quota page limit on new or existing crawl workflows (remove setting it on new workflows) to allow updated quotas to take affect for next crawl
- fixes #2369
@github-project-automation github-project-automation bot moved this from In Review to Done! in Webrecorder Projects Feb 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ikreymer ikreymer

Labels
bug Something isn't working
Projects
Webrecorder Projects
Status: Done!
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix max pages quota setting and display webrecorder/browsertrix
1 participant
@ikreymer