feat: add job for marking stalled entity imports as stalled #833
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
deer-wmde
approved these changes
Jul 8, 2024
m90
added a commit
that referenced
this pull request
Jul 16, 2024
m90
added a commit
that referenced
this pull request
Jul 30, 2024
m90
added a commit
that referenced
this pull request
Jul 30, 2024
* Add API endpoints for handling entity imports (#823) * feat: scaffold model for storing data about transferbot runs * feat: scaffold controller logic * fix: wiki access needs to be scoped down to users * fix: use consistent plural form for table name * feat: dispatch mock job on creation * test: scaffold test setup * test: cover further functionality * fix: use proper controller reference * fix: store entire data * fix: only update pending imports * chore: import is not used * chore: rename dummy job * feat: add input validation * refactor: handle enum validation through laravel features * feat: add job for marking stalled entity imports as stalled (#833) * feat: tighten validation on entity parameters (#838) * Run `transferbot` in a Kubernetes Job (#830) * feat: scaffold entity import job * feat: fetch credentials from mediawiki * feat: scaffold control flow for supervising job * feat: add parameterized spec * chore: connect proper job to http layer * fix: adjust kubernetes job spec to pass * feat: add callback handling * refactor: move enum to dedicated file * fix: adjust oauth grants so they work with platform user * fix: tighten validation for domain parameter * test: add test case for import job * chore: change job configuration values * chore: lock transferbot image version * refactor: address review feedback * chore: bump transferbot image version * docs: add CHANGELOG
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ticket https://phabricator.wikimedia.org/T368021
This should have been included in #823 but wasn't.
If the Kubernetes jobs from #830 fail to signal back failure or success back to API for whatever, we should auto-sweep pending jobs as failed so people can at least try another import.