-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(auto-cancel): server-side logic for auto canceling obsolete builds #911
Conversation
Codecov Report
@@ Coverage Diff @@
## main #911 +/- ##
==========================================
- Coverage 71.10% 70.39% -0.72%
==========================================
Files 313 315 +2
Lines 13028 13195 +167
==========================================
+ Hits 9263 9288 +25
- Misses 3299 3440 +141
- Partials 466 467 +1
|
|
||
req, err := http.NewRequestWithContext(context.Background(), "DELETE", u, nil) | ||
if err != nil { | ||
return err |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what are your thoughts on attempting the other cancels in the event of an error on 1 of them? maybe track an error in the outer scope and check that at the end? idk
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean like if there are multiple running builds to auto-cancel, and one of those attempts fails?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, i was looking at the loop over executors
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only one build will match, so there should only be one attempt at cancellation.
But we do loop over pending/running builds: https://github.com/go-vela/server/pull/911/files#diff-2e48a4018b70aac91235c0a2f5ec50d7ae186de39e3bd603d7e0fcfe6fd5ab07R25-R65.
I was unsure as to whether or not to immediately error out upon failing one of those (provided they match all the other criteria). What do you think? If this policy was implemented, there really should only be one build at a time that gets the boot. Each new build will supersede the older build, which superseded a previous build, and so on.
All's that to say, even though there are a couple loops here, we're really just performing one operation overall — I think... 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like the current implementation has the potential to take some time before the current build makes it onto the queue dependent on how many builds need to be canceled and how many containers each build spun up.
It waits until all builds are canceled before pushing the current build to the queue. Should we send current build to queue and, in best effort, send off cancellation attempts to the builds that we need to cancel?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also, if it fails canceling any one of the builds it will not attempt to cancel any of the other builds it needs to cancel, i believe? i could be wrong - but if not, i think we want to continue attempting to cancel the other builds.
Depends on go-vela/types#299