Fix Map/Parallel States checking container_runner#status! of finished states #296
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
agrare
commented
Nov 21, 2024
| @@ -61,7 +61,8 @@ def finish(context) | |||
| end | |||
|
|
|||
| def running?(context) | |||
| return true if waiting?(context) | |||
| return true if waiting?(context) | |||
| return false if finished?(context) | |||
There was a problem hiding this comment.
This is the fix for this issue, I'd also like to make sure we don't check running? for a sub-workflow that we know to be finished but this is also good insurance at the Task level.
Fryguy
reviewed
Nov 21, 2024
| @@ -182,8 +182,8 @@ def docker_event_status_to_event(status) | |||
|
|
|||
| def inspect_container(container_id) | |||
| JSON.parse(docker!("inspect", container_id).output).first | |||
| rescue | |||
| nil | |||
| rescue AwesomeSpawn::CommandResultError => err | |||
There was a problem hiding this comment.
Not for this PR, but it's weird to me that docker! method fails and we catch AwesomeSpawn::CommandResultError, as that requires the caller to know the implementation of the callee. Does it make more sense to have docker! do the rescue/catch and raise something?
There was a problem hiding this comment.
That being said, it's all self contained in the same file, so probably not a big deal.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If a state is finished, map/parallel states are re-checking to see if these are ready which causes us to check status of a container that has already been removed.
This leads to:
kubeclient-4.12.0/lib/kubeclient/common.rb:130:inrescue in handle_exception': pods "floe-sleep-f9149969" not found (Kubeclient::ResourceNotFoundError)`This was only seen on kubernetes/openshift because docker&podman were eating the exception and returning
nil.Since I don't think you should check the status of a container that we already knowingly deleted, checking container status and getting an error back should be a Floe::ExecutionError.