Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using invalid container causes action run to hang unless force cancelled #3845

Closed
4 tasks done
PythonCoderAS opened this issue Dec 12, 2024 · 1 comment
Closed
4 tasks done
Labels
bug Something isn't working gha-runner-scale-set Related to the gha-runner-scale-set mode

Comments

@PythonCoderAS
Copy link

Checks

Controller Version

0.9.3

Deployment Method

Helm

Checks

  • This isn't a question or user support case (For Q&A and community support, go to Discussions).
  • I've read the Changelog before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes

To Reproduce

Make a workflow where one of the jobs uses an image that does not exist (such as `ghcr.io/actions/totally-fake-container:1`)

Describe the bug

The workflow gets stuck at "Initializing containers" and does not exit, and the cancel button on the UI does not work. I have to use the REST API's "secret" force-cancel endpoint to stop the workflow.

Describe the expected behavior

The action would fail as soon as it couldn't pull the image

Additional Context

githubConfigUrl: "https://github.com/HARP-research-Inc"
githubConfigSecret:
  github_token: "<redacted>"
containerMode:
  type: "kubernetes"
  kubernetesModeWorkVolumeClaim:
    accessModes: ["ReadWriteOnce"]
    storageClassName: "local-path"
    resources:
      requests:
        storage: 2Gi
minRunners: 3

Controller Logs

https://gist.github.com/PythonCoderAS/3d9dada9b51173822750c42641716d19

Runner Pod Logs

https://gist.github.com/PythonCoderAS/e9f519a1b8a1dfa6f30418527708c398
https://gist.github.com/PythonCoderAS/ebf144381398630d2d3832feb265a132
@PythonCoderAS PythonCoderAS added bug Something isn't working gha-runner-scale-set Related to the gha-runner-scale-set mode needs triage Requires review from the maintainers labels Dec 12, 2024
@nikola-jokic
Copy link
Collaborator

Hey @PythonCoderAS,

The workflow and runner provisioning are completely distinct actions. The scale set in response to a workflow tries to provision runners that will potentially execute the job. The runner is not tied to the workflow it is provisioned for.

For example, if you have min-runners of 1, and 1 workflow is scheduled, then the available runner will pick it up. This is intended behavior. If you fix the image and install the scale set, it should be able to pick up where it left off and execute your workflow.

Closing this issue, but feel free to comment on it 🙂

@nikola-jokic nikola-jokic removed the needs triage Requires review from the maintainers label Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working gha-runner-scale-set Related to the gha-runner-scale-set mode
Projects
None yet
Development

No branches or pull requests

2 participants