Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retag images to reduce time spent on recomputing unchanged docker images #1167

Merged
merged 8 commits into from
Feb 28, 2024

Conversation

corneliusroemer
Copy link
Contributor

@corneliusroemer corneliusroemer commented Feb 28, 2024

Summary

Bypass building docker images when all relevant input files have not changed (akin to NixOS)

  • Calculate hash of all files relevant for docker build
  • Add that hash as a tag to all built images
  • For future builds, only run builds when there's no docker image with the input file dir hash

This way, docker builds are only done when necessary. You change website only in a commit, then no need to build a new backend docker image.

Docker build times were roughly:

  • 1.30min for Nextclade, though sometimes only 30s if (separate from this PR) docker cache hit
  • 2-4min for website
  • 2-3min for backend

The bypass workflow completes in 20s.

Benefit is not just that we save a few Github action minutes, but also that preview builds are up much faster when only config is changed, this should be particularly helpful for @theosanderson

@corneliusroemer corneliusroemer added the preview Triggers a deployment to argocd label Feb 28, 2024
@corneliusroemer corneliusroemer marked this pull request as ready for review February 28, 2024 01:02
@corneliusroemer corneliusroemer enabled auto-merge (squash) February 28, 2024 01:03
Copy link
Member

@theosanderson theosanderson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks cool

@chaoran-chen
Copy link
Member

Cool! How much time is it saving and could you please provide a brief summary of what you changed? :)

@corneliusroemer corneliusroemer merged commit e7f2d9b into main Feb 28, 2024
12 checks passed
@corneliusroemer corneliusroemer deleted the retag-images branch February 28, 2024 16:06
@corneliusroemer
Copy link
Contributor Author

corneliusroemer commented Feb 28, 2024

Done @chaoran-chen

this is how fast it is now:

image

This was previously:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
preview Triggers a deployment to argocd
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants