Chunking/diffs of ostree Dockerfiles?

[expitau]

Use case

I want to use Fedora CoreOS, with custom layering (https://github.com/coreos/layering-examples), but I'm concerned about wire efficiency during updates. Specifically, I want to layer very large packages (vscode, steam, chromium, etc.), in a Dockerfile, but this causes rpm-ostree updates to be very large, essentially downloading all of the packages in their entirety every update (even if the package itself didn't change, simply because the base image has updated). Ideally, I would be able to apply kernel and library updates without downloading the entire image again.

Constraints:

• Everything must be served over a container registry, a custom http server is too much overhead to maintain

Attempted solutions

I thought that I could use ostree-rs-ext to unencapsulate container images, push to a local registry, and then use rpm-ostree to re-encapsulate them into chunked container images. I suspect this chunking would be sufficient for my use case, but unfortunately ostree-rs-ext cannot unencapsulate images built with docker (unless I am misunderstanding). Attempting this returned an error that the image has non-ostree layers. Is there a way or planned way for rpm-ostree to convert existing images into this chunked format? Is there another tool I could use?

I also saw (https://blogs.gnome.org/alexl/2020/05/13/putting-container-updates-on-a-diet/), which provides a way to upload a diff to the registry. While I don't prefer this option (as I need to explicitly specify to/from tags to diff), I could make it work. However, I don't know if there's a way to make rpm-ostree actually use these diffs, to my knowledge this hasn't been implemented. Is there currently a way to pull diffs with rpm-ostree?

TL;DR

Are you aware of a solution where I can build a chunked or diffed docker image from a Dockerfile, that can be used by rpm-ostree?

[cgwalters]

Yes, we should have one central doc for this issue. There are multiple techniques here. One that helps is building with SOURCE_DATE_EPOCH set. However, this runs today still into rpm-software-management/rpm#2219

There is also a ton of discussion in containers/image#902 (comment) (see the whole PR)

I also was unaware until recently that flatpak is aiming to standardize https://github.com/flatpak/flatpak-oci-specs/blob/main/image-deltas.md but this needs to be supported in containers/image too.