Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker, BlobInfoCache: try to reuse blobs from set of all known compressed blobs when pushing across registries #1645

Merged

Commits on Nov 1, 2023

  1. docker, BlobInfoCache: try to reuse compressed blobs when pushing acr…

    …oss registries
    
    It seems we try to reuse blobs only for the specified registry, however
    we can have valid known compressed digests across registry as well
    following pr attempts to use that by doing following steps.
    
    * `CandidateLocations2` now processes all known blobs and appends them
      to returned candidates at the lowest priority. As a result when
    `TryReusingBlob` tries to process these candidates and if the blobs
    filtered by the `Opaque` set by the `transport` fail to match then
    attempt is made against all known blobs (ones which do not belong to the
    current registry).
    
    * Increase the sample set of potential blob reuse to all known
      compressed digests , also involving the one which do not belong to
    current registry.
    
    * If a blob is found match it against the registry where we are
      attempting to push. If blob is already there consider it a `CACHE
    HIT!` and reply skipping blob, since its already there.
    
    How to verify this ?
    
    * Remove all images `buildah rmi --all` // needed so all new blobs can
      be tagged again in common bucket
    * Remove any previous `blob-info-cache` by
    
    ```console
    rm /home/<user>/.local/share/containers/cache/blob-info-cache-v1.boltdb
    ```
    
    ```console
    $ skopeo copy docker://registry.fedoraproject.org/fedora-minimal docker://quay.io/fl/test:some-tag
    $ buildah pull registry.fedoraproject.org/fedora-minimal
    $ buildah tag registry.fedoraproject.org/fedora-minimal quay.io/fl/test
    $ buildah push quay.io/fl/test
    ```
    
    ```console
    Getting image source signatures
    Copying blob a3497ca15bbf skipped: already exists
    Copying config f7e02de757 done
    Writing manifest to image destination
    Storing signatures
    ```
    
    Signed-off-by: Aditya R <arajan@redhat.com>
    flouthoc committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    539f9ee View commit details
    Browse the repository at this point in the history