Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Record zstd:chunked format, and annotations, in BlobInfoCache #2487

Merged
merged 7 commits into from
Aug 7, 2024
Merged

Record zstd:chunked format, and annotations, in BlobInfoCache #2487

merged 7 commits into from
Aug 7, 2024

Commits on Aug 6, 2024

  1. Unrelated: Fix a bug in SQLite BlobInfoCache 

    If we don't know an uncompressed digest, don't try using "".

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    cb4dfed View commit details
    Browse the repository at this point in the history

  2. Fix a comment 

    Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    5dcb348 View commit details
    Browse the repository at this point in the history

  3. Add digest -> specific variant, annotation data to BIC 

    The cache implementations are recording both the base and specific compression variant;
CandidateLocations2 all call CandidateTemplateWithCompression to choose the
appropriate variants to return based on CandidateLocations2Options.

This way, neither the BIC implementations nor the transports are not responsible for
converting zstd:chunked entries to zstd entries if the user wants the latter.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    f9d27e8 View commit details
    Browse the repository at this point in the history

  4. Record the specific variant, and TOC annotations, for blobs we compress 

    Introduce distinct uploadedCompressorBaseVariantName and
uploadedCompressorSpecificVariantName fields; that way we now never
call RecordDigestCompressorData with inconsistent zstd / zstd:chunked in one field,
so we can always record data when we see, or create, a zstd:chunked layer,
removing the current hack.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    76af27c View commit details
    Browse the repository at this point in the history

  5. Extend private.ReusedBlob to allow zstd:chunked reuses 

    - Add a CompressionAnnotations field
- Allow turning a known-zstd blob into a zstd:chunked one if we
  know the right annotations

This just adds the fields, nothing sets them yet, should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    243b49d View commit details
    Browse the repository at this point in the history

  6. Allow dockerImageDestination to reuse zstd:chunked blobs 

    - Return the required annotations, if we have them
- If we have a zstd blob and the BIC contains the annotations,
  we don't check for the blob's presence initially. In that case,
  don't skip it if we find the BIC annotations.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    ac2ca25 View commit details
    Browse the repository at this point in the history

  7. Detect zstd:chunked format in source blobs 

    ... instead of only treating it as zstd.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
    @mtrmac
    mtrmac committed Aug 6, 2024
    Configuration menu
    Copy the full SHA
    3d38dae View commit details
    Browse the repository at this point in the history