Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Image Cache #1114

Merged
merged 25 commits into from
Sep 28, 2024
Merged

Image Cache #1114

merged 25 commits into from
Sep 28, 2024

Conversation

amywieliczka
Copy link
Collaborator

@amywieliczka amywieliczka commented Sep 24, 2024

Addresses #1109

Stores thumbnail and media components on s3 at a configured CONTENT_COMPONENT_CACHE set to s3://<bucket>/<prefix>/ at the key: <collection_id>/<request.url>/<component_type: media|thumbnail>/<ETag>/<Last-Modified>

Where request.url, etag, and last-modified have all been run through urllib.parse.quote_plus to encode them.

Thumbnail and Media Components are defined as:

thumbnail_component = {
    'mimetype': 'image/jpeg',
    'path': content_s3_filepath,                # s3 filepath to generated thumbnail
    'dimensions': dimensions,                   # dimensions of generated thumbnail
    'component_content_harvest_metadata': {
        'md5': source_component['md5'],                                 # md5 of response content from mapped thumbnail source url
        'src_content-type': source_component['Content-Type'],           # content type response header from mapped thumbnail source url,
        'mapped_mimetype': mapped_thumbnail_source.get('mimetype'),     # mapped thumbnail source mimetype
        'src_size': source_component['size'],                           # size in bytes of response content from mapped thumbnail source url
        'date_content_component_created': datetime.now().isoformat()
    }
}

media_component = {
    'mimetype': mimetype,                                           # mimetype of generated media
    'path': content_s3_filepath,                                    # s3 path to generated media
    'format': NUXEO_MEDIA_TYPE_MAP.get(mapped_nuxeotype or ''),     # format, used by UI to determine display
    'component_content_harvest_metadata': {
        'md5': source_component['md5'],                             # md5 hash of source file
        'src_content-type': source_component['Content-Type'],       # content type response header from source file
        'mapped_mimetype': mapped_mimetype,                         # mimetype from mapper
        'src_size': source_component['size'],                       # size in bytes of source file
        'date_content_component_created': datetime.now().isoformat()
    }
}

@amywieliczka amywieliczka marked this pull request as ready for review September 25, 2024 22:03
bibliotechy
bibliotechy previously approved these changes Sep 26, 2024
Copy link
Contributor

@bibliotechy bibliotechy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s3 implementation looks good. One small non-blocker comment

Copy link
Collaborator

@barbarahui barbarahui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!!

@amywieliczka amywieliczka merged commit 73b8ae6 into main Sep 28, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants