Skip to content

Reapply the new storage config#601

Merged
decko merged 2 commits intomainfrom
reapply_new_storage_config
Jul 28, 2025
Merged

Reapply the new storage config#601
decko merged 2 commits intomainfrom
reapply_new_storage_config

Conversation

@decko
Copy link
Member

@decko decko commented Jul 25, 2025

This reverts commit 90373d9.

Summary by Sourcery

Reapply storage configuration migration in clowdapp.yaml by consolidating S3 settings into a unified STORAGES dict, updating the domain-storage script to leverage Django storages options, and renaming environment variables to follow the PULP_STORAGES__default__… scheme.

Enhancements:

  • Restructure S3 storage settings into a single STORAGES dictionary with nested BACKEND and OPTIONS entries
  • Update the domain-storage-map script to extract AWS credentials and bucket info from Django storages default options
  • Rename deployment environment variables to the new PULP_STORAGES__default__OPTIONS__… and PULP_STORAGES__default__BACKEND scheme
  • Remove deprecated individual storage parameters like AWS_S3_USE_SIGV4 and DEFAULT_FILE_STORAGE

@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Jul 25, 2025

Reviewer's Guide

This PR reapplies the new storage configuration by refactoring the domain-storage script to use Django’s storages API, consolidating all S3 settings into a STORAGES mapping, renaming PULP env vars to nested STORAGES keys, and removing deprecated parameters.

File-Level Changes

Change Details Files
Refactor domain-storage script to use settings.storages.default.options
  • Replaced direct django.conf.settings import alias
  • Switched config generation to pull access_key, secret_key, bucket_name from settings.storages.default.options
deploy/clowdapp.yaml
Consolidate flat AWS env vars into a STORAGES mapping
  • Replaced multiple AWS_S3 and DEFAULT_FILE_STORAGE vars with a unified STORAGES dict
  • Added OPTIONS block for default backend and staticfiles backend entry
deploy/clowdapp.yaml
Rename PULP environment variables to nested STORAGES keys
  • Replaced PULP_AWS_DEFAULT_ACL with PULP_STORAGES__default__OPTIONS__default_acl
  • Replaced PULP_AWS_S3_SIGNATURE_VERSION and ADDRESSING_STYLE with nested keys
  • Replaced PULP_DEFAULT_FILE_STORAGE with PULP_STORAGES__default__BACKEND
deploy/clowdapp.yaml
Remove deprecated storage parameters
  • Removed PULP_S3_USE_SIGV4 env var entries
  • Dropped S3_USE_SIGV4 and related Helm parameters
deploy/clowdapp.yaml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @decko - I've reviewed your changes and they look great!

Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments

### Comment 1
<location> `deploy/clowdapp.yaml:139` </location>
<code_context>
+                  "access_key": "$S3_ACCESS_KEY_ID",
+                  "secret_key": "$S3_SECRET_ACCESS_KEY",
+                  "bucket_name": "$S3_BUCKET_NAME",
+                  "default_acl": "@none None",
+                  "signature_version": "s3v4",
+                  "addressing_style": "path",
</code_context>

<issue_to_address>
The value '@none None' for 'default_acl' may not be recognized by all S3 backends.

Verify that '@none None' is a supported value for 'default_acl' with your S3 backend to avoid permission issues or errors.
</issue_to_address>

### Comment 2
<location> `deploy/clowdapp.yaml:135` </location>
<code_context>
+              "BACKEND": "storages.backends.s3boto3.S3Boto3Storage",
+              "OPTIONS": {
+                  "endpoint_url": "http://${S3_HOSTNAME}:9000",
+                  "region_name": "east",
+                  "access_key": "$S3_ACCESS_KEY_ID",
+                  "secret_key": "$S3_SECRET_ACCESS_KEY",
</code_context>

<issue_to_address>
The region name 'east' may not match AWS region naming conventions.

Standard AWS region names like 'us-east-1' should be used to ensure compatibility with S3 backends and libraries.
</issue_to_address>

<suggested_fix>
<<<<<<< SEARCH
                  "region_name": "east",
=======
                  "region_name": "us-east-1",
>>>>>>> REPLACE

</suggested_fix>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

"access_key": "$S3_ACCESS_KEY_ID",
"secret_key": "$S3_SECRET_ACCESS_KEY",
"bucket_name": "$S3_BUCKET_NAME",
"default_acl": "@none None",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question (bug_risk): The value '@none None' for 'default_acl' may not be recognized by all S3 backends.

Verify that '@none None' is a supported value for 'default_acl' with your S3 backend to avoid permission issues or errors.

Co-authored-by: sourcery-ai[bot] <58596630+sourcery-ai[bot]@users.noreply.github.com>
@decko decko merged commit ff1d49d into main Jul 28, 2025
1 of 3 checks passed
@decko decko deleted the reapply_new_storage_config branch July 28, 2025 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant