-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support custom S3 endpoint URIs #209
Comments
Thanks for the info, I'll take a peek once time permits, hopefully within a week or two. |
Absolutely! |
@seesemichaelj Please try out https://github.com/DanEngelbrecht/golongtail/releases/tag/v0.3.4-pre14 and see how the new option feels and works out., |
@DanEngelbrecht I can confirm that |
To be able to use non-AWS S3 buckets (Cloudflare R2, DigitalOcean Spaces, etc), you need to specify a custom endpoint URI. Unfortunately, the AWS SDK doesn't provide an environment or config file variable to override this value (aws/aws-cli#4454); it only supports a CLI parameter or using the direct SDK variable). So for longtail to support these custom endpoints, the SDK references would need to be changed
Using the docs here, here, here, and here, I believe that
golongtail/longtailstorelib/s3Store.go
Line 60 in 883905d
or
where
ConfiguredBaseEndpointUri
could be some new CLI flag (i.e.--s3-base-endpoint
).It's also very possible that it's as simple as the below (I've seen another example say
aws.Config
hasEndpoint
, but the official docs say otherwise)Once the endpoint can be configured, users can configure the region and credentials using the
~/.aws/config
and~/.aws/credentials
files mentioned in longtail's readme or useAWS_REGION
,AWS_ACCESS_KEY_ID
,AWS_SECRET_ACCESS_KEY
, andAWS_SESSION_TOKEN
environment variables (which get read in during theLoadDefaultConfig
call, so there's no reason to make any changes to support those variables.I would attempt to implement this myself and make a PR, but my golang experience is measured in weeks and I would likely just be going down a rabbithole for simple issues
The text was updated successfully, but these errors were encountered: