Feature: Configure Services' Endpoint URLs via Environment Variables

[bradynpoulsen]

I've been looking the documentation about environment variables and see that you are able to configure credentials, regions, and profiles using environment variables, but I do not see a way to configure the endpoint url option to point to our internal proxy.

My suggestion is to configure the endpoint url if either the AWS_<REGION>_<SERVICE>_ENDPOINT or AWS_<SERVICE>_ENDPOINT (in that order of priority) environment variables are available.

I suggest making this change in one of these locations (that I have found so far) in order of suggestion preference:

• Aws\ClientResolver::_default_endpoint_provider
• Aws\Endpoint\PartitionEndpointProvider::getPartition

For reference purposes:

• Aws\Credentials\CredentialsProvider use of environment variables
• PHP getenv Documentation

[bradynpoulsen]

As a heads up, I am starting a PR to provide an implementation of this

[imshashank]

@bradynpoulsen You can provide the endpoint as part of the client.

Please have a look at the documentation here: https://docs.aws.amazon.com/aws-sdk-php/v3/guide/guide/configuration.html#endpoint

We would like to know more about your scenario and why would you prefer setting up endpoints as environment variable.

[bradynpoulsen]

We have 2 different use cases right now.

1. Local Environment
   We inherit everything from the environment currently and are wanting to stub out the APIs. But it feels weird that we can currently configure everything through the environment but have to add some sort of if condition to add the 'endpoint' configuration only during non-production environments.

2. Feature Deployments via Docker
   We are deploying our application in a disposable way for integration testing against feature/bug branches to verify issues are being resolved. Same thing as above that everything is currently configured through the environment (excluding the version), but they have to go through a proxy to pass through the outbound firewall.

Maybe it's just me, but it feel inconsistent to allow key things (credentials/environment) to easily be configured through environment variables, but then prevent something like the endpoint configuration from being allowed.

[imshashank]

@bradynpoulsen Hi, thanks a lot for this suggestion and for opening the PR. This is a design change and would affect all existing customers. We are working out the standards and practices to follow before allowing endpoints to be configured via endpoints URL. Please give us some time before we implement the feature. Thanks a lot for your patience.

[bradynpoulsen]

@imshashank Sounds good to me. I think these environment variables could be useful with any of the SDKs, so if a standard can be established, that always sounds like the better direction.

[glebsts]

+1. We are using Atlassian LocalStack running in docker to test our app running in docker, and we can set i.e. access key and default region to same as in localstack through app container env vars, but we cannot change endpoint to point to localstack url in same manner. Seems weird.

[bvitale]

+1

[hexpunk]

Any movement on this? It's been a while since we last heard about this.

[hholst80]

I know this use-case isn't going to fly but... It would be much better to be able to NOT having to specify an alias or wrapper for aws and "just work". If I know the awscli is installed somewhere in the container the following would work and it would allow us to spawn a regular S3 enabled container and configure the S3 access through environment variables, only.

.env

AWS_SECRET_KEY_ID=foo
AWS_SECRET_ACCESS_KEY=bar
AWS_S3_ENDPOINT=endpoint.csp.com

docker-compose.yml

version: 3
services:
  foo:
    image: ubuntu:18.04
    environment:
      - AWS_SECRET_KEY_ID
      - AWS_SECRET_ACCESS_KEY
      - AWS_S3_ENDPOINT

[ajit0003]

Has this been implemented yet?

[hholst80]

I implemented it here https://github.com/morecontainers/awscli as a patch on vanilla awscli. It should probably be implemented properly in boto but I have not found a problem yet with my work around.

[shadiramadan]

Whatever happened here? The PR was closed and then left to die. @kstich any ideas? I'm also using localstack and having to wrap all the client calls with custom code to only pass in an endpoint url via a custom defined env var. It's weird this isn't a default.

[kellertk]

Thank you for posting your feedback here, and our apologies that we’ve been thinking this over for a long time without much forward motion. There are similar requests to implement this feature in a few of the AWS SDKs and the AWS CLI, so in order to coordinate those teams - and hopefully make the discussions a little easier to follow - we’ve created a new issue in aws/aws-sdk here: aws/aws-sdk#229

[tim-finnigan]

Hi all,

We recently added a pull request (aws/aws-sdk#230) that contains a proposal based on community comments and suggestions and our own discussions. This document proposes to extend the options for configuring the endpoint to allow users to provide an endpoint URL independently for each AWS service via an environment variable or a profile subsection in the shared configuration file.

You can read the proposal here.

For more information on how to give feedback, please see this comment on the aws/aws-sdk repository:

aws/aws-sdk#229 (comment)

Thanks!

[stobrien89]

Hi all,

As mentioned, aws/aws-sdk#230 contains a proposal for this feature based on community comments and suggestions. Work on this is still ongoing, but I think we've reached a place where this feature request is no longer needed on the PHP SDK's repository— The AWS SDKs and Tools team as a whole has assumed responsibility for this feature. I think we'll be seeing some updates on this very soon.