Support for https localstack endpoints

[youngm]

Module

LocalStack

Proposal

Several real AWS services (such as S3) no longer support http. This is beginning to trickle down to into the SDKs. For example, the AWS SDK recently made a change that messes up the contentEncoding property when invoking the service via http (aws/aws-sdk-java-v2#4725). It is unclear if the Java SDK will ever fix this since arguably S3 only supports https.

It appears localstack supports https. It would be nice if it were easy to use https localstack endpoints for services like S3 (localstack/localstack#9782)

[eddumelendez]

Hi, please check this comment in LocalStack. So, in order to add the valid certificate and accept localhost or 127.0.0.1, you should follow this guide. For what I see, Testcontainers API already provides everything to make this happen

[youngm]

@eddumelendez I dug into this a bit deeper and actually localstack https already works without any configuration changes. The only issue I see with TestContainers it test containers doesn't make it easy to discover the https endpoint since org.testcontainers.containers.localstack.LocalStackContainer.getEndpointOverride(EnabledService) hard codes http and doesn't use the localstack.cloud urls. So, it would be nice if LocalStackContainer made it easier to discover and use the https endpoint. This sample code works for me for S3:

    s3Client =
        S3Client.builder()
            .credentialsProvider(awsCredentialsProvider)
            .region(Region.of(LOCAL_STACK.getRegion()))
            .endpointOverride(
                URI.create(
                    "https://s3.localhost.localstack.cloud:"
                        + LOCAL_STACK
                            .getEndpointOverride(LocalStackContainer.Service.S3)
                            .getPort()))
            .build();

[eddumelendez]

Thanks for sharing a working example. getEndpointOverride(LocalStackContainer.Service.S3) is not needed with recent LocalStack and Testcontainers versions, instead use getMappedPort(4566).

I think the best place to add this information is https://docs.localstack.cloud/user-guide/integrations/testcontainers/ sending a PR to LocalStack docs repository because it is a useful information for all Testcontainers modules.

I'm reopening the issue and wait for LocalStack team on this.

[bentsku]

Hello, just chiming in here, but AWS S3 still supports HTTP:

$ % curl -v http://s3.amazonaws.com/<bucket-name>/test.txt
*   Trying 52.217.105.54:80...
* Connected to s3.amazonaws.com (52.217.105.54) port 80 (#0)
> GET /<bucket-name>/test.txt HTTP/1.1
> Host: s3.amazonaws.com
> User-Agent: curl/8.1.2
> Accept: */*
> 
< HTTP/1.1 200 OK
< x-amz-id-2: OaOOtJDvnS4NATj47JDgU488Pgd0OiN7r9dmnv57tUUAqkHnriBjDt/S9s85rfzInNqkNfi/tn4=
< x-amz-request-id: G6DPR9XJZ7HDCM2H
< Date: Wed, 06 Dec 2023 09:53:11 GMT
< Last-Modified: Wed, 06 Dec 2023 09:51:42 GMT
< ETag: "2ca44c915180271304dc38eece525ac4"
< x-amz-server-side-encryption: AES256
< Accept-Ranges: bytes
< Content-Type: binary/octet-stream
< Server: AmazonS3
< Content-Length: 570

So I believe the AWS Java SDK team should still fix the issue above as it could affect real S3 users (not sure how the SDK handles the case when the bucket has . in the name, as it won't support HTTPS when using virtual hosted requests due to SSL certs not matching with a bucket like this test.dot.name.s3.amazonaws.com).

[eddumelendez]

Thanks for the input @bentsku !

[youngm]

I agree. For the record here is the issue I created for the aws-sdk aws/aws-sdk-java-v2#4746