S3 path style access enabled does not work since 2.18.0 with presigned urls

[OlivierHartmann]

Describe the bug

Hi,
I tried to update the lib from 2.17.243 to the latest. However my tests failed on the presigned urls generation since the EndpointProvider internal logic updates of the 2.18 version.

Expected Behavior

Indeed, instead of getting http://localhost:4567/cdn-test/assets/__2daf5bbb0c374c74a8105b414535cae0?partNumber=1&uploadId=tAgYrawDOiQqe3fyv1ErpTr9bPRYm0QZZrr7U99jlg7LIUohkAp1DQ&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230713T112707Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=test%2F20230713%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Signature=647816a139b7aa57b1b921f1a07b865e9b666b5c250e7f4e4092b2775b98c3f2

Current Behavior

i'm getting http://cdn-test.localhost:4567/assets/__2daf5bbb0c374c74a8105b414535cae0?partNumber=1&uploadId=tAgYrawDOiQqe3fyv1ErpTr9bPRYm0QZZrr7U99jlg7LIUohkAp1DQ&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230713T112707Z&X-Amz-SignedHeaders=host&X-Amz-Expires=86400&X-Amz-Credential=test%2F20230713%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Signature=647816a139b7aa57b1b921f1a07b865e9b666b5c250e7f4e4092b2775b98c3f2

Reproduction Steps

while I've still activated the S3Configuration.pathStyleAccessEnabled(true) option.

Possible Solution

No response

Additional Information/Context

Thank you

AWS Java SDK version used

2.18

JDK version used

17

Operating System and version

Mac 11.7.7

[bhoradc]

Hi @OlivierHartmann,

Thanks for reporting this issue. Unfortunately, I am unable to reproduce it using AWS SDK for Java version 2.18.0.

With the below Presigner configuration with pathStyleAccessEnabled set to true, I am still getting the Presigned url in the https://region-endpoint/bucketname/object.. format.

S3Presigner presigner = S3Presigner.builder()
                .region(region)
                .serviceConfiguration(S3Configuration.builder()
                        .pathStyleAccessEnabled(true)
                        .build())
                .build();

Can you please share the code snippet experiencing this issue?

Regards,
Chaitanya

[OlivierHartmann]

Hi Chaitanya,
You're right it's working. My bad. But I do not understand why it was working before with 2.17, when I had :

        val s3Client = S3Client.builder()
            .region(awsRegionProvider.region)
            .credentialsProvider(awsCredentialsProvider)
            .serviceConfiguration(
                S3Configuration.builder()
                .pathStyleAccessEnabled(true)
                .build()
            ).build()


        val s3Presigner = S3Presigner.builder()
            .credentialsProvider(awsCredentialsProvider)
            .region(awsRegionProvider.region)


        val createMultipartResponse = s3Client.createMultipartUpload(createMultipartRequest)
        
        val preSignedUrls = arrayListOf<URL>()
        while(...) {
            val uploadPartRequest = UploadPartRequest.builder()
                .bucket(assetsSettings.bucket)
                .key(path(key))
                .partNumber(partNumber)
                .uploadId(createMultipartResponse.uploadId())
                .build()
            val uploadPartPresignRequest = UploadPartPresignRequest.builder()
                .uploadPartRequest(uploadPartRequest)
                .signatureDuration(Duration.ofDays(UPLOAD_EXPIRATION_DAYS))
                .build()
            val presignedUploadPartRequest = s3Presigner.presignUploadPart(uploadPartPresignRequest)

            preSignedUrls.add(presignedUploadPartRequest.url())
        }

Adding the S3Configuration to the S3Presigner builder solved the problem.
Thank you,

[bhoradc]

Hi @OlivierHartmann,

I am glad that you were able to identify the issue. I will close this now, but feel free to open again in case you have any new findings on it.

Regards,
Chaitanya

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[LoranceChen]

I'm encounter this problem again. And solved with upgrade s3 sdk to 2.22.3 (latest version for now).

                       val builder = S3AsyncClient
                         .builder()
                         .serviceConfiguration(new Consumer[S3Configuration.Builder] {
                           override def accept(build: S3Configuration.Builder): Unit =
                             build.pathStyleAccessEnabled(false)
                         })
                         .credentialsProvider(credentials)
                         .region(region.region)
                       uriEndpoint.foreach(builder.endpointOverride)
                       builder

Hope my case can give help.