Regression: cannot set S3Object contentEncoding

[youngm]

Describe the bug

No matter what I set for contentEncoding it is always changed to "aws-chunked"

Expected Behavior

The contentEncoding I provide will be stored with the object.

Current Behavior

the contentEncoding is changed to "aws-chunked"

Reproduction Steps

Here is a test case:

    var putRequest =
        PutObjectRequest.builder()
            .bucket(TEST_BUCKET)
            .key(TEST_FILE)
            .contentEncoding("bogus")
            .build();

    s3Client.putObject(putRequest, RequestBody.fromBytes(TEST_BYTES));
    var response = s3Client.getObject(GetObjectRequest.builder().bucket(TEST_BUCKET).key(TEST_FILE).build());
    assertThat(response.response().contentEncoding()).isEqualTo("bogus");

Possible Solution

This appears to be a regression related to #4653 but I'm not sure. The problem started with version 2.21.16

Additional Information/Context

No response

AWS Java SDK version used

2.21.16

JDK version used

17

Operating System and version

Windows 11

[debora-ito]

Hi @youngm I can't reproduce the error, I get "bogus" in the getObject response. I tried version 2.21.16 and the latest version 2.21.34, with a vanilla s3 sync client.

Questions:

• How are you creating the S3Client, can you share a sample code?
• Can you enable the verbose wirelogs? They will show the exact value of content-encoding that is sent in the putObject request - instructions for enabling the wirelogs are here. These are the verbose wirelogs of my local tests:

PutObject

>> "PUT /file.txt HTTP/1.1[\r][\n]"
>> "Host: mybucket.s3.ap-south-1.amazonaws.com[\r][\n]"
>> "amz-sdk-invocation-id: 53c49fef-9f1c-7e08-a9e3-97a7448d8da0[\r][\n]"
>> "amz-sdk-request: attempt=1; max=4[\r][\n]"
>> "Authorization: AWS4-HMAC-SHA256 Credential=xxx/20231130/ap-south-1/s3/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-request;content-encoding;content-length;content-type;host;x-amz-content-sha256;x-amz-date;x-amz-security-token, Signature=xxx[\r][\n]"
>> "Content-Encoding: bogus[\r][\n]"
>> "Content-Type: application/octet-stream[\r][\n]"
>> "Expect: 100-continue[\r][\n]"
>> "User-Agent: aws-sdk-java/2.21.16 xxx
>> "x-amz-content-sha256: UNSIGNED-PAYLOAD[\r][\n]"
>> "X-Amz-Date: 20231130T005850Z[\r][\n]"
>> "Content-Length: 14[\r][\n]"
>> "Connection: Keep-Alive[\r][\n]"
>> "[\r][\n]"
<< "HTTP/1.1 100 Continue[\r][\n]"
<< "[\r][\n]"
>> "Hello S3 World"
<< "HTTP/1.1 200 OK[\r][\n]"
<< "x-amz-id-2: xxx
<< "x-amz-request-id: EXWR84KGYSF5VYZD[\r][\n]"
<< "Date: Thu, 30 Nov 2023 00:58:52 GMT[\r][\n]"
<< "x-amz-server-side-encryption: AES256[\r][\n]"
<< "ETag: "9b5f10434fd4f06c98a5b070ac781771"[\r][\n]"
<< "Server: AmazonS3[\r][\n]"
<< "Content-Length: 0[\r][\n]"
<< "[\r][\n]"

See "Content-Encoding: bogus" is one of the request headers.

[youngm]

Oh maybe it is a problem with localstack and the PR I mentioned. I'll double check with real s3 tomorrow. Thanks!

[youngm]

Yeah, I confirmed this works great with real S3 this is a bug with localstack and sra auth for s3. I'll submit an issue there. Thanks!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[youngm]

I created an issue with open stack. They determined that the issue happens anytime you use http and not https as the endpoint and use url-connection-client. localstack/localstack#9782 I'll start using https with localstack. But, I thought I'd reopen this issue in case the SDK team thinks this might be an issue they want to fix.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.