NPE on PutObject with Checksum

[william00179]

Describe the bug

When uploading an object and requesting checksum generation, the following exception will be thrown

java.lang.NullPointerException: Cannot read field "value" because "s1" is null
  at java.base/java.lang.String$CaseInsensitiveComparator.compare(String.java:2077) ~[na:na]
  at java.base/java.lang.String$CaseInsensitiveComparator.compare(String.java:2070) ~[na:na]
  at java.base/java.util.TreeMap.getEntryUsingComparator(TreeMap.java:409) ~[na:na]
  at java.base/java.util.TreeMap.getEntry(TreeMap.java:379) ~[na:na]
  at java.base/java.util.TreeMap.get(TreeMap.java:290) ~[na:na]
  at software.amazon.awssdk.http.DefaultSdkHttpFullRequest.firstMatchingHeader(DefaultSdkHttpFullRequest.java:130) ~[http-client-spi-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.util.HttpChecksumUtils.isHttpChecksumPresent(HttpChecksumUtils.java:153) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.HttpChecksumStage.flexibleChecksumInHeaderRequired(HttpChecksumStage.java:220) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.HttpChecksumStage.execute(HttpChecksumStage.java:83) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.HttpChecksumStage.execute(HttpChecksumStage.java:54) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:222) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45) ~[sdk-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53) ~[aws-core-2.21.0.jar!/:na]
  at software.amazon.awssdk.services.s3.DefaultS3Client.putObject(DefaultS3Client.java:8975) ~[s3-2.21.0.jar!/:na]

from code

s3Client.putObject(PutObjectRequest.builder()
                        .key(key)
                        .bucket(bucket)
                        .storageClass(storageClass)
                        .tagging(getTagging())
                        .contentLength((long) buffer.remaining())
                        .contentEncoding(contentEncoding)
                        .contentMD5(md5Digest(buffer))
                        .checksumAlgorithm(ChecksumAlgorithm.CRC32_C)
                        .build(), RequestBody.fromByteBuffer(buffer));

Expected Behavior

The file should upload without exception and return the CRC32C checksum

Current Behavior

An exception is thrown and the upload fails

Reproduction Steps

Attempt a Put Object request with checksum algo CRC32C.

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.21.2

JDK version used

openjdk 21 2023-09-19 LTS OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS) OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

Operating System and version

Windows 11

[felixb]

I've the same problem. Most requests work fine, but some are broken.

[bhoradc]

Hello @william00179,

Thanks for reporting the issue. Can you please provide more details on s1?
Cannot read field "value" because "s1" is null

It will be helpful if you can provide a self-contained code snippet that I can use to reproduce the issue.

Also, kindly confirm if the Java SDK version you are using is 2.21.0 (per your logs) or 2.21.2 (as mentioned in AWS Java SDK version used section).

Regards,
Chaitanya

[debora-ito]

@william00179 @felixb we can reproduce the issue, we are actively investigating.

Edit: actually we can't reproduce the error. @felixb Is there any pattern you can identify on the requests that fail?

[sugmanue]

@william00179 @felixb we are actively working on this, while we work on this if you're blocked please revert back to using 2.20.162. We will update this issue when we root cause and find the correct fix.

[dagnir]

Hi, we have reverted the changes causing this issue in 2.21.6 2.21.5 (via #4621).

[debora-ito]

The issue is now fixed, so I'll go ahead and close this.

If you find an issue after upgrading to SDK version 2.21.5, please open a new Github issue.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.