software.amazon.awssdk.core.exception.NonRetryableException: A retry was attempted, but AsyncRequestBody.split does not support retries. #5293

brycechesternewman · 2024-06-13T15:33:57Z

brycechesternewman
Jun 13, 2024

Hi,
What is the cause of software.amazon.awssdk.core.exception.NonRetryableException: A retry was attempted, but AsyncRequestBody.split does not support retries." and how to prevent it? We don't retry in our code so it would appear something in the AWS code is retrying something it should not?

AWS SDK version is: 2.25.56

Here is the client that is used.

...
 S3AsyncClient.builder()
          .credentialsProvider(creds)
          .endpointOverride(conf.s3.endpoint)
          .multipartEnabled(true)
          .multipartConfiguration(
            MultipartConfiguration.builder()
              .minimumPartSizeInBytes(conf.s3MultiPartSize)
              .apiCallBufferSizeInBytes(conf.s3MultiPartSize)
              .build()
          )
          .serviceConfiguration(
            S3Configuration.builder()
              .checksumValidationEnabled(true)
              .build()
          )
          .httpClient(
            new DefaultSdkAsyncHttpClientBuilder()
              .buildWithDefaults(
                AttributeMap.builder()
                  .put[java.lang.Integer](SdkHttpConfigurationOption.MAX_CONNECTIONS, conf.s3MaxConnections)
                  .build()
              )
          )
          .build()
...

Here is the put call.

...
client.putObject(put, AsyncRequestBody.fromInputStream(bytes, uploadMetadata.length, blocking.asExecutionContextExecutorService))
...

brycechesternewman · 2024-06-13T15:59:17Z

brycechesternewman
Jun 13, 2024
Author

For completeness, here is the exception being thrown.

 cause:  Exception in thread "zio-fiber-" software.amazon.awssdk.core.exception.NonRetryableException: A retry was attempted, but AsyncRequestBody.split does not support retries.
	at software.amazon.awssdk.core.exception.NonRetryableException$BuilderImpl.build(NonRetryableException.java:109)
	at software.amazon.awssdk.core.exception.NonRetryableException.create(NonRetryableException.java:49)
	at software.amazon.awssdk.core.internal.async.SplittingPublisher$SplittingSubscriber$DownstreamBody.subscribe(SplittingPublisher.java:296)
	at software.amazon.awssdk.core.internal.async.ChecksumCalculatingAsyncRequestBody.lambda$alwaysInvokeOnNext$2(ChecksumCalculatingAsyncRequestBody.java:160)
	at software.amazon.awssdk.core.async.SdkPublisher.lambda$map$1(SdkPublisher.java:86)
	at software.amazon.awssdk.core.async.SdkPublisher.lambda$flatMapIterable$2(SdkPublisher.java:97)
	at software.amazon.awssdk.core.internal.async.ChecksumCalculatingAsyncRequestBody.subscribe(ChecksumCalculatingAsyncRequestBody.java:156)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage$SdkHttpContentPublisherAdapter.subscribe(MakeAsyncHttpRequestStage.java:279)
	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor$StreamedRequest.subscribe(NettyRequestExecutor.java:431)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.subscribeSubscriberToStream(HttpStreamsHandler.java:139)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsClientHandler.channelRead(HttpStreamsClientHandler.java:150)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1338)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1387)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
	at software.amazon.awssdk.http.nio.netty.internal.OneTimeReadTimeoutHandler.channelRead(OneTimeReadTimeoutHandler.java:37)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.base/java.lang.Thread.run(Thread.java:833)

2 replies

brycechesternewman Jun 13, 2024
Author

A different exception is also thrown. We don't retry in our code so it would appear something in the AWS code is retrying something it should not?

cause:  Exception in thread "zio-fiber-" software.amazon.awssdk.core.exception.NonRetryableException: Request cannot be retried, because the request stream could not be reset.
	at software.amazon.awssdk.core.exception.NonRetryableException$BuilderImpl.build(NonRetryableException.java:109)
	at software.amazon.awssdk.core.exception.NonRetryableException.create(NonRetryableException.java:53)
	at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.tryReset(InputStreamWithExecutorAsyncRequestBody.java:94)
	at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.subscribe(InputStreamWithExecutorAsyncRequestBody.java:76)
	at software.amazon.awssdk.core.internal.async.ChecksumCalculatingAsyncRequestBody.lambda$alwaysInvokeOnNext$2(ChecksumCalculatingAsyncRequestBody.java:160)
	at software.amazon.awssdk.core.async.SdkPublisher.lambda$map$1(SdkPublisher.java:98)
	at software.amazon.awssdk.core.async.SdkPublisher.lambda$flatMapIterable$2(SdkPublisher.java:109)
	at software.amazon.awssdk.core.internal.async.ChecksumCalculatingAsyncRequestBody.subscribe(ChecksumCalculatingAsyncRequestBody.java:156)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage$SdkHttpContentPublisherAdapter.subscribe(MakeAsyncHttpRequestStage.java:279)
	at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor$StreamedRequest.subscribe(NettyRequestExecutor.java:431)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.subscribeSubscriberToStream(HttpStreamsHandler.java:139)
	at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsClientHandler.channelRead(HttpStreamsClientHandler.java:150)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1338)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1387)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
	at software.amazon.awssdk.http.nio.netty.internal.OneTimeReadTimeoutHandler.channelRead(OneTimeReadTimeoutHandler.java:37)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at java.base/java.lang.Thread.run(Thread.java:1583)
	Suppressed: java.io.IOException: mark/reset not supported
		at java.base/java.io.InputStream.reset(InputStream.java:741)
		at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.tryReset(InputStreamWithExecutorAsyncRequestBody.java:91)
		at software.amazon.awssdk.core.internal.async.InputStreamWithExecutorAsyncRequestBody.subscribe(InputStreamWithExecutorAsyncRequestBody.java:76)
		at software.amazon.awssdk.core.internal.async.ChecksumCalculatingAsyncRequestBody.lambda$alwaysInvokeOnNext$2(ChecksumCalculatingAsyncRequestBody.java:160)
		at software.amazon.awssdk.core.async.SdkPublisher.lambda$map$1(SdkPublisher.java:98)
		at software.amazon.awssdk.core.async.SdkPublisher.lambda$flatMapIterable$2(SdkPublisher.java:109)
		at software.amazon.awssdk.core.internal.async.ChecksumCalculatingAsyncRequestBody.subscribe(ChecksumCalculatingAsyncRequestBody.java:156)
		at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage$SdkHttpContentPublisherAdapter.subscribe(MakeAsyncHttpRequestStage.java:279)
		at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor$StreamedRequest.subscribe(NettyRequestExecutor.java:431)
		at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsHandler.subscribeSubscriberToStream(HttpStreamsHandler.java:139)
		at software.amazon.awssdk.http.nio.netty.internal.nrs.HttpStreamsClientHandler.channelRead(HttpStreamsClientHandler.java:150)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
		at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
		at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
		at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
		at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
		at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
		at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
		at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
		at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
		at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
		at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
		at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
		at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1338)
		at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1387)
		at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
		at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
		at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
		at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
		at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:289)
		at software.amazon.awssdk.http.nio.netty.internal.OneTimeReadTimeoutHandler.channelRead(OneTimeReadTimeoutHandler.java:37)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
		at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
		at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
		at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
		at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
		at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
		at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
		at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
		at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
		at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
		at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
		at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
		at java.base/java.lang.Thread.run(Thread.java:1583)

T-TRz879 Jun 29, 2024

I have also encountered such errors, but there is no official explanation. In the end it went nowhere

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

software.amazon.awssdk.core.exception.NonRetryableException: A retry was attempted, but AsyncRequestBody.split does not support retries. #5293

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment 2 replies

{{title}}

{{title}}

{{title}}

Select a reply

software.amazon.awssdk.core.exception.NonRetryableException: A retry was attempted, but AsyncRequestBody.split does not support retries. #5293

brycechesternewman Jun 13, 2024

Replies: 1 comment · 2 replies

brycechesternewman Jun 13, 2024 Author

brycechesternewman Jun 13, 2024 Author

T-TRz879 Jun 29, 2024

brycechesternewman
Jun 13, 2024

Replies: 1 comment 2 replies

brycechesternewman
Jun 13, 2024
Author

brycechesternewman Jun 13, 2024
Author