Kinesis PutRecord doesn't return an InternalFailureException

[mpayne-coveo]

Describe the bug

When Kinesis returns an HTTP 500 (server-side error) on PutRecord, it returns something like this-

KinesisException: Service returned error code InternalFailure (Service: Kinesis, Status Code: 500, Request ID: xxx, Extended Request ID: xxxx)

According to the documentation, InternalFailureException extends KinesisException, so I expect it to be used to handle error codes of InternalFailure. These exceptions can be very noisy.

Full stack trace-

software.amazon.awssdk.services.kinesis.model.KinesisException: Service returned error code InternalFailure (Service: Kinesis, Status Code: 500, Request ID: xxx, Extended Request ID: xxxx)
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleErrorResponse(CombinedResponseHandler.java:125)
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handleResponse(CombinedResponseHandler.java:82)
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:60)
    at software.amazon.awssdk.core.internal.http.CombinedResponseHandler.handle(CombinedResponseHandler.java:41)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:50)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:38)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:72)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:55)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:39)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
    at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:50)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:32)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
    at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
    at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:224)
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:173)
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:80)
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
    at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:74)
    at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
    at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:53)
    at software.amazon.awssdk.services.kinesis.DefaultKinesisClient.putRecord(DefaultKinesisClient.java:2068)

Expected Behavior

InternalFailureException extends KinesisException, so I expect it to be used to handle error codes of InternalFailure.

Current Behavior

These HTTP 500s are being returned with KinesisException and an error code of InternalFailure, with a status code of 500.

Reproduction Steps

KinesisClient kinesisClient = KinesisClient.builder()
            .region(Region.US_EAST_1)
            .build();
PutRecordRequest request = PutRecordRequest.builder()
            .streamName("test")
            .data(SdkBytes.fromString("data"))
            .build();
try {
  kinesisClient.putRecord(request);
} catch (InternalFailureException e) {
  System.out.println("Never gets reached on InternalFailure");
}

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.21.42

JDK version used

Java17

Operating System and version

Amazon Linux 2

[debora-ito]

@mpayne-coveo For the Java SDK to throw a specific exception type, that exception needs to be modeled and associated with the operation.

Looking at the Kinesis model here:

aws-sdk-java-v2/services/kinesis/src/main/resources/codegen-resources/service-2.json

Line 411 in 21192c0

"PutRecord":{

InternalFailureException is not listed as a possible exception for PutRecord, that's why it is falling back to the generic base KinesisException.

I'll ask the Kinesis team to fix their model. Will comment here when they have an update.

[debora-ito]

An easier way to check the list of exceptions is to look at the method in Javadoc -

https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/kinesis/KinesisClient.html#putRecord(software.amazon.awssdk.services.kinesis.model.PutRecordRequest)

[debora-ito]

@mpayne-coveo the Kinesis team will consider updating their service model, and added a task in their backlog for it. They don't have a timeline to share at this moment.

I'll go ahead and close this, as there's no pending action from the Java SDK team.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.