Socket timeout while reading the data from s3 bucket after upgrading of aws sdk 2.x

[sandeepsinnha2109]

Describe the bug

While downloading the data from s3 bucket facing the issue that is Socket time out exception.
This was happening after the upgrade of aws sdk2.x

Expected Behavior

File should be downloaded from s3 bucket.

Current Behavior

-Sockettimeout on reads from s3 bucket.
We are seeing out of memory for heap space.

Reproduction Steps

Download the file from s3 bucket.

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.x

JDK version used

17

Operating System and version

Kubernetes Cluster

[debora-ito]

@sandeepsinnha2109 please provide a reproducible code and the full stacktrace of the error.
Also provide the full SDK version you are using - for example, 2.20.47.

Socket timeout errors indicate the operation timed out before if finished, and it can have several causes like latency in the network connectivity, service side latency, etc. If it's something you see constantly, consider increasing the relevant timeouts in the client configuration of the HTTP client you are using.

[sandeepsinnha2109]

Hi, we are getting issues related to JDBC connection error and java.net.SocketException: Socket is closed, when our on-prem application is ingesting data from AWS application APIs to download files/data, which is in RDS and S3.

[sandeepsinnha2109]

DEBUG org.springframework.web.client.RestTemplate - Response 500 INTERNAL_SERVER_ERROR
13:50:22.736 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection [id: 0][route: {s}->URL of the application] can be kept alive indefinitely
13:50:22.736 [main] DEBUG org.apache.http.impl.conn.PoolingHttpClientConnectionManager - Connection released: [id: 0][route: {s}

[sandeepsinnha2109]

@debora-ito please help me on above response

[debora-ito]

It looks like your RDS is not reachable, this is likely a network configuration issue (although "500 INTERNAL_SERVER_ERROR" indicates a server side issue, but with the log snippet that was provided I can't tell which server you tried to connect to).

You can try the troubleshooting steps listed here - https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Troubleshooting.html

There isn't enough information here to help you with more specific guidance. If you are using the Java SDK to connect to RDS or S3, please share a self-contained sample code we can use to reproduce the error. Also provide full logs and stacktraces with the error.

Keep in mind that this channel is intended to talk about SDK issues, not aws issues in general. We may lack the expertise to help with the latter.

[sandeepsinnha2109]

@debora-ito
Please refer the below response
JDK1.8
aws-crt 0.21.16
SDK-2.20.68
S3AsyncClient.crtBuilder()
.credentialsProvider(DefaultCredentialsProvider.create())
.region(Region.of("Ireland"))
.targetThroughputInGbps(1.0)
.minimumPartSizeInBytes(8 * MB)
.build();
S3TransferManager s3tm =
S3TransferManager.builder()
.s3Client(s3AsyncClient)
.build();

SDK-1.12.261
AmazonS3ClientBuilder.standard()
.withRegion(s3Configurer.getRegion())
.build();

Download code:
GetObjectRequest getObjectRequest = GetObjectRequest.builder().bucket(bucketName).key(key).build();
CompletableFuture<ResponseInputStream> responseFuture =
s3AsyncClient.getObject( getObjectRequest, AsyncResponseTransformer.toBlockingInputStream());
try(ResponseInputStream responseInputStream = responseFuture.join()) {
InputStreamResource streamResource = new InputStreamResource(responseInputStream);
FileInfo fileInfo = Optional.of(FileInfo.builder()
.resource(streamResource)
.name(key)
.schemaName("ID")
.size(getSize())
.created(getCreated())
.build());
}

Earlier JDBC connection issue we suspect was due to the below exception which is blocking database connection until file downloads.
API call triggers from consumer application, file download starts but in a couple of minutes fails for below exception
i.e. java.lang.OutOfMemory just single line. Memory is not the issue as EKS pod is running reached only 65% to 70% memory
Please advise.
Note: SDK 1 object is for s3select operation.

[debora-ito]

Can you share the full stacktrace?

[sandeepsinnha2109]

Hi @debora-ito ,
Please refer the below stack trace for jdbc exception
Caused by: org.springframework.web.client.HttpServerErrorException$InternalServerError:
500 : [class
org. springframework. dao. DataAccessResourceFailureException-Unable to acquire JDBC Connection; nested exception is
234
org. hibernate.exception.JDBCConnectionException: Unable to acquire JDBC Connection: Unable to acquire JDBC Connection]
at org. springframework. web.client. HttpServerErrorException. create (HttpServerErrorException. java: 100) at org. springframework.web.client. DefaultResponseErrorHandler.handleError (DefaultResponseErrorHandler. java: 188)
at erg. springframework. web.client. DefaultResponseErrorHandler.handleError (DefaultResponseErrorHandler. java: 125) at org. springframework. web.client.ResponseErrorHandler.handleError (ResponseErrorHandler. java: 63) at org. springframework. web. client. RestTemplate handleResponse (RestTemplate. java:819) at org. springframework. web. client. RestTemplate.doExecute (RestTemplate. java: 777) at org.springframework. web.client. RestTemplate. execute (RestTemplate. java: 711)
at org. springframework. web. client. RestTemplate. exchange (RestTemplate. java: 602)

[debora-ito]

The error is not coming from the Java SDK, there's no Java SDK class in the stacktrace.

"Unable to acquire JDBC Connection" indicates a network connectivity issue. Check for network configurations that would prevent the database to be reachable.

I'll go ahead and close this, as there's no indication the error is caused by the Java SDK.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.