Exception when using ARN of the bucket

[rmunna]

I am trying to create s3client which works for cross regions by enabling useArnRegionEnabled flag and by passing arn of the bucket as suggested here . S3Client is created with region us-west-1 and the bucket is in us-east-2.

S3Client s3client = S3Client.builder().region(Region.US_WEST_1)
        .serviceConfiguration(s-> s.useArnRegionEnabled(true).build())
        .credentialsProvider(StaticCredentialsProvider.create(credentials)).build();
    String location = s3client.getBucketLocation(b-> b.bucket("arn:aws:s3:::mybucket")
        .build()).locationConstraintAsString();
    System.out.println(location);

But the sdk is throwing the following error

Exception in thread "main" java.lang.IllegalArgumentException: Unknown ARN type
	at software.amazon.awssdk.services.s3.internal.resource.S3ArnConverter.lambda$convertArn$0(S3ArnConverter.java:61)
	at java.base/java.util.Optional.orElseThrow(Optional.java:408)
	at software.amazon.awssdk.services.s3.internal.resource.S3ArnConverter.convertArn(S3ArnConverter.java:61)
	at software.amazon.awssdk.services.s3.internal.endpoints.S3AccessPointEndpointResolver.applyEndpointConfiguration(S3AccessPointEndpointResolver.java:61)
	at software.amazon.awssdk.services.s3.internal.handlers.EndpointAddressInterceptor.modifyHttpRequest(EndpointAddressInterceptor.java:53)
	at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.modifyHttpRequestAndHttpContent(ExecutionInterceptorChain.java:90)
	at software.amazon.awssdk.core.internal.handler.BaseClientHandler.runModifyHttpRequestAndHttpContentInterceptors(BaseClientHandler.java:163)
	at software.amazon.awssdk.core.internal.handler.BaseClientHandler.finalizeSdkHttpFullRequest(BaseClientHandler.java:88)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:143)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:112)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:167)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:94)
	at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
	at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
	at software.amazon.awssdk.services.s3.DefaultS3Client.getBucketLocation(DefaultS3Client.java:3170)
	at software.amazon.awssdk.services.s3.S3Client.getBucketLocation(S3Client.java:5007)

Bucket ARN is copied from the S3 console

Your Environment

• AWS Java SDK version used: 2.5.15
• JDK version used: JDK 11
• Operating System and version: Windows 10

[BLasan]

@zoewangg Is this a valid issue?

[ryanthon]

I am having the same exact issue. Any updates on this?

[markjschreiber]

Agree, this is still an issue in SDK 2.16.34

[debora-ito]

Hi everyone, I'm sorry for the delay in response.

The scope of the useArnRegionEnabled flag is for using Access Point ARNs, as mentioned in the S3Configuration docs:

public boolean useArnRegionEnabled()

Returns whether the client is allowed to make cross-region calls when an S3 Access Point ARN has a different region to the one configured on the client.

That's why you see an error when using a standard bucket ARN. Access Point ARNs use the format arn:aws:s3:<region>:<account-id>:accesspoint:<resource>.

In addition, Access Point ARNs are not compatible with all S3 operations, see the list of the supported operations here:
https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-points-usage-examples.html

All that said, @rmunna: I've tested your sample code without the useArnRegionEnabled flag and it runs without issues if I use the bucket name:

S3Client s3client = S3Client.builder()
                            .region(Region.US_WEST_1)
                            .build();

String location = s3client.getBucketLocation(b-> b.bucket(BUCKET_NAME).build())
                          .locationConstraintAsString();

System.out.println(location); //prints us-east-2

It doesn't work when I use arn:aws:s3:::mybucket ARN though, I'll investigate why not.

[debora-ito]

So I responded my own question in my previous comment, ARN inputs are only supported when using S3 Access Points, regular bucket ARNs are not supported.

We'll work to make the error message more clear in this case.

[dhivakarcs]

Hi @debora-ito
S3 client is created with region "us-east-1" but bucket exists in "ap-southeast-1".

Below code

S3Client s3Client = S3Client.builder().region(Region.US_EAST_1) .credentialsProvider(StaticCredentialsProvider.create(AccessDetails.getV2AwsCredentials())).build();
String bucketRegion = s3Client.getBucketLocation(r -> r.bucket(BUCKET_NAME).build()) .locationConstraintAsString();
System.out.println(bucketRegion);

throw a 400 error software.amazon.awssdk.services.s3.model.S3Exception: The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'ap-southeast-1' (Service: S3, Status Code: 400)

Am I missing anything here?

BR
Dhiva

[ispringer]

Hi @dhivakarcs,

We also hit this issue and were finally able to fix it using a suggestion I found in another github issue. If you specify both region and endpoint override when building the client and set them to consistent values, the getBucketLocation call will work. Intuitive right?

Sample code:

private static Region getRegionFromBucketName(String s3bucketName) {
    // Region and endpoint override must both be specified and must be consistent, otherwise the getBucketLocation call will fail!
    S3Client s3Client = S3Client.builder()
            .region(Region.of(Region.US_EAST_1))
            .endpointOverride(URI.create("https://s3.us-east-1.amazonaws.com"))
            .build();
    GetBucketLocationRequest request = GetBucketLocationRequest.builder()
            .bucket(s3bucketName)
            .build();
    GetBucketLocationResponse response = s3Client.getBucketLocation(request);
    String region = response.locationConstraintAsString();
    if (region == null) {
        throw new RuntimeException("Could not determine region for bucket [" + s3BucketName + "].");
    }
    return Region.of(region);
}