Skip to content

Commit

Permalink
Add support for SDK's cross-region S3 client. (#960)
Browse files Browse the repository at this point in the history
Fixes #848
  • Loading branch information
maciejwalkowiak committed Dec 10, 2023
1 parent 415ba37 commit 1c96101
Show file tree
Hide file tree
Showing 5 changed files with 24 additions and 30 deletions.
30 changes: 0 additions & 30 deletions docs/src/main/asciidoc/s3.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -74,36 +74,6 @@ Transfer Manager works the best with CRT S3 Client. To auto-configure CRT based

When no `S3AsyncClient` bean is created, the default `S3AsyncClient` created through AWS SDK is used. To benefit from maximum throughput, multipart upload/download and resumable file upload consider using CRT based `S3AsyncClient`.

=== Using Cross-Region S3 client

`S3Client` implementation provided in AWS SDK is region specific - meaning it can be used only to operate on buckets and objects stored in region for which the client has been configured to use.

For example, assuming that `DefaultAwsRegionProviderChain` resolves a region `us-east-1`, running any S3 operation that uses a bucket created in another region would result in an exception:

[source]
----
software.amazon.awssdk.services.s3.model.S3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. (Service: S3, Status Code: 301, Request ID: ..., Extended Request ID: ...)
----

Spring Cloud AWS provides a `CrossRegionS3Client` that solves this problem by maintaining an internal dictionary of `S3Client` objects per region. If you need to customize these clients, you can create a custom `S3ClientBuilder` bean that acts as a template to create region-specific S3 clients in `CrossRegionS3Client`.
There is no extra work needed to use cross-region S3 client - it is the `S3Client` auto-configured by Spring Cloud AWS.

To disable `CrossRegionS3Client` creation and use instead a regular region-specific `S3Client`, you can either create a custom `S3Client` bean, or exclude `spring-cloud-aws-s3-cross-region-client` dependency:

[source,xml]
----
<dependency>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-starter-s3</artifactId>
<exclusions>
<exclusion>
<groupId>io.awspring.cloud</groupId>
<artifactId>spring-cloud-aws-s3-cross-region-client</artifactId>
</exclusion>
</exclusions>
</dependency>
----

=== S3 Objects as Spring Resources

https://docs.spring.io/spring/docs/current/spring-framework-reference/html/resources.html[Spring Resources] are an abstraction for a number of low-level resources, such as file system files, classpath files, servlet context-relative files, etc.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,9 @@ S3ClientBuilder s3ClientBuilder(AwsClientBuilderConfigurer awsClientBuilderConfi
ObjectProvider<AwsClientCustomizer<S3ClientBuilder>> configurer) {
S3ClientBuilder builder = awsClientBuilderConfigurer.configure(S3Client.builder(), this.properties,
configurer.getIfAvailable());

Optional.ofNullable(this.properties.getCrossRegionEnabled()).ifPresent(builder::crossRegionAccessEnabled);

builder.serviceConfiguration(this.properties.toS3Configuration());
return builder;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ S3AsyncClient s3AsyncClient(AwsCredentialsProvider credentialsProvider) {
.region(this.awsClientBuilderConfigurer.resolveRegion(this.properties));
Optional.ofNullable(this.awsProperties.getEndpoint()).ifPresent(builder::endpointOverride);
Optional.ofNullable(this.properties.getEndpoint()).ifPresent(builder::endpointOverride);
Optional.ofNullable(this.properties.getCrossRegionEnabled()).ifPresent(builder::crossRegionAccessEnabled);

if (this.properties.getCrt() != null) {
S3CrtClientProperties crt = this.properties.getCrt();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,12 @@ public class S3Properties extends AwsClientProperties {
@Nullable
private Boolean useArnRegionEnabled;

/**
* Enables cross-region bucket access.
*/
@Nullable
private Boolean crossRegionEnabled;

/**
* Configuration properties for {@link S3TransferManager} integration.
*/
Expand Down Expand Up @@ -132,6 +138,15 @@ public void setUseArnRegionEnabled(@Nullable Boolean useArnRegionEnabled) {
this.useArnRegionEnabled = useArnRegionEnabled;
}

@Nullable
public Boolean getCrossRegionEnabled() {
return crossRegionEnabled;
}

public void setCrossRegionEnabled(@Nullable Boolean crossRegionEnabled) {
this.crossRegionEnabled = crossRegionEnabled;
}

@Nullable
public S3TransferManagerProperties getTransferManager() {
return this.transferManager;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@
import software.amazon.awssdk.services.s3.waiters.S3Waiter;
import software.amazon.awssdk.utils.SdkAutoCloseable;

/**
* @deprecated use cross-region client included in SDK: <a href=
* "https://aws.amazon.com/blogs/developer/introducing-s3-cross-region-support-in-the-aws-sdk-for-java-2-x/">...</a>
*/
@Deprecated
public class CrossRegionS3Client extends AbstractCrossRegionS3Client {

private static final Logger LOGGER = LoggerFactory.getLogger("io.awspring.cloud.s3.CrossRegionS3Client");
Expand Down

0 comments on commit 1c96101

Please sign in to comment.