DoesS3BucketExistV2Async throws an exception #2617

MaksimSimpleCode · 2023-05-26T12:46:14Z

MaksimSimpleCode
May 26, 2023

Hi, could you please tell me why checking for the existence of a cart gives an exception?
In my code there is the following check:
var found = await AmazonS3Util.DoesS3BucketExistV2Async(client, settings.BucketName);
If the bucket exists it returns true and then I can do CRUD operations, if the bucket doesn't exist
"error making request with error code forbidden and http status code forbidden. no further error information was returned by the service."

What can be the problem?

Answered by ashishdhingra

Jun 1, 2023

Hi @MaksimSimpleCode,

Good morning.

Per AmazonS3Util.DoesS3BucketExistV2Async(), False is returned in case S3 responds with a NoSuchBucket error. True is returned in case of success, AccessDenied error or PermanentRedirect error. An exception is thrown in case of any other error.. If you refer code for this method, it makes AmazonS3Client.GetACLAsync() call to check the existence of the bucket.

I tried with the below code (which enables verbose logging):

using (AmazonS3Client client = new AmazonS3Client(RegionEndpoint.USWest2))
{
      AWSConfigs.LoggingConfig.LogTo = LoggingOptions.Console;
      AWSConfigs.LoggingConfig.LogResponses = ResponseLoggingOption.Always;
      AWSConfigs.Loggi…

View full answer

ashishdhingra · 2023-06-01T18:57:06Z

ashishdhingra
Jun 1, 2023
Maintainer

Hi @MaksimSimpleCode,

Good morning.

Per AmazonS3Util.DoesS3BucketExistV2Async(), False is returned in case S3 responds with a NoSuchBucket error. True is returned in case of success, AccessDenied error or PermanentRedirect error. An exception is thrown in case of any other error.. If you refer code for this method, it makes AmazonS3Client.GetACLAsync() call to check the existence of the bucket.

I tried with the below code (which enables verbose logging):

using (AmazonS3Client client = new AmazonS3Client(RegionEndpoint.USWest2))
{
      AWSConfigs.LoggingConfig.LogTo = LoggingOptions.Console;
      AWSConfigs.LoggingConfig.LogResponses = ResponseLoggingOption.Always;
      AWSConfigs.LoggingConfig.LogMetrics = true;
      var exists = AmazonS3Util.DoesS3BucketExistV2Async(client, "<<bucket-name>>").Result;
}

The specified bucket in some other AWS account was in us-west-2 region. It logged the below response log and returned true (in exists variable).

ellExecute - False RedirectStandardError - True, RedirectStandardOutput - True, CreateNoWindow - True
AWSSDKUtils 2|2023-06-01T11:45:56.627Z|INFO|Process started
ProcessAWSCredentials 3|2023-06-01T11:45:58.784Z|INFO|Process ends with exitcode - 0
AmazonS3Client 4|2023-06-01T11:45:59.316Z|ERROR|An exception of type HttpErrorResponseExceptionwas handled in ErrorHandler. --> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
AmazonS3Client 5|2023-06-01T11:45:59.349Z|ERROR|Received error response: [<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId><<REDACTED>></RequestId><HostId><<REDACTED>></HostId></Error>] --> Amazon.S3.AmazonS3Exception: Access Denied
 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
AmazonS3Client 6|2023-06-01T11:45:59.355Z|ERROR|AmazonS3Exception making request GetACLRequest to https://<<REDACTED>>.s3.us-west-2.amazonaws.com/. Attempt 1. --> Amazon.S3.AmazonS3Exception: Access Denied
 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken)
   at Amazon.Runtime.Internal.HttpHandler`1.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception,Stream responseStream)
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ExceptionHandler`1.HandleAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)

If the bucket is in a different region than the one set at AmazonS3Client level it logged the below response log and returned true (in exists variable)

AWSSDKUtils 1|2023-06-01T11:57:50.296Z|INFO|Starting a process with the following ProcessInfo: UseShellExecute - False RedirectStandardError - True, RedirectStandardOutput - True, CreateNoWindow - True
AWSSDKUtils 2|2023-06-01T11:57:50.350Z|INFO|Process started
ProcessAWSCredentials 3|2023-06-01T11:57:52.691Z|INFO|Process ends with exitcode - 0
AmazonS3Client 4|2023-06-01T11:57:53.292Z|ERROR|An exception of type HttpErrorResponseExceptionwas handled in ErrorHandler. --> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.Internal.RedirectHandler.HandleRedirect(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
AmazonS3Client 5|2023-06-01T11:57:53.330Z|ERROR|Received error response: [<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>PermanentRedirect</Code><Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message><Endpoint>aws4ps-dev-jonatnun-cfn.s3-us-west-2.amazonaws.com</Endpoint><Bucket><<REDACTED>></Bucket><RequestId><<REDACTED>></RequestId><HostId><<REDACTED>></HostId></Error>] --> Amazon.S3.AmazonS3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.Internal.RedirectHandler.HandleRedirect(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
AmazonS3Client 6|2023-06-01T11:57:53.336Z|ERROR|AmazonS3Exception making request GetACLRequest to https://<<REDACTED>>.us-east-2.amazonaws.com/. Attempt 1. --> Amazon.S3.AmazonS3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
 ---> Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
   at Amazon.Runtime.Internal.RedirectHandler.HandleRedirect(IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RedirectHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.S3.Internal.AmazonS3ResponseHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   --- End of inner exception stack trace ---
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionStream(IRequestContext requestContext, IWebResponseData httpErrorResponse, HttpErrorResponseException exception,Stream responseStream)
   at Amazon.Runtime.Internal.HttpErrorResponseExceptionHandler.HandleExceptionAsync(IExecutionContext executionContext, HttpErrorResponseException exception)
   at Amazon.Runtime.Internal.ExceptionHandler`1.HandleAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.ProcessExceptionAsync(IExecutionContext executionContext, Exception exception)
   at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.Signer.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)
   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)

So in both cases, the S3 service returned AccessDenied or PermanentRedirect error code, which AmazonS3Util.DoesS3BucketExistV2Async() handles correctly and returns true.

You may try enabling verbose logging and check what error is returned by S3 for your use case.

Thanks,
Ashish

0 replies

2023-12-01T20:10:32Z

github-actions[bot]
bot Dec 1, 2023

Hello! Reopening this discussion to make it searchable.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DoesS3BucketExistV2Async throws an exception #2617

{{title}}

Replies: 2 comments

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

DoesS3BucketExistV2Async throws an exception #2617

MaksimSimpleCode May 26, 2023

Replies: 2 comments

ashishdhingra Jun 1, 2023 Maintainer

github-actions[bot] bot Dec 1, 2023

MaksimSimpleCode
May 26, 2023

ashishdhingra
Jun 1, 2023
Maintainer

github-actions[bot]
bot Dec 1, 2023