Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase default max retries and expose environment variable to override #830

Merged
merged 2 commits into from
Apr 2, 2024
Merged

Increase default max retries and expose environment variable to override #830

merged 2 commits into from
Apr 2, 2024

Commits on Apr 1, 2024

  1. Increase default max retries and expose environment variable to override 

    We were using the SDK's default retry configuration (actually, slightly
wrong -- it's supposed to be 3 total attempts, but we configured 3
*retries*, so 4 attempts). This isn't a good default for file systems,
as it works out to only retrying for about 2 seconds before giving up,
and applications are rarely equipped to gracefully handle transient
errors.

This change increases the default to 10 total attempts, which takes
about a minute on average. This is in the same ballpark as NFS's
defaults (3 attempts, 60 seconds linear backoff), though still a little
more aggressive. There's probably scope to go even further (20?), but
this is a reasonable step for now.

To allow customers to further tweak this, the S3CrtClient now respects
the `AWS_MAX_ATTEMPTS` environment variable, and its value overrides the
defaults. This is only a partial solution, as SDKs are supposed to also
respect the `max_attempts` config file setting, but we don't have any of
the infrastructure for that today (similar issue as awslabs#389).

Signed-off-by: James Bornholt <bornholt@amazon.com>
    @jamesbornholt
    jamesbornholt committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    b8409f8 View commit details
    Browse the repository at this point in the history

  2. Surprised Clippy doesn't yell about this 

    Signed-off-by: James Bornholt <bornholt@amazon.com>
    @jamesbornholt
    jamesbornholt committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    2186683 View commit details
    Browse the repository at this point in the history