s3 expires implementation #3232

alexgromero · 2024-08-06T14:31:48Z

This PR customizes the S3 Expires shape in preparation for model changes. Instead of throwing an error, operations will now complete with fallback behavior and a warning.

Ensures the Expires header remains a timestamp for backwards compatibility.
Omits the Expires key in the response if parsing fails and logs a warning.

Operations with Expires in their output shapes will:

Add a synthetic member called ExpiresString to retain the raw value of Expires.
Deprecate Expires and encourage users to use the new field ExpiresString through documentation.

codecov-commenter · 2024-08-06T14:41:47Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.17%. Comparing base (c68aa1a) to head (95797f3).
Report is 157 commits behind head on develop.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #3232      +/-   ##
===========================================
+ Coverage    93.12%   93.17%   +0.05%     
===========================================
  Files           66       66              
  Lines        14252    14331      +79     
===========================================
+ Hits         13272    13353      +81     
+ Misses         980      978       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

.changes/next-release/enhancement-s3-94466.json

botocore/handlers.py

tests/functional/test_s3.py

botocore/parsers.py

tests/unit/test_parsers.py

botocore/handlers.py

botocore/parsers.py

botocore/data/s3/2006-03-01/service-2.sdk-extras.json

tests/functional/test_s3.py

botocore/data/s3/2006-03-01/service-2.sdk-extras.json

tests/functional/test_s3.py

tests/unit/test_parsers.py

botocore/parsers.py

…ad of unittest

botocore/parsers.py

tests/functional/test_s3.py

botocore/parsers.py

tests/functional/test_s3.py

tests/unit/test_parsers.py

Co-authored-by: Nate Prewitt <nate.prewitt@gmail.com>

tests/functional/test_service_headers.py

jonathan343

LGTM!

botocore/parsers.py

tests/functional/test_service_headers.py

tests/unit/test_handlers.py

…test_s3.py

botocore/endpoint.py

tests/unit/test_endpoint.py

Co-authored-by: Nate Prewitt <nate.prewitt@gmail.com>

botocore/handlers.py

…shape

jonathan343

@SamRemis

received approvals from @SamRemis and @jonathan343

* release-1.35.16: Bumping version to 1.35.16 Update endpoints model Update to latest models remove outdated documentation page (#3249) s3 expires implementation (#3232) Update awcrt version to 0.21.5 (#3248)

* implementation to gracefully handle invalid timestamps returned in the Expires header

s3 expires implementation

20a8ffd

alexgromero requested a review from SamRemis August 6, 2024 14:31

adding a changelog and fixing files with pre-commit

61efb89

alexgromero force-pushed the s3-expires-timestamp branch from 8137d13 to dff343c Compare August 15, 2024 17:03

implement S3 specific parser to handle Expires shape

567c109

alexgromero force-pushed the s3-expires-timestamp branch from dff343c to 567c109 Compare August 19, 2024 14:12

Merge branch 'boto:develop' into s3-expires-timestamp

f928f61

SamRemis reviewed Aug 20, 2024

View reviewed changes

jonathan343 self-requested a review August 20, 2024 19:58

implement feedback from PR

6c35c17

jonathan343 reviewed Aug 20, 2024

View reviewed changes

update tests and implementation based on feedback

22be252

alexgromero requested review from jonathan343 and SamRemis August 21, 2024 16:20

SamRemis reviewed Aug 21, 2024

View reviewed changes

tests/functional/test_s3.py Outdated Show resolved Hide resolved

alexgromero added 3 commits August 22, 2024 10:09

remove redundant assert calls in tests

8e74d5b

update parser test with known invalid expires test cases

e2475e8

fix functional test

2ebdba4

nateprewitt reviewed Aug 22, 2024

View reviewed changes

botocore/data/s3/2006-03-01/service-2.sdk-extras.json Outdated Show resolved Hide resolved

tests/functional/test_s3.py Outdated Show resolved Hide resolved

tests/unit/test_parsers.py Outdated Show resolved Hide resolved

jonathan343 reviewed Aug 22, 2024

View reviewed changes

botocore/parsers.py Outdated Show resolved Hide resolved

updating parser implementation and changing tests to use pytest inste…

d1d67d5

…ad of unittest

alexgromero requested review from jonathan343, SamRemis and nateprewitt August 23, 2024 17:52

jonathan343 reviewed Aug 23, 2024

View reviewed changes

botocore/parsers.py Outdated Show resolved Hide resolved

botocore/parsers.py Outdated Show resolved Hide resolved

SamRemis reviewed Aug 23, 2024

View reviewed changes

tests/functional/test_s3.py Outdated Show resolved Hide resolved

nateprewitt reviewed Aug 23, 2024

View reviewed changes

botocore/parsers.py Outdated Show resolved Hide resolved

botocore/parsers.py Outdated Show resolved Hide resolved

tests/functional/test_s3.py Outdated Show resolved Hide resolved

nateprewitt reviewed Aug 23, 2024

View reviewed changes

tests/unit/test_parsers.py Outdated Show resolved Hide resolved

Update tests/functional/test_s3.py

5e80ad8

Co-authored-by: Nate Prewitt <nate.prewitt@gmail.com>

alexgromero requested a review from jonathan343 August 28, 2024 20:06

jonathan343 reviewed Aug 29, 2024

View reviewed changes

tests/functional/test_service_headers.py Outdated Show resolved Hide resolved

update test assertion to be more descriptive

5741723

alexgromero requested a review from jonathan343 August 29, 2024 14:49

SamRemis approved these changes Aug 29, 2024

View reviewed changes

jonathan343 approved these changes Sep 4, 2024

View reviewed changes

nateprewitt previously requested changes Sep 4, 2024

View reviewed changes

botocore/parsers.py Outdated Show resolved Hide resolved

botocore/parsers.py Outdated Show resolved Hide resolved

tests/functional/test_service_headers.py Outdated Show resolved Hide resolved

tests/unit/test_handlers.py Outdated Show resolved Hide resolved

implement feedback and move test_only_s3_targets_expires_header() to …

0f0861d

…test_s3.py

alexgromero requested review from nateprewitt, SamRemis and jonathan343 September 4, 2024 19:09

change implementation to use event system and update tests

822dfe6

nateprewitt reviewed Sep 5, 2024

View reviewed changes

botocore/endpoint.py Outdated Show resolved Hide resolved

botocore/endpoint.py Outdated Show resolved Hide resolved

SamRemis reviewed Sep 5, 2024

View reviewed changes

tests/unit/test_endpoint.py Outdated Show resolved Hide resolved

alexgromero and others added 3 commits September 5, 2024 19:24

clean up test typo

67ad92b

Update botocore/endpoint.py

c6242c0

Co-authored-by: Nate Prewitt <nate.prewitt@gmail.com>

change additional_response_keys to customized_response_dict

56cca67

alexgromero requested review from SamRemis and nateprewitt September 6, 2024 15:17

jonathan343 reviewed Sep 6, 2024

View reviewed changes

botocore/handlers.py Outdated Show resolved Hide resolved

update handler to only target output structures that contain Expires …

95797f3

…shape

alexgromero requested a review from jonathan343 September 6, 2024 22:27

SamRemis approved these changes Sep 7, 2024

View reviewed changes

jonathan343 approved these changes Sep 9, 2024

View reviewed changes

alexgromero merged commit 4c2771a into boto:develop Sep 10, 2024
34 checks passed

hswong3i pushed a commit to alvistack/boto-botocore that referenced this pull request Sep 11, 2024

s3 expires implementation (boto#3232)

0190a3a

* implementation to gracefully handle invalid timestamps returned in the Expires header

aemous mentioned this pull request Sep 20, 2024

[v2] S3 expires timestamp port aws/aws-cli#8935

Merged

alexgromero deleted the s3-expires-timestamp branch October 21, 2024 16:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

s3 expires implementation #3232

s3 expires implementation #3232

alexgromero commented Aug 6, 2024 •

edited

Loading

codecov-commenter commented Aug 6, 2024 •

edited

Loading

jonathan343 left a comment

jonathan343 left a comment

s3 expires implementation #3232

s3 expires implementation #3232

Conversation

alexgromero commented Aug 6, 2024 • edited Loading

codecov-commenter commented Aug 6, 2024 • edited Loading

Codecov Report

jonathan343 left a comment

Choose a reason for hiding this comment

jonathan343 left a comment

Choose a reason for hiding this comment

alexgromero commented Aug 6, 2024 •

edited

Loading

codecov-commenter commented Aug 6, 2024 •

edited

Loading