|
56 | 56 | - Drop-in replacement for `boto3.session.Session` |
57 | 57 | - Supports `assume_role` configuration, custom STS clients, and profile / region configuration, as well as all other parameters supported by `boto3.session.Session` |
58 | 58 | - Tested, documented, and published to PyPI |
59 | | -- Used in production at major tech companies |
60 | 59 |
|
61 | | -## Adoption and Recognition |
| 60 | +## Recognition, Adoption, and Testimonials |
62 | 61 |
|
63 | | -[Mentioned in TL;DR Sec](https://tldrsec.com/p/tldr-sec-282). |
| 62 | +[Featured in TL;DR Sec.](https://tldrsec.com/p/tldr-sec-282) |
64 | 63 |
|
65 | | -Received honorable mention during AWS Community Day Midwest on June 5th, 2025. |
| 64 | +Recognized during AWS Community Day Midwest on June 5th, 2025. |
66 | 65 |
|
67 | | -Used by multiple teams and large companies including FAANG. |
| 66 | +A testimonial from a Cyber Security Engineer at a FAANG company: |
68 | 67 |
|
69 | | -The following line plot illustrates the adoption of BRS from the last three months in terms of average daily downloads over a rolling seven day window. |
| 68 | +> _Most of my work is on tooling related to AWS security, so I'm pretty choosy about boto3 credentials-adjacent code. I often opt to just write this sort of thing myself so I at least know that I can reason about it. But I found boto3-refresh-session to be very clean and intuitive [...] We're using the RefreshableSession class as part of a client cache construct [...] We're using AWS Lambda to perform lots of operations across several regions in hundreds of accounts, over and over again, all day every day. And it turns out that there's a surprising amount of overhead to creating boto3 clients (mostly deserializing service definition json), so we can run MUCH more efficiently if we keep a cache of clients, all equipped with automatically refreshing sessions._ |
| 69 | +
|
| 70 | +The following line plot illustrates the adoption of BRS over the last three months in terms of average daily downloads over a rolling seven day window. |
70 | 71 |
|
71 | 72 | <p align="center"> |
72 | 73 | <img src="https://raw.githubusercontent.com/michaelthomasletts/boto3-refresh-session/refs/heads/main/doc/downloads.png" /> |
73 | 74 | </p> |
74 | 75 |
|
75 | | -## Testimonials |
76 | | - |
77 | | -From a Cyber Security Engineer at a FAANG company: |
78 | | - |
79 | | -> _Most of my work is on tooling related to AWS security, so I'm pretty choosy about boto3 credentials-adjacent code. I often opt to just write this sort of thing myself so I at least know that I can reason about it. But I found boto3-refresh-session to be very clean and intuitive [...] We're using the RefreshableSession class as part of a client cache construct [...] We're using AWS Lambda to perform lots of operations across several regions in hundreds of accounts, over and over again, all day every day. And it turns out that there's a surprising amount of overhead to creating boto3 clients (mostly deserializing service definition json), so we can run MUCH more efficiently if we keep a cache of clients, all equipped with automatically refreshing sessions._ |
80 | | -
|
81 | 76 | ## Installation |
82 | 77 |
|
83 | 78 | ```bash |
@@ -125,27 +120,28 @@ buckets = s3.list_buckets() |
125 | 120 |
|
126 | 121 | ## Raison d'être |
127 | 122 |
|
128 | | -It is common for data pipelines and workflows that interact with the AWS API via |
129 | | -`boto3` to run for a long time and, accordingly, for temporary credentials to |
130 | | -expire. |
| 123 | +Long-running data pipelines, security tooling, ETL jobs, and cloud automation scripts frequently interact with the AWS API using boto3 — and often run into the same problem: |
| 124 | + |
| 125 | +**Temporary credentials expire.** |
| 126 | + |
| 127 | +When that happens, engineers typically fall back on one of two strategies: |
| 128 | + |
| 129 | +- Wrapping AWS calls in try/except blocks that catch ClientError exceptions |
| 130 | +- Writing ad hoc logic to refresh credentials using botocore credentials internals |
131 | 131 |
|
132 | | -Usually, engineers deal with that problem one of two ways: |
| 132 | +Both approaches are fragile, tedious to maintain, and error-prone at scale. |
133 | 133 |
|
134 | | -- `try except` blocks that catch `ClientError` exceptions |
135 | | -- A similar approach as that used in this project -- that is, using methods available |
136 | | - within `botocore` for refreshing temporary credentials automatically. |
137 | | - |
138 | | -Speaking personally, variations of the code found herein exists in code bases at |
139 | | -nearly every company where I have worked. Sometimes, I turned that code into a module; |
140 | | -other times, I wrote it from scratch. Clearly, that is inefficient. |
| 134 | +Over the years, I noticed that every company I worked for — whether a scrappy startup or FAANG — ended up with some variation of the same pattern: |
| 135 | +a small in-house module to manage credential refresh, written in haste, duplicated across services, and riddled with edge cases. Things only |
| 136 | +got more strange and difficult when I needed to run things in parallel. |
141 | 137 |
|
142 | | -I decided to finally turn that code into a proper Python package with unit testing, |
143 | | -automatic documentation, and quality checks; the idea being that, henceforth, depending |
144 | | -on my employer's open source policy, I may simply import this package instead of |
145 | | -reproducing the code herein for the Nth time. |
| 138 | +Eventually, I decided to build boto3-refresh-session as a proper open-source Python package: |
146 | 139 |
|
147 | | -If any of that sounds relatable, then `boto3-refresh-session` should help you. |
| 140 | +- Fully tested |
| 141 | +- Extensible |
| 142 | +- Integrated with boto3 idioms |
| 143 | +- Equipped with automatic documentation and CI tooling |
148 | 144 |
|
149 | | ---- |
| 145 | +**The goal:** to solve a real, recurring problem once — cleanly, consistently, and for everyone -- with multiple refresh strategies. |
150 | 146 |
|
151 | | -📄 Licensed under the MIT License. |
| 147 | +If you've ever written the same AWS credential-refresh boilerplate more than once, this library is for you. |
0 commit comments