WIP: Add support for Cloudflare R2 object storage

docs/cloud-object-storage.md

@@ -2,7 +2,7 @@
 
 ## Overview
 
-Dotenv files are commonly kept in [cloud object storage](https://en.wikipedia.org/wiki/Cloud_storage), but environment variable management packages typically don't integrate with object storage clients. Additional logic is therefore required to download the files from object storage prior to loading environment variables. This project offers integration with S3-compatible object storage. [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) and [Backblaze B2](https://www.backblaze.com/b2/docs/) are directly supported and tested.
+Dotenv files are commonly kept in [cloud object storage](https://en.wikipedia.org/wiki/Cloud_storage), but environment variable management packages typically don't integrate with object storage clients. Additional logic is therefore required to download the files from object storage prior to loading environment variables. This project offers integration with S3-compatible object storage. [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html), [Backblaze B2](https://www.backblaze.com/b2/docs/), and [Cloudflare R2](https://developers.cloudflare.com/r2/) are directly supported and tested.
 
 !!!note "Why not Boto3?"
 
@@ -370,10 +370,25 @@ Here's an example of how this could be implemented.
 
 ### Cloudflare R2
 
-_Coming soon!_
-
--   [Cloudflare Blog 2021-07-23: AWS's egregious egress](https://blog.cloudflare.com/aws-egregious-egress/)
--   [Cloudflare Blog 2021-09-28: Announcing Cloudflare R2 Storage](https://blog.cloudflare.com/introducing-r2-object-storage/)
+-   [Pricing](https://developers.cloudflare.com/r2/platform/pricing/)
+-   [S3-compatible API](https://developers.cloudflare.com/r2/platform/s3-compatibility/api/)
+-   URIs
+    -   Regions are handled automatically. "When using the S3 API, the region for an R2 bucket is `auto`. For compatibility with tools that do not allow you to specify a region, an empty value and `us-east-1` will alias to the `auto` region."
+    -   The Cloudflare account ID is included in bucket URIs, which is different from other platforms.
+    -   Path style URL: `https://<ACCOUNT_ID>.r2.cloudflarestorage.com/<bucketname>` (they don't show position of bucket name in the docs?)
+    -   Virtual-hosted-style URL: `https://<BUCKET>.<ACCOUNT_ID>.r2.cloudflarestorage.com` (added [2022-05-16](https://developers.cloudflare.com/r2/platform/changelog/#2022-05-16))
+    -   Presigned URLs are supported
+        -   Added [2022-06-17](https://developers.cloudflare.com/r2/platform/changelog/#2022-06-17)
+        -   Note that there may still be CORS limitations for client-side uploads ([cloudflare/cloudflare-docs#4455](https://github.com/cloudflare/cloudflare-docs/issues/4455#issuecomment-1170770935))
+-   Identity and Access Management (IAM):
+    -   [Requires generation of a static access key](https://developers.cloudflare.com/r2/data-access/s3-api/tokens/). Does not appear to support temporary credentials from IAM roles (AWS session tokens). Does not appear to support OpenID Connect (OIDC).
+    -   Access keys can be set to either read-only or edit permissions.
+    -   Access keys can be scoped to specific Cloudflare products, Cloudflare accounts, and IP addresses.
+-   Docs
+    -   [Cloudflare R2 docs](https://developers.cloudflare.com/r2/)
+    -   [Cloudflare Blog 2021-07-23: AWS's egregious egress](https://blog.cloudflare.com/aws-egregious-egress/)
+    -   [Cloudflare Blog 2021-09-28: Announcing Cloudflare R2 Storage](https://blog.cloudflare.com/introducing-r2-object-storage/)
+    -   [Cloudflare Blog 2022-09-21: R2 is now Generally Available](https://blog.cloudflare.com/r2-ga/)
 
 ### DigitalOcean Spaces
 

docs/contributing.md

@@ -170,6 +170,9 @@ BACKBLAZE_B2_ACCESS_KEY_FASTENV="paste-here"
  BACKBLAZE_B2_SECRET_KEY_FASTENV="paste-here"
 BACKBLAZE_B2_BUCKET_HOST="paste-here"
 BACKBLAZE_B2_BUCKET_REGION="paste-here"
+CLOUDFLARE_R2_ACCESS_KEY_FASTENV="paste-here"
+ CLOUDFLARE_R2_SECRET_KEY_FASTENV="paste-here"
+CLOUDFLARE_R2_BUCKET_HOST="paste-here"
 
 # get AWS account ID from STS (replace fx with jq or other JSON parser as needed)
 AWS_ACCOUNT_ID=$(aws sts get-caller-identity | fx .Account)
@@ -306,6 +309,12 @@ A [B2 application key](https://www.backblaze.com/b2/docs/application_keys.html)
 
 See the [Backblaze B2 S3-compatible API docs](https://www.backblaze.com/b2/docs/s3_compatible_api.html) for further info.
 
+### GitHub Actions and Cloudflare R2
+
+A [Cloudflare S3 auth token](https://developers.cloudflare.com/r2/data-access/s3-api/tokens/) (access key) is stored in GitHub Secrets, along with the corresponding bucket host in "virtual-hosted-style" format (`https://<BUCKET>.<ACCOUNT_ID>.r2.cloudflarestorage.com`).
+
+See the [Cloudflare R2 docs](https://developers.cloudflare.com/r2/) for further info.
+
 ## Maintainers
 
 -   **The default branch is `develop`.**

fastenv/cloud/object_storage.py

@@ -26,11 +26,12 @@ class ObjectStorageConfig:
     """Configure S3-compatible object storage.
     ---
 
-    AWS S3 and Backblaze B2 are directly supported and tested.
+    AWS S3, Backblaze B2, and Cloudflare R2 are directly supported and tested.
 
     Buckets can be specified in "virtual-hosted-style", like
-    `<BUCKET_NAME>.s3.<REGION>.amazonaws.com` for AWS S3 or
-    `<BUCKET_NAME>.s3.<REGION>.backblazeb2.com` for Backblaze B2.
+    `<BUCKET_NAME>.s3.<REGION>.amazonaws.com` for AWS S3,
+    `<BUCKET_NAME>.s3.<REGION>.backblazeb2.com` for Backblaze B2, or
+    `<BUCKET_NAME>.<ACCOUNT_ID>.r2.cloudflarestorage.com` for Cloudflare R2.
     For AWS S3 only, the bucket can be also provided as just `<BUCKET_NAME>`.
 
     If credentials are not provided as arguments, this class will auto-detect
@@ -73,8 +74,10 @@ def __init__(
         if not bucket_host and not bucket_name:
             raise AttributeError(
                 "Required bucket info not provided. Please provide a bucket, "
-                "like `<BUCKET_NAME>.s3.<REGION>.amazonaws.com` for AWS S3 or "
-                "`<BUCKET_NAME>.s3.<REGION>.backblazeb2.com` for Backblaze B2."
+                "like `<BUCKET_NAME>.s3.<REGION>.amazonaws.com` for AWS S3, "
+                "`<BUCKET_NAME>.s3.<REGION>.backblazeb2.com` for Backblaze B2, "
+                "or `<BUCKET_NAME>.<ACCOUNT_ID>.r2.cloudflarestorage.com` "
+                "for Cloudflare R2."
             )
         elif bucket_host and not bucket_name:
             if (
@@ -119,15 +122,16 @@ class ObjectStorageClient:
     """Instantiate a client to connect to S3-compatible object storage.
     ---
 
-    AWS S3 and Backblaze B2 are directly supported and tested.
+    AWS S3, Backblaze B2, and Cloudflare R2 are directly supported and tested.
 
     This class requires both an HTTPX client and an `ObjectStorageConfig` instance.
     They will be automatically instantiated if not provided as arguments.
     Any additional arguments will be used to instantiate `ObjectStorageConfig`.
 
     Buckets can be specified in "virtual-hosted-style", like
-    `<BUCKET_NAME>.s3.<REGION>.amazonaws.com` for AWS S3 or
-    `<BUCKET_NAME>.s3.<REGION>.backblazeb2.com` for Backblaze B2.
+    `<BUCKET_NAME>.s3.<REGION>.amazonaws.com` for AWS S3,
+    `<BUCKET_NAME>.s3.<REGION>.backblazeb2.com` for Backblaze B2, or
+    `<BUCKET_NAME>.<ACCOUNT_ID>.r2.cloudflarestorage.com` for Cloudflare R2.
     For AWS S3 only, the bucket can be also provided as just `<BUCKET_NAME>`.
 
     https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html

tests/conftest.py

@@ -58,13 +58,21 @@ def anyio_backend() -> str:
     bucket_host_variable="BACKBLAZE_B2_BUCKET_HOST",
     bucket_region_variable="BACKBLAZE_B2_BUCKET_REGION",
 )
+_cloud_params_cloudflare_static = CloudParams(
+    access_key_variable="CLOUDFLARE_R2_ACCESS_KEY_FASTENV",
+    secret_key_variable="CLOUDFLARE_R2_SECRET_KEY_FASTENV",
+    session_token_variable="",
+    bucket_host_variable="CLOUDFLARE_R2_BUCKET_HOST",
+    bucket_region_variable="auto",
+)
 
 
 @pytest.fixture(
     params=(
         _cloud_params_aws_session,
         _cloud_params_aws_static,
         _cloud_params_backblaze_static,
+        _cloud_params_cloudflare_static,
     ),
     scope="session",
 )