Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2026 Amit Vikram Raj

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
36 changes: 33 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,29 @@
# Files API

This project is a more polished version of the [cloud-engineering-project](https://github.com/avr2002/cloud-engineering-project) that we built during the [MLOps Club](https://mlops-club.org/) cohort of Taking Python to Production on AWS by [me](https://www.linkedin.com/in/avr27/) and [Eric Riddoch](https://www.linkedin.com/in/eric-riddoch/).
<p align="left">
<a href="https://www.linkedin.com/in/avr27/" target="_blank">
<img src="https://img.shields.io/badge/Maintained%20by-Amit%20Vikram%20Raj-F4BBFF?style=for-the-badge">
</a>
<a href="https://mlops-club.org" target="_blank">
<img src="https://img.shields.io/badge/MLOps%20Club-05998B?style=for-the-badge">
</a>
</p>

This project is a more polished version of the [cloud-engineering-project](https://github.com/avr2002/cloud-engineering-project) that we built during the [MLOps Club](https://mlops-club.org/) cohort of Taking Python to Production on AWS by [Eric Riddoch](https://www.linkedin.com/in/eric-riddoch/) and [me](https://www.linkedin.com/in/avr27/).


<p align="center">
<img src="docs/images/deployed-api.png" alt="files-api"/>
</p>

In this project, we built --
***In this project, we built ––***

* A [RESTful API](https://aws.amazon.com/what-is/restful-api/) using FastAPI to do CRUD operations against an S3 bucket.
* Implemented principles of [12-factor app](https://12factor.net/) and [RESTful API design](https://restfulapi.net/).
* Dockerized the application for easy local development and deployment.
* Rigorously tested it using pytest with mocks for AWS S3 and OpenAI services.
* Setup load testing with Locust
* We wrote API Contract using OpenAPI spec, auto-generated from FastAPI code, with pre-commit hooks using `oasdiff` to catch breaking changes and OpenAPI Generator to create a Python client SDK for the API.
* Serverless Deployment: Deployed the app using AWS CDK with Docker on AWS Lambda and exposed it via API Gateway.
* CI/CD Pipeline: Automated testing and deployment using GitHub Actions.
* Observability & Monitoring:
Expand All @@ -24,7 +34,18 @@ In this project, we built --

## Setup

Install [`uv`](https://docs.astral.sh/uv/getting-started/installation/) and node.js before running.
Install [`uv`](https://docs.astral.sh/uv/getting-started/installation/), [aws-cli v2](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and [node.js](https://nodejs.org/en/download) before running.


- Clone the repo
```bash
git clone https://github.com/avr2002/files-api.git
```

- Install dependencies
```bash
uv sync --all-groups
```

- If you are using AWS SSO, you can activate your profile by running the following command:

Expand Down Expand Up @@ -73,6 +94,7 @@ Install [`uv`](https://docs.astral.sh/uv/getting-started/installation/) and node
```bash
./run run-locust
```
^^^ If you want to load test against deployed API, modify the [docker-compose.locust.yaml](./docker-compose.locust.yaml) file with the deployed API Gateway URL

- Generate OpenAPI spec
```bash
Expand Down Expand Up @@ -156,3 +178,11 @@ Install [`uv`](https://docs.astral.sh/uv/getting-started/installation/) and node

Similary, you can view other metrics like Lambda Invocations, Duration, Errors, Throttles etc. and for S3 as well.


## Contributing

Contributions are welcome!
- Complete the setup process above
- Make your changes
- Run tests and linting, and
- Submit a pull request.
7 changes: 3 additions & 4 deletions docker-compose.locust.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,9 @@ services:
locust:
image: locustio/locust:latest
command: >
--locustfile /locustfile.py --host=https://k4cspk2km1.execute-api.us-west-2.amazonaws.com/prod --users 100 --spawn-rate 5 --autostart
# --locustfile /locustfile.py --host=http://fastapi:8000 --users 1 --spawn-rate 1 --autostart
# --locustfile /locustfile.py --host=https://3rbeuvjyd3.execute-api.us-west-2.amazonaws.com/prod --users 1 --spawn-rate 1 --autostart
# --locustfile /locustfile.py --host=https://j0o3bozns6.execute-api.ap-south-1.amazonaws.com/prod/ --users 1 --spawn-rate 1 --autostart
--locustfile /locustfile.py --host=http://fastapi:8000 --users 1 --spawn-rate 1 --autostart
# --locustfile /locustfile.py --host=https://k4cspk2km1.execute-api.us-west-2.amazonaws.com/prod --users 100 --spawn-rate 5 --autostart

ports:
- "8089:8089"
volumes:
Expand Down
Binary file added docs/images/mlops-club.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
[project]
name = "files-api"
version = "0.1.0"
version = "0.1.1"
description = "Files API to upload and download files from S3 using FastAPI."
readme = "README.md"
authors = [
{ name = "avr2002", email = "avr13405@gmail.com" }
]
license = { text="MIT" }
requires-python = ">=3.12"
dependencies = [
"aws-embedded-metrics>=3.3.1",
Expand Down
73 changes: 73 additions & 0 deletions scripts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Scripts

Utility scripts for OpenAPI schema generation, load testing, and SDK usage examples.

## OpenAPI Schema Generation

- **`generate-openapi.py`**:
Feature-rich script to generate and validate OpenAPI schemas from the FastAPI application.

**Commands:**

- **Generate** - Create OpenAPI schema from FastAPI app:
```shell
uv run ./scripts/generate-openapi.py generate --output-spec=openapi.json
```

- **Generate and Diff** - Generate schema and compare with existing spec:
```shell
uv run ./scripts/generate-openapi.py generate-and-diff \
--existing-spec ./openapi.json \
--output-spec ./openapi.json \
--fail-on-diff
```

Used in pre-commit hooks to ensure the OpenAPI schema stays synchronized with code changes.

- **`generate-openapi-simple.py`**

Simplified version demonstrating the core OpenAPI generation logic. Generates schema and compares it with the existing `openapi.json` file, exiting with an error if they differ. Useful for understanding the basics before working with the full-featured `generate-openapi.py`.

**Usage:**
```shell
uv run ./scripts/generate-openapi-simple.py
```

## Load Testing

**`locustfile.py`**

Locust configuration for load testing the Files API. Simulates realistic user flows including file uploads, downloads, deletions, and AI file generation.

**Tasks:**
- `file_operations_flow`: Tests basic CRUD operations (list, upload, describe, download, delete)
- `generate_ai_files_flow`: Tests AI-powered file generation (text, image, audio)

**Usage:**
```shell
# Start with docker-compose
./run run-locust

# Or run Locust directly
uv run locust -f scripts/locustfile.py --host=http://localhost:8000
```

Access the Locust web UI at http://localhost:8089 to configure and monitor load tests.

## SDK Testing

**`try_client.py`**

Example script demonstrating usage of the auto-generated Python SDK (`files-api-sdk`). Shows how to upload files using the client library.

**Usage:**
```shell
# First, generate and install the SDK
./run generate-client-library
./run install-generated-sdk

# Then run the example
uv run ./scripts/try_client.py
```

Make sure the API is running locally before executing this script.
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.