Sofatutor tweaks

.github/workflows/sofatutor_image.yml

@@ -0,0 +1,59 @@
+# This workflow uses actions that are not certified by GitHub.
+# They are provided by a third-party and are governed by
+# separate terms of service, privacy policy, and support
+# documentation.
+
+# GitHub recommends pinning actions to a commit SHA.
+# To get a newer version, you will need to update the SHA.
+# You can also reference a tag or branch, but the action may change without warning.
+---
+name: Create and publish a Container image
+
+on:
+  push:
+    branches:
+      - sofatutor-tweaks
+    tags:
+      - "v*"
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository_owner }}/${{ github.repository }}
+
+jobs:
+  build-and-push-image:
+    runs-on: ubuntu-22.04
+    permissions:
+      contents: read
+      packages: write
+
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+
+      - name: Log in to the Container registry
+        uses: sofatutor/docker-login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: sofatutor/docker-metadata-action@57396166ad8aefe6098280995947635806a0e6ea
+        with:
+          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+          tags: |
+            type=ref,event=branch
+            type=ref,event=pr
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}}
+
+      - name: Build and push Docker image
+        uses: sofatutor/docker-build-push-action@c56af957549030174b10d6867f20e78cfd7debc5
+        with:
+          context: .
+          push: true
+          pull: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}

docs/feature-cloudwatch-logging-plan.md

@@ -0,0 +1,95 @@
+# Feature: CloudWatch Logging Support
+
+> **Note:** All relevant patch diffs for this feature are available in `/tmp/patch_*.diff`. Refer to these files for manual or automated patch application as needed.
+
+## Summary
+This document tracks the changes required to add AWS CloudWatch logging support to LiteLLM. The goal is to integrate new logging modules, refactor callback handling, and ensure robust logging for CloudWatch.
+
+## Checklist
+- [x] Identify all relevant code changes from the diff
+- [x] Apply new CloudWatch integration module
+- [x] Refactor core logging utilities to support CloudWatch
+- [x] Update proxy and utility modules for new logging hooks and call types
+- [x] Test logging functionality in development/staging
+- [x] Document configuration and usage
+- [x] Fix tests to properly handle CloudWatch integration
+
+## Sequential Application Plan
+
+1. **litellm/integrations/cloud_watch.py**
+   - [x] Check if file exists (it does not)
+   - [x] Apply patch directly (new file)
+
+2. **litellm/litellm_core_utils/litellm_logging.py**
+   - [x] Attempt to apply patch (conflicts detected)
+   - [x] Review .rej file and patch contents
+   - [x] Identify where CloudWatch logging changes need to be integrated
+   - [x] Manually merge relevant changes
+   - [x] Test logging functionality
+
+3. **litellm/proxy/proxy_server.py**
+   - [x] Review patch and compare with current file
+   - [x] Manually merge relevant changes
+   - [x] Test proxy logging hooks
+
+4. **litellm/proxy/utils.py**
+   - [x] Review patch and compare with current file
+   - [x] Manually merge relevant changes
+   - [x] Test call type handling
+
+5. **litellm/utils.py**
+   - [x] Review patch and compare with current file
+   - [x] Manually merge relevant changes
+   - [x] Test async callback logic
+
+6. **litellm/__init__.py**
+   - [x] Add `cloudwatch_callback_params` attribute to fix test errors
+
+## Changes Applied (from diff)
+| File | Change Type | Description | Status |
+|------|-------------|-------------|--------|
+| litellm/integrations/cloud_watch.py | Add | New integration for logging to AWS CloudWatch | ✅ Complete |
+| litellm/litellm_core_utils/litellm_logging.py | Modify | Refactors AWS logging, adds CloudWatch logger, improves callback handling | ✅ Complete |
+| litellm/proxy/proxy_server.py | Modify | Adds/updates logging hooks, filters model, adds call IDs, updates headers | ✅ Complete |
+| litellm/proxy/utils.py | Modify | Adds new call types to pre_call_hook | ✅ Complete |
+| litellm/utils.py | Modify | Adds cloudwatch to async callback logic, supports litellm_metadata | ✅ Complete |
+| litellm/__init__.py | Modify | Added `cloudwatch_callback_params` attribute to fix test errors | ✅ Complete |
+
+## Testing Coverage
+
+### Existing Tests Updated
+- [x] Updated `test_embedding_input_array_of_tokens` in `tests/litellm/proxy/test_proxy_server.py` to handle detailed proxy_server_request logging data
+- [x] Verified all tests pass with our changes (432 tests in the proxy module passed)
+
+### New Tests Implemented
+
+1. **CloudWatch Logger Tests**
+   - [x] Created/Updated `tests/litellm/integrations/test_cloud_watch.py` with:
+     - Unit tests for CloudWatch logger initialization
+     - Mocked AWS CloudWatch API calls to test log delivery
+     - Tests for different log event formats
+
+3. **Test Fixes**
+   - [x] Fixed test initialization issues with `ProxyLogging` class by properly mocking the required `user_api_key_cache` parameter
+   - [x] Ensured proper cleanup in test mocks to avoid test interference
+   - [x] Made sure all tests properly exercise the actual functionality rather than stubbing essential components
+
+## Configuration Details
+
+CloudWatch logging can be configured in the LiteLLM config using:
+
+```yaml
+litellm_settings:
+  telemetry: True
+  success_callback: ["cloudwatch"]
+  cloudwatch_callback_params:
+    log_group_name: /litellm
+    aws_region: eu-central-1
+```
+
+## Next Steps
+- Consider additional parameterization for CloudWatch logging (e.g., customizable log streams)
+- Monitor production usage for any performance impacts
+- Document best practices for log retention and analysis
+
+<!-- End of Feature Tracking --> 

docs/my-website/docs/proxy/logging.md

@@ -1316,7 +1316,6 @@ This will log all successful LLM calls to s3 Bucket
 ```shell
 AWS_ACCESS_KEY_ID = ""
 AWS_SECRET_ACCESS_KEY = ""
-AWS_REGION_NAME = ""
 ```
 
 **Step 2**: Create a `config.yaml` file and set `litellm_settings`: `success_callback`
@@ -2626,3 +2625,39 @@ litellm_settings:
 `thresholds` are not required by default, but you can tune the values to your needs.
 Default values is `4` for all categories
 ::: -->
+
+## CloudWatch Logging
+
+Log LLM input/output to AWS CloudWatch.
+
+| Property | Details |
+|----------|---------|
+| Description | Log LLM calls to AWS CloudWatch |
+
+#### Basic Setup
+
+1. Add `cloudwatch` to your config.yaml
+```yaml
+litellm_settings:
+  success_callback: ["cloudwatch"]
+  cloudwatch_callback_params:
+    log_group_name: /litellm
+    aws_region: us-west-2
+```
+
+2. Set AWS credentials as environment variables
+```shell
+AWS_ACCESS_KEY_ID=""
+AWS_SECRET_ACCESS_KEY=""
+AWS_REGION="us-west-2"
+```
+
+3. Start the proxy
+```
+litellm --config /path/to/config.yaml
+```
+
+#### Fields Logged to CloudWatch
+
+- Standard LLM request/response data
+- All logs include a unique `litellm_call_id` for tracing