v1.52.6

What's Changed

• LiteLLM Minor Fixes & Improvements (11/12/2024) by @krrishdholakia in #6705
• (feat) helm hook to sync db schema by @ishaan-jaff in #6715
• (fix proxy redis) Add redis sentinel support by @ishaan-jaff in #6154
• Fix: Update gpt-4o costs to those of gpt-4o-2024-08-06 by @klieret in #6714
• (fix) using Anthropic response_format={"type": "json_object"} by @ishaan-jaff in #6721
• (feat) Add cost tracking for Azure Dall-e-3 Image Generation + use base class to ensure basic image generation tests pass by @ishaan-jaff in #6716

New Contributors

• @klieret made their first contribution in #6714

Full Changelog: v1.52.5...v1.52.6

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.6

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	240.0	266.21521040425523	6.127671245386762	0.0	1833	0	215.80195500001764	2902.9665340000292
Aggregated	Passed ✅	240.0	266.21521040425523	6.127671245386762	0.0	1833	0	215.80195500001764	2902.9665340000292

v1.52.5

What's Changed

• Litellm dev 11 11 2024 by @krrishdholakia in #6693
• Add docs to export logs to Laminar by @dinmukhamedm in #6674
• (Feat) Add langsmith key based logging by @ishaan-jaff in #6682
• (fix) OpenAI's optional messages[].name does not work with Mistral API by @ishaan-jaff in #6701
• (feat) add xAI on Admin UI by @ishaan-jaff in #6680
• (docs) add benchmarks on 1K RPS by @ishaan-jaff in #6704
• (feat) add cost tracking stable diffusion 3 on Bedrock by @ishaan-jaff in #6676
• fix raise correct error 404 when /key/info is called on non-existent key by @ishaan-jaff in #6653

New Contributors

• @dinmukhamedm made their first contribution in #6674

Full Changelog: v1.52.4...v1.52.5

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.5

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	200.0	216.13288200000045	6.215294300193555	0.0	1859	0	166.97629999998753	1726.1806539999611
Aggregated	Passed ✅	200.0	216.13288200000045	6.215294300193555	0.0	1859	0	166.97629999998753	1726.1806539999611

v1.52.4

What's Changed

• (feat) Add support for logging to GCS Buckets with folder paths by @ishaan-jaff in #6675
• (feat) add bedrock image gen async support by @ishaan-jaff in #6672
• (feat) Add Bedrock Stability.ai Stable Diffusion 3 Image Generation models by @ishaan-jaff in #6673
• (Feat) 273% improvement GCS Bucket Logger - use Batched Logging by @ishaan-jaff in #6679

Full Changelog: v1.52.3...v1.52.4

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.4

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	260.0	290.15274785816086	6.102299282865334	0.0	1826	0	221.48416699997142	3998.8694860000464
Aggregated	Passed ✅	260.0	290.15274785816086	6.102299282865334	0.0	1826	0	221.48416699997142	3998.8694860000464

v1.52.3

What's Changed

• Litellm Minor Fixes & Improvements (11/08/2024) by @krrishdholakia in #6658
• (pricing): Fix multiple mistakes in Claude pricing by @Manouchehri in #6666

Full Changelog: v1.52.2...v1.52.3

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.3

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	210.0	236.59706194640916	6.234242656262243	0.0	1866	0	180.61705699994945	3424.5764140000006
Aggregated	Passed ✅	210.0	236.59706194640916	6.234242656262243	0.0	1866	0	180.61705699994945	3424.5764140000006

v1.52.2-dev1

Full Changelog: v1.52.3...v1.52.2-dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.2-dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.2-dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	200.0	220.40195196940357	6.227773108800657	0.0	1863	0	180.672578000042	2967.1460419999676
Aggregated	Passed ✅	200.0	220.40195196940357	6.227773108800657	0.0	1863	0	180.672578000042	2967.1460419999676

v1.52.2

What's Changed

• chore: comment for maritalk by @nobu007 in #6607
• Update gpt-4o-2024-08-06, and o1-preview, o1-mini models in model cost map by @emerzon in #6654
• (QOL improvement) add unit testing for all static_methods in litellm_logging.py by @ishaan-jaff in #6640
• (feat) log error class, function_name on prometheus service failure hook + only log DB related failures on DB service hook by @ishaan-jaff in #6650
• Update several Azure AI models in model cost map by @emerzon in #6655
• ci(conftest.py): reset conftest.py for local_testing/ by @krrishdholakia in #6657
• Litellm dev 11 07 2024 by @krrishdholakia in #6649

New Contributors

• @emerzon made their first contribution in #6654

Full Changelog: v1.52.1...v1.52.2

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	230.0	251.09411961031876	6.087114215107422	0.0	1822	0	198.72582000004968	1667.4085729999888
Aggregated	Passed ✅	230.0	251.09411961031876	6.087114215107422	0.0	1822	0	198.72582000004968	1667.4085729999888

v1.52.1

What's Changed

• (DB fix) don't run apply_db_fixes on startup by @ishaan-jaff in #6604
• LiteLLM Minor Fixes & Improvements (11/04/2024) by @krrishdholakia in #6572
• ci: remove redundant lint.yml workflow by @krrishdholakia in #6622
• LiteLLM Minor Fixes & Improvements (11/05/2024) by @krrishdholakia in #6590
• LiteLLM Minor Fixes & Improvements (11/06/2024) by @krrishdholakia in #6624
• (feat) GCS Bucket logging. Allow using IAM auth for logging to GCS by @ishaan-jaff in #6628
• Update opentelemetry_integration.md - Fix typos by @ronensc in #6618
• (fix) ProxyStartup - Check that prisma connection is healthy when starting an instance of LiteLLM by @ishaan-jaff in #6627
• Update team_budgets.md by @superpoussin22 in #6611
• (feat) Allow failed DB connection requests to allow virtual keys with allow_failed_db_requests by @ishaan-jaff in #6605
• fix(pattern_match_deployments.py): default to user input if unable to… by @krrishdholakia in #6646
• fix(pattern_match_deployments.py): default to user input if unable to… by @krrishdholakia in #6632

New Contributors

• @ronensc made their first contribution in #6618

Full Changelog: v1.52.0...v1.52.1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	260.0	292.8286898309638	6.110969302244283	0.0	1828	0	230.12115400001676	2643.3588609999674
Aggregated	Passed ✅	260.0	292.8286898309638	6.110969302244283	0.0	1828	0	230.12115400001676	2643.3588609999674

v1.52.0-stable

What's Changed

• LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
• Litellm dev 11 02 2024 by @krrishdholakia in #6561
• build(deps): bump cookie and express in /docs/my-website by @dependabot in #6566
• docs(virtual_keys.md): update Dockerfile reference by @emmanuel-ferdman in #6554
• (proxy fix) - call connect on prisma client when running setup by @ishaan-jaff in #6534
• Add 3.5 haiku by @paul-gauthier in #6588
• Litellm perf improvements 3 by @krrishdholakia in #6573
• (fix) /image/generation - ImageObject conversion when content_filter_results exists by @ishaan-jaff in #6584
• (fix) litellm.text_completion raises a non-blocking error on simple usage by @ishaan-jaff in #6546
• (feat) add Predicted Outputs for OpenAI by @ishaan-jaff in #6594
• (fix) Vertex Improve Performance when using image_url by @ishaan-jaff in #6593
• fix(lowest_tpm_rpm_routing.py): fix parallel rate limit check by @krrishdholakia in #6577

New Contributors

• @emmanuel-ferdman made their first contribution in #6554

Full Changelog: v1.51.3...v1.52.0-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	240.0	270.29554346208295	6.124428891308869	0.0	1833	0	212.83106800001406	1622.2440090000418
Aggregated	Passed ✅	240.0	270.29554346208295	6.124428891308869	0.0	1833	0	212.83106800001406	1622.2440090000418

v1.52.0

What's Changed

• LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
• Litellm dev 11 02 2024 by @krrishdholakia in #6561
• build(deps): bump cookie and express in /docs/my-website by @dependabot in #6566
• docs(virtual_keys.md): update Dockerfile reference by @emmanuel-ferdman in #6554
• (proxy fix) - call connect on prisma client when running setup by @ishaan-jaff in #6534
• Add 3.5 haiku by @paul-gauthier in #6588
• Litellm perf improvements 3 by @krrishdholakia in #6573
• (fix) /image/generation - ImageObject conversion when content_filter_results exists by @ishaan-jaff in #6584
• (fix) litellm.text_completion raises a non-blocking error on simple usage by @ishaan-jaff in #6546
• (feat) add Predicted Outputs for OpenAI by @ishaan-jaff in #6594
• (fix) Vertex Improve Performance when using image_url by @ishaan-jaff in #6593
• fix(lowest_tpm_rpm_routing.py): fix parallel rate limit check by @krrishdholakia in #6577

New Contributors

• @emmanuel-ferdman made their first contribution in #6554

Full Changelog: v1.51.3...v1.52.0

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	210.0	231.0704959909717	6.291122309918775	0.0	1883	0	180.74613400000317	2618.3897060000163
Aggregated	Passed ✅	210.0	231.0704959909717	6.291122309918775	0.0	1883	0	180.74613400000317	2618.3897060000163

v1.51.3-dev1

What's Changed

• LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
• Litellm dev 11 02 2024 by @krrishdholakia in #6561

Full Changelog: v1.51.3...v1.51.3-dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.3-dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	292.3714877928421	6.163980651581093	0.0	1844	0	226.11442700002726	2207.691740000001
Aggregated	Passed ✅	250.0	292.3714877928421	6.163980651581093	0.0	1844	0	226.11442700002726	2207.691740000001