Releases: BerriAI/litellm
Releases · BerriAI/litellm
v1.52.0-stable
What's Changed
- LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
- Litellm dev 11 02 2024 by @krrishdholakia in #6561
- build(deps): bump cookie and express in /docs/my-website by @dependabot in #6566
- docs(virtual_keys.md): update Dockerfile reference by @emmanuel-ferdman in #6554
- (proxy fix) - call connect on prisma client when running setup by @ishaan-jaff in #6534
- Add 3.5 haiku by @paul-gauthier in #6588
- Litellm perf improvements 3 by @krrishdholakia in #6573
- (fix) /image/generation - ImageObject conversion when
content_filter_resultsexists by @ishaan-jaff in #6584 - (fix) litellm.text_completion raises a non-blocking error on simple usage by @ishaan-jaff in #6546
- (feat) add
Predicted Outputsfor OpenAI by @ishaan-jaff in #6594 - (fix) Vertex Improve Performance when using
image_urlby @ishaan-jaff in #6593 - fix(lowest_tpm_rpm_routing.py): fix parallel rate limit check by @krrishdholakia in #6577
New Contributors
- @emmanuel-ferdman made their first contribution in #6554
Full Changelog: v1.51.3...v1.52.0-stable
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0-stable
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 240.0 | 270.29554346208295 | 6.124428891308869 | 0.0 | 1833 | 0 | 212.83106800001406 | 1622.2440090000418 |
| Aggregated | Passed ✅ | 240.0 | 270.29554346208295 | 6.124428891308869 | 0.0 | 1833 | 0 | 212.83106800001406 | 1622.2440090000418 |
Contributors
krrishdholakia, dependabot, and 3 other contributors
Assets 4
v1.52.0
What's Changed
- LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
- Litellm dev 11 02 2024 by @krrishdholakia in #6561
- build(deps): bump cookie and express in /docs/my-website by @dependabot in #6566
- docs(virtual_keys.md): update Dockerfile reference by @emmanuel-ferdman in #6554
- (proxy fix) - call connect on prisma client when running setup by @ishaan-jaff in #6534
- Add 3.5 haiku by @paul-gauthier in #6588
- Litellm perf improvements 3 by @krrishdholakia in #6573
- (fix) /image/generation - ImageObject conversion when
content_filter_resultsexists by @ishaan-jaff in #6584 - (fix) litellm.text_completion raises a non-blocking error on simple usage by @ishaan-jaff in #6546
- (feat) add
Predicted Outputsfor OpenAI by @ishaan-jaff in #6594 - (fix) Vertex Improve Performance when using
image_urlby @ishaan-jaff in #6593 - fix(lowest_tpm_rpm_routing.py): fix parallel rate limit check by @krrishdholakia in #6577
New Contributors
- @emmanuel-ferdman made their first contribution in #6554
Full Changelog: v1.51.3...v1.52.0
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.52.0
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 210.0 | 231.0704959909717 | 6.291122309918775 | 0.0 | 1883 | 0 | 180.74613400000317 | 2618.3897060000163 |
| Aggregated | Passed ✅ | 210.0 | 231.0704959909717 | 6.291122309918775 | 0.0 | 1883 | 0 | 180.74613400000317 | 2618.3897060000163 |
Contributors
krrishdholakia, dependabot, and 3 other contributors
Assets 4
v1.51.3-dev1
What's Changed
- LiteLLM Minor Fixes & Improvements (11/01/2024) by @krrishdholakia in #6551
- Litellm dev 11 02 2024 by @krrishdholakia in #6561
Full Changelog: v1.51.3...v1.51.3-dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.3-dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 292.3714877928421 | 6.163980651581093 | 0.0 | 1844 | 0 | 226.11442700002726 | 2207.691740000001 |
| Aggregated | Passed ✅ | 250.0 | 292.3714877928421 | 6.163980651581093 | 0.0 | 1844 | 0 | 226.11442700002726 | 2207.691740000001 |
Contributors
krrishdholakia
Assets 4
v1.51.3
What's Changed
- Support specifying exponential backoff retry strategy when calling completions() by @dbczumar in #6520
- (fix) slack alerting - don't spam the failed cost tracking alert for the same model by @ishaan-jaff in #6543
- (feat) add XAI ChatCompletion Support by @ishaan-jaff in #6373
- LiteLLM Minor Fixes & Improvements (10/30/2024) by @krrishdholakia in #6519
Full Changelog: v1.51.2...v1.51.3
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.3
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 200.0 | 220.3819331893052 | 6.253936592654308 | 0.0 | 1870 | 0 | 179.7343989999831 | 3185.1700670000014 |
| Aggregated | Passed ✅ | 200.0 | 220.3819331893052 | 6.253936592654308 | 0.0 | 1870 | 0 | 179.7343989999831 | 3185.1700670000014 |
Contributors
krrishdholakia, ishaan-jaff, and dbczumar
Assets 4
v1.51.1-stable
What's Changed
- (UI) Delete Internal Users on Admin UI by @ishaan-jaff in #6442
- (testing) increase prometheus.py test coverage to 90% by @ishaan-jaff in #6466
- (Feat) New Logging integration - add Datadog LLM Observability support by @ishaan-jaff in #6449
- (feat) add pricing for amazon.titan-embed-image-v1 by @ishaan-jaff in #6444
- LiteLLM Minor Fixes & Improvements (10/24/2024) by @krrishdholakia in #6441
- Litellm dev 10 26 2024 by @krrishdholakia in #6472
- (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected by @ishaan-jaff in #6471
- redis otel tracing + async support for latency routing by @krrishdholakia in #6452
- (fix) Prometheus - Log Postgres DB latency, status on prometheus by @ishaan-jaff in #6484
- (router_strategy/) ensure all async functions use async cache methods by @ishaan-jaff in #6489
- (fix) proxy - fix when
STORE_MODEL_IN_DBshould be set by @ishaan-jaff in #6492 - (fix)
PrometheusServicesLogger_get_metricshould return metric in Registry by @ishaan-jaff in #6486 - Add
azure/gpt-4o-mini-2024-07-18to model_prices_and_context_window… by @xingyaoww in #6477 - Update utils.py by @vibhanshu-ob in #6468
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:litellm_stable_nov1-v1.51.1
Full Changelog: v1.51.2...v1.51.1-stable
Contributors
krrishdholakia, ishaan-jaff, and 2 other contributors
Assets 2
custom-docker-image-test-dev
What's Changed
- Support specifying exponential backoff retry strategy when calling completions() by @dbczumar in #6520
Full Changelog: v1.51.2...custom-docker-image-test-dev
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-custom-docker-image-test-dev
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 277.2980743547866 | 6.178135563258271 | 0.0 | 1849 | 0 | 222.02958399998352 | 3219.9342000000115 |
| Aggregated | Passed ✅ | 250.0 | 277.2980743547866 | 6.178135563258271 | 0.0 | 1849 | 0 | 222.02958399998352 | 3219.9342000000115 |
Contributors
dbczumar
Assets 4
v1.51.2
What's Changed
- (perf) Litellm redis router fix - ~100ms improvement by @krrishdholakia in #6483
- LiteLLM Minor Fixes & Improvements (10/28/2024) by @krrishdholakia in #6475
- Litellm dev 10 29 2024 by @krrishdholakia in #6502
- Litellm router max depth by @krrishdholakia in #6501
- (UI) fix bug with rendering max budget = 0 by @ishaan-jaff in #6506
- (UI) fix + test displaying number of keys an internal user owns by @ishaan-jaff in #6507
- (UI) Fix viewing members, keys in a team + added testing by @ishaan-jaff in #6514
Full Changelog: v1.51.1...v1.51.2
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.2
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 250.0 | 287.73103211135674 | 6.039141910660648 | 0.0 | 1805 | 0 | 213.5724959999834 | 2930.6253560000073 |
| Aggregated | Passed ✅ | 250.0 | 287.73103211135674 | 6.039141910660648 | 0.0 | 1805 | 0 | 213.5724959999834 | 2930.6253560000073 |
Contributors
krrishdholakia and ishaan-jaff
Assets 4
v1.51.1-staging
What's Changed
- (perf) Litellm redis router fix - ~100ms improvement by @krrishdholakia in #6483
- LiteLLM Minor Fixes & Improvements (10/28/2024) by @krrishdholakia in #6475
- Litellm dev 10 29 2024 by @krrishdholakia in #6502
- Litellm router max depth by @krrishdholakia in #6501
- (UI) fix bug with rendering max budget = 0 by @ishaan-jaff in #6506
- (UI) fix + test displaying number of keys an internal user owns by @ishaan-jaff in #6507
- (UI) Fix viewing members, keys in a team + added testing by @ishaan-jaff in #6514
Full Changelog: v1.51.1...v1.51.1-staging
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.1-staging
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 270.0 | 311.93605914725106 | 6.080288332872121 | 0.0033408177653143525 | 1820 | 1 | 117.93499300000576 | 3293.080912999983 |
| Aggregated | Failed ❌ | 270.0 | 311.93605914725106 | 6.080288332872121 | 0.0033408177653143525 | 1820 | 1 | 117.93499300000576 | 3293.080912999983 |
Contributors
krrishdholakia and ishaan-jaff
Assets 4
v1.51.1
What's Changed
- (UI) Delete Internal Users on Admin UI by @ishaan-jaff in #6442
- (testing) increase prometheus.py test coverage to 90% by @ishaan-jaff in #6466
- (Feat) New Logging integration - add Datadog LLM Observability support by @ishaan-jaff in #6449
- (feat) add pricing for amazon.titan-embed-image-v1 by @ishaan-jaff in #6444
- LiteLLM Minor Fixes & Improvements (10/24/2024) by @krrishdholakia in #6441
- Litellm dev 10 26 2024 by @krrishdholakia in #6472
- (Testing) Add unit testing for DualCache - ensure in memory cache is used when expected by @ishaan-jaff in #6471
- redis otel tracing + async support for latency routing by @krrishdholakia in #6452
- (fix) Prometheus - Log Postgres DB latency, status on prometheus by @ishaan-jaff in #6484
- (router_strategy/) ensure all async functions use async cache methods by @ishaan-jaff in #6489
- (fix) proxy - fix when
STORE_MODEL_IN_DBshould be set by @ishaan-jaff in #6492 - (fix)
PrometheusServicesLogger_get_metricshould return metric in Registry by @ishaan-jaff in #6486 - Add
azure/gpt-4o-mini-2024-07-18to model_prices_and_context_window… by @xingyaoww in #6477 - Update utils.py by @vibhanshu-ob in #6468
New Contributors
- @xingyaoww made their first contribution in #6477
- @vibhanshu-ob made their first contribution in #6468
Full Changelog: v1.51.0...v1.51.1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Passed ✅ | 200.0 | 217.96900764879396 | 6.232037758758388 | 0.0 | 1865 | 0 | 178.63059899997324 | 1220.8741560000362 |
| Aggregated | Passed ✅ | 200.0 | 217.96900764879396 | 6.232037758758388 | 0.0 | 1865 | 0 | 178.63059899997324 | 1220.8741560000362 |
Contributors
krrishdholakia, ishaan-jaff, and 2 other contributors
Assets 4
v1.51.0.dev1
Full Changelog: v1.51.0...v1.51.0.dev1
Docker Run LiteLLM Proxy
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.0.dev1
Don't want to maintain your internal proxy? get in touch 🎉
Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat
Load Test LiteLLM Proxy Results
| Name | Status | Median Response Time (ms) | Average Response Time (ms) | Requests/s | Failures/s | Request Count | Failure Count | Min Response Time (ms) | Max Response Time (ms) |
|---|---|---|---|---|---|---|---|---|---|
| /chat/completions | Failed ❌ | 270.0 | 304.38755454794625 | 6.1009349714042544 | 0.0 | 1825 | 0 | 232.0200620000037 | 3500.7745139999997 |
| Aggregated | Failed ❌ | 270.0 | 304.38755454794625 | 6.1009349714042544 | 0.0 | 1825 | 0 | 232.0200620000037 | 3500.7745139999997 |