Releases · BerriAI/litellm

26 Oct 07:51

github-actions

v1.51.0-stable

b3141e1

v1.51.0-stable

What's Changed

perf: remove 'always_read_redis' - adding +830ms on each llm call by @krrishdholakia in #6414
feat(litellm_logging.py): refactor standard_logging_payload function … by @krrishdholakia in #6388
LiteLLM Minor Fixes & Improvements (10/23/2024) by @krrishdholakia in #6407
allow configuring httpx hooks for AsyncHTTPHandler (#6290) by @krrishdholakia in #6415
feat(proxy_server.py): check if views exist on proxy server startup +… by @krrishdholakia in #6360
feat(litellm_pre_call_utils.py): support 'add_user_information_to_llm… by @krrishdholakia in #6390
(admin ui) - show created_at for virtual keys by @ishaan-jaff in #6429
(feat) track created_at, updated_at for virtual keys by @ishaan-jaff in #6428
Code cov - add checks for patch and overall repo by @ishaan-jaff in #6436
(admin ui / auth fix) Allow internal user to call /key/{token}/regenerate by @ishaan-jaff in #6430
LiteLLM Minor Fixes & Improvements (10/24/2024) by @krrishdholakia in #6421
(proxy audit logs) fix serialization error on audit logs by @ishaan-jaff in #6433

Full Changelog: v1.50.4...v1.51.0-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.0-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	220.0	259.348547705819	6.147561516829862	0.0	1839	0	207.74116500001583	1588.2848330000456
Aggregated	Passed ✅	220.0	259.348547705819	6.147561516829862	0.0	1839	0	207.74116500001583	1588.2848330000456

Contributors

krrishdholakia and ishaan-jaff

Assets 4

26 Oct 06:55

github-actions

v1.51.0

b3141e1

v1.51.0

What's Changed

perf: remove 'always_read_redis' - adding +830ms on each llm call by @krrishdholakia in #6414
feat(litellm_logging.py): refactor standard_logging_payload function … by @krrishdholakia in #6388
LiteLLM Minor Fixes & Improvements (10/23/2024) by @krrishdholakia in #6407
allow configuring httpx hooks for AsyncHTTPHandler (#6290) by @krrishdholakia in #6415
feat(proxy_server.py): check if views exist on proxy server startup +… by @krrishdholakia in #6360
feat(litellm_pre_call_utils.py): support 'add_user_information_to_llm… by @krrishdholakia in #6390
(admin ui) - show created_at for virtual keys by @ishaan-jaff in #6429
(feat) track created_at, updated_at for virtual keys by @ishaan-jaff in #6428
Code cov - add checks for patch and overall repo by @ishaan-jaff in #6436
(admin ui / auth fix) Allow internal user to call /key/{token}/regenerate by @ishaan-jaff in #6430
LiteLLM Minor Fixes & Improvements (10/24/2024) by @krrishdholakia in #6421
(proxy audit logs) fix serialization error on audit logs by @ishaan-jaff in #6433

Full Changelog: v1.50.4...v1.51.0

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.51.0

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	230.0	256.2776533033099	6.163517714105049	0.0	1843	0	210.4747610000004	1438.3136239999885
Aggregated	Passed ✅	230.0	256.2776533033099	6.163517714105049	0.0	1843	0	210.4747610000004	1438.3136239999885

Contributors

krrishdholakia and ishaan-jaff

Assets 4

24 Oct 19:21

github-actions

v1.50.4-stable

0f0470f

v1.50.4-stable

What's Changed

(feat) Arize - Allow using Arize HTTP endpoint by @ishaan-jaff in #6364
LiteLLM Minor Fixes & Improvements (10/22/2024) by @krrishdholakia in #6384
build(deps): bump http-proxy-middleware from 2.0.6 to 2.0.7 in /docs/my-website by @dependabot in #6395
(docs + testing) Correctly document the timeout value used by litellm proxy is 6000 seconds + add to best practices for prod by @ishaan-jaff in #6339
(refactor) move convert dict to model response to llm_response_utils/ by @ishaan-jaff in #6393
(refactor) litellm.Router client initialization utils by @ishaan-jaff in #6394
(fix) Langfuse key based logging by @ishaan-jaff in #6372
Revert "(refactor) litellm.Router client initialization utils " by @ishaan-jaff in #6403
(fix) using /completions with echo by @ishaan-jaff in #6401
(refactor) prometheus async_log_success_event to be under 100 LOC by @ishaan-jaff in #6416
(refactor) router - use static methods for client init utils by @ishaan-jaff in #6420
(code cleanup) remove unused and undocumented logging integrations - litedebugger, berrispend by @ishaan-jaff in #6406

Full Changelog: v1.50.2...v1.50.4-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.4-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	250.0	282.12398419383265	6.070437193170591	0.0	1816	0	215.3092099999867	6805.4257369999505
Aggregated	Passed ✅	250.0	282.12398419383265	6.070437193170591	0.0	1816	0	215.3092099999867	6805.4257369999505

Contributors

krrishdholakia, dependabot, and ishaan-jaff

Assets 4

24 Oct 18:27

github-actions

v1.50.4

0f0470f

v1.50.4

What's Changed

(feat) Arize - Allow using Arize HTTP endpoint by @ishaan-jaff in #6364
LiteLLM Minor Fixes & Improvements (10/22/2024) by @krrishdholakia in #6384
build(deps): bump http-proxy-middleware from 2.0.6 to 2.0.7 in /docs/my-website by @dependabot in #6395
(docs + testing) Correctly document the timeout value used by litellm proxy is 6000 seconds + add to best practices for prod by @ishaan-jaff in #6339
(refactor) move convert dict to model response to llm_response_utils/ by @ishaan-jaff in #6393
(refactor) litellm.Router client initialization utils by @ishaan-jaff in #6394
(fix) Langfuse key based logging by @ishaan-jaff in #6372
Revert "(refactor) litellm.Router client initialization utils " by @ishaan-jaff in #6403
(fix) using /completions with echo by @ishaan-jaff in #6401
(refactor) prometheus async_log_success_event to be under 100 LOC by @ishaan-jaff in #6416
(refactor) router - use static methods for client init utils by @ishaan-jaff in #6420
(code cleanup) remove unused and undocumented logging integrations - litedebugger, berrispend by @ishaan-jaff in #6406

Full Changelog: v1.50.2...v1.50.4

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.4

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	280.0	312.6482922531862	6.037218908394318	0.0	1805	0	231.8999450000092	2847.2051709999846
Aggregated	Failed ❌	280.0	312.6482922531862	6.037218908394318	0.0	1805	0	231.8999450000092	2847.2051709999846

Contributors

krrishdholakia, dependabot, and ishaan-jaff

Assets 4

23 Oct 12:43

github-actions

v1.50.1.dev1

ab8a0e9

v1.50.1.dev1

What's Changed

(fix) get_response_headers for Azure OpenAI by @ishaan-jaff in #6344
fix(litellm-helm): correctly use dbReadyImage and dbReadyTag values by @Hexoplon in #6336
fix(proxy_server.py): add 'admin' user to db by @krrishdholakia in #6223
refactor(redis_cache.py): use a default cache value when writing to r… by @krrishdholakia in #6358
LiteLLM Minor Fixes & Improvements (10/21/2024) by @krrishdholakia in #6352

New Contributors

@Hexoplon made their first contribution in #6336

Full Changelog: v1.50.1...v1.50.1.dev1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1.dev1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	200.0	220.3880747854055	6.181213384368117	0.0	1850	0	179.4118180000055	2854.2284040000254
Aggregated	Passed ✅	200.0	220.3880747854055	6.181213384368117	0.0	1850	0	179.4118180000055	2854.2284040000254

Contributors

krrishdholakia, ishaan-jaff, and Hexoplon

Assets 4

23 Oct 00:19

github-actions

v1.50.2-stable

f943410

v1.50.2-stable

What's Changed

(fix) get_response_headers for Azure OpenAI by @ishaan-jaff in #6344
fix(litellm-helm): correctly use dbReadyImage and dbReadyTag values by @Hexoplon in #6336
fix(proxy_server.py): add 'admin' user to db by @krrishdholakia in #6223
refactor(redis_cache.py): use a default cache value when writing to r… by @krrishdholakia in #6358
LiteLLM Minor Fixes & Improvements (10/21/2024) by @krrishdholakia in #6352
Refactor: apply early return by @Haknt in #6369
(refactor) remove berrispendLogger - unused logging integration by @ishaan-jaff in #6363
(fix) standard logging metadata + add unit testing by @ishaan-jaff in #6366
Revert "(fix) standard logging metadata + add unit testing " by @ishaan-jaff in #6381
Add new Claude 3.5 sonnet model card by @lowjiansheng in #6378
Add claude 3 5 sonnet 20241022 models for all provides by @Manouchehri in #6380

New Contributors

@Hexoplon made their first contribution in #6336
@Haknt made their first contribution in #6369

Full Changelog: v1.50.1...v1.50.2-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.2-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.2-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	190.0	210.17145067557462	6.253172473880384	0.0	1871	0	177.3328190000143	1816.3144349999811
Aggregated	Passed ✅	190.0	210.17145067557462	6.253172473880384	0.0	1871	0	177.3328190000143	1816.3144349999811

Contributors

Haknt, Manouchehri, and 4 other contributors

Assets 4

22 Oct 23:22

github-actions

v1.50.2

f943410

v1.50.2

What's Changed

(fix) get_response_headers for Azure OpenAI by @ishaan-jaff in #6344
fix(litellm-helm): correctly use dbReadyImage and dbReadyTag values by @Hexoplon in #6336
fix(proxy_server.py): add 'admin' user to db by @krrishdholakia in #6223
refactor(redis_cache.py): use a default cache value when writing to r… by @krrishdholakia in #6358
LiteLLM Minor Fixes & Improvements (10/21/2024) by @krrishdholakia in #6352
Refactor: apply early return by @Haknt in #6369
(refactor) remove berrispendLogger - unused logging integration by @ishaan-jaff in #6363
(fix) standard logging metadata + add unit testing by @ishaan-jaff in #6366
Revert "(fix) standard logging metadata + add unit testing " by @ishaan-jaff in #6381
Add new Claude 3.5 sonnet model card by @lowjiansheng in #6378
Add claude 3 5 sonnet 20241022 models for all provides by @Manouchehri in #6380

New Contributors

@Hexoplon made their first contribution in #6336
@Haknt made their first contribution in #6369

Full Changelog: v1.50.1...v1.50.2

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.2

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	240.0	271.2844291307854	6.2111756488034775	0.0	1858	0	210.62568199999987	3226.4373430000433
Aggregated	Passed ✅	240.0	271.2844291307854	6.2111756488034775	0.0	1858	0	210.62568199999987	3226.4373430000433

Contributors

Haknt, Manouchehri, and 4 other contributors

Assets 4

23 Oct 10:49

github-actions

v1.50.1-stable.1

c8d248c

v1.50.1-stable.1

What's Changed

fix(anthropic/chat/transformation.py): fix anthropic header [STABLE BRANCH] by @krrishdholakia in #6365

Full Changelog: v1.50.1...v1.50.1-stable.1

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1-stable.1

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	190.0	209.56840744045144	6.231012229664211	0.0	1864	0	177.2575180000331	3604.4288230000348
Aggregated	Passed ✅	190.0	209.56840744045144	6.231012229664211	0.0	1864	0	177.2575180000331	3604.4288230000348

Contributors

krrishdholakia

Assets 4

21 Oct 17:23

github-actions

v1.50.1.dev4

41f96d9

v1.50.1.dev4

Full Changelog: v1.50.1...v1.50.1.dev4

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1.dev4

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Failed ❌	270.0	318.21334593326156	6.115601652019426	0.0	1828	0	235.8845429999974	3021.9188690000465
Aggregated	Failed ❌	270.0	318.21334593326156	6.115601652019426	0.0	1828	0	235.8845429999974	3021.9188690000465

Assets 4

21 Oct 13:08

github-actions

v1.50.1-stable

fb523b7

v1.50.1-stable

What's Changed

doc - using gpt-4o-audio-preview by @ishaan-jaff in #6326
(refactor) get_cache_key to be under 100 LOC function by @ishaan-jaff in #6327
Litellm openai audio streaming by @krrishdholakia in #6325
LiteLLM Minor Fixes & Improvements (10/18/2024) by @krrishdholakia in #6320
LiteLLM Minor Fixes & Improvements (10/19/2024) by @krrishdholakia in #6331
fix - unhandled jsonDecodeError in convert_to_model_response_object by @ishaan-jaff in #6338
(testing) add test coverage for init custom logger class by @ishaan-jaff in #6341

Full Changelog: v1.50.0...v1.50.1-stable

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Docker Run LiteLLM Proxy

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:main-v1.50.1-stable

Don't want to maintain your internal proxy? get in touch 🎉

Hosted Proxy Alpha: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat

Load Test LiteLLM Proxy Results

Name	Status	Median Response Time (ms)	Average Response Time (ms)	Requests/s	Failures/s	Request Count	Failure Count	Min Response Time (ms)	Max Response Time (ms)
/chat/completions	Passed ✅	270.0	297.1474404657832	6.054198370866852	0.0	1812	0	229.8348699999906	1982.3816379999926
Aggregated	Passed ✅	270.0	297.1474404657832	6.054198370866852	0.0	1812	0	229.8348699999906	1982.3816379999926

Contributors

krrishdholakia and ishaan-jaff

Assets 4

Releases: BerriAI/litellm

v1.51.0-stable

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.51.0

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.50.4-stable

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.50.4

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.50.1.dev1

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.50.2-stable

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.50.2

What's Changed

New Contributors

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.50.1-stable.1

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors

v1.50.1.dev4

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

v1.50.1-stable

What's Changed

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Docker Run LiteLLM Proxy

Don't want to maintain your internal proxy? get in touch 🎉

Load Test LiteLLM Proxy Results

Contributors