Skip to content

Conversation

AlexsanderHamir
Copy link
Collaborator

@AlexsanderHamir AlexsanderHamir commented Oct 7, 2025

Title

[Fix] Networking: remove limitations

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • I have added a screenshot of my new test passing locally
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🧹 Refactoring

Changes

  • Remove low hardcoded defaults.

Performance Improvement

Local networking test

Metric Before (worse) After (better)
Total requests 100 100
Successful 100 100
Failed 0 0
Total time 0.75s 0.50s
Min Latency (ms) 62.3 41.5
Avg Latency (ms) 221.4 147.6
P50 Latency (ms) 151.8 101.2
P95 Latency (ms) 512.3 341.5
P99 Latency (ms) 556.4 370.9
Max Latency (ms) 556.4 370.9
New connections 50 50

Copy link

vercel bot commented Oct 7, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
litellm Ready Ready Preview Comment Oct 7, 2025 11:31pm

Copy link
Contributor

@ishaan-jaff ishaan-jaff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we see the perf impact of this on the PR

Copy link
Contributor

@ishaan-jaff ishaan-jaff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ishaan-jaff
Copy link
Contributor

linting errors: https://github.com/BerriAI/litellm/actions/runs/18328434261/job/52198349192?pr=15302

Copy link
Contributor

@ishaan-jaff ishaan-jaff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ishaan-jaff ishaan-jaff merged commit 49e04e0 into main Oct 7, 2025
29 of 50 checks passed
@6matt 6matt mentioned this pull request Oct 8, 2025
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants