Skip to content

fix(llm): separate retry counters in stream() - transient errors used wrong cap#6195

Open
Sri-Likhita-adru wants to merge 1 commit intoaden-hive:mainfrom
Sri-Likhita-adru:fix/stream-transient-retry-cap
Open

fix(llm): separate retry counters in stream() - transient errors used wrong cap#6195
Sri-Likhita-adru wants to merge 1 commit intoaden-hive:mainfrom
Sri-Likhita-adru:fix/stream-transient-retry-cap

Conversation

@Sri-Likhita-adru
Copy link

Closes #6105

## Changes
- Add 'STREAM_TRANSIENT_MAX_RETRIES = 3' constant
- Track 'empty_stream_attempts', 'transient_attempts', 'rate_limit_attempts'  as independent counters in 'stream()'
- Each retry branch checks and increments only its own counter
- Transient errors no longer consume the rate-limit retry budget

## Files Changed
'core/framework/llm/litellm.py'

@github-actions
Copy link

PR Requirements Warning

This PR does not meet the contribution requirements.
If the issue is not fixed within ~24 hours, it may be automatically closed.

PR Author: @Sri-Likhita-adru
Found issues: #6105 (assignees: none)
Problem: The PR author must be assigned to the linked issue.

To fix:

  1. Assign yourself (@Sri-Likhita-adru) to one of the linked issues
  2. Re-open this PR

Exception: To bypass this requirement, you can:

  • Add the micro-fix label or include micro-fix in your PR title for trivial fixes
  • Add the documentation label or include doc/docs in your PR title for documentation changes

Micro-fix requirements (must meet ALL):

Qualifies Disqualifies
< 20 lines changed Any functional bug fix
Typos & Documentation & Linting Refactoring for "clean code"
No logic/API/DB changes New features (even tiny ones)

Why is this required? See #472 for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-requirements-warning PR doesn't follow contribution guidelines. Please fix or it will be auto-closed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant