Skip to content

Conversation

habema
Copy link
Contributor

@habema habema commented Sep 30, 2025

Resolves #1831

Adds cost tracking for LiteLLM models when users enable it via ModelSettings(track_cost=True).

Changes

  • Added cost field to Usage class
  • Added track_cost setting to ModelSettings. Opt-in (defaults to False)
  • LiteLLM cost extraction:
    • Non-streaming: reads from _hidden_params["response_cost"]
    • Streaming: calculates via litellm.completion_cost()
    • Includes cached & reasoning tokens for accurate pricing

@seratch seratch added enhancement New feature or request feature:lite-llm labels Sep 30, 2025
@seratch seratch marked this pull request as draft September 30, 2025 18:12
Copy link
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unlocking the access itself is fine, but we'd like to hold off relying on an internal modules within a different package.

@seratch seratch changed the title Add opt-in cost tracking for LiteLLM models feat: #1831 Add opt-in cost tracking for LiteLLM models Sep 30, 2025
@habema habema marked this pull request as ready for review October 2, 2025 10:36
@habema
Copy link
Contributor Author

habema commented Oct 2, 2025

litellm.completion_cost() is not used for both streaming and non-streaming responses.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request feature:lite-llm
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add cost tracking for LiteLLM models
2 participants