Simplifying swapping between Anthropic / Deepseek thinking models

[krrishdholakia]

Starting a discussion to understand how we can simplify swapping between anthropic and deepseek thinking models.

Problem:

• Anthropic requires a thinking={"type": "enabled", "budget_tokens": [value]} param to be passed in
• Anthropic signs thinking blocks with a base64 signature, to validate it's from them

Potential Ideas:

• for problem 1: Expose a anthropic/thinking_enabled/<model> route -> automatically passes the flag
• for problem 2: If reasoning_content in message (eg. from deepseek) - put as assistant content message to anthropic api.

Thought: It would be interesting to expore how openrouter / other unified endpoints approach this, to see if there are learnings we can incorporate.

Ideal Interface

from litellm import completion
import os

## set ENV variables
os.environ["ANTHROPIC_API_KEY"] = "your-api-key"
os.environ["DEEPSEEK_API_KEY"] = "your-api-key"

response = completion(
  model="anthropic/thinking_enabled/claude-3-7-sonnet-latest",
  messages=[{ "content": "Hello, how are you?","role": "user"}]
)

print(response)


response = completion(
  model="deepseek/deepseek-chat",
  messages=[{ "content": "Hello, how are you?","role": "user"}]
)

print(response)

cc: @maykcaldas @iurimateus @xingyaoww

[iurimateus]

Note: openrouter/deepseek/deepseek-r1 uses reasoning and not reasoning_content.

[maykcaldas]

Hey all,

Just listing here, it seems that the current implemented interfaces are:

• openrouter/deepseek uses reasoning = {effort: 'high | medium | low', ...} and return ['choices'][0]['message']['reasoning']: https://openrouter.ai/docs/use-cases/reasoning-tokens

• anthropic models use thinking={enable: True} and returns content where an entry has type thinking: https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking

Probably in the future, new providers will implement their own keywords, and it'll take a while until we have an accepted interface. I was wondering if litellm could have its own interface, for instance:

response = completion(
  model="provider/model",
  messages=[{ "content": "Hello, how are you?","role": "user"}],
  reasoning={"foo": bar}
)

And reasoning here is converted to the correct provider keyword to pass the dict downstream. I don't know if it's too troublesome to maintain a list of providers' accepted keywords, but that would make litellm usage more consistent, and it'd be easier for the user to change between models.