Fix deserialization failure in ChatCompletionMessageToolCallChunk by making index optional #432
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
The async_openai crate enforces strict deserialization for OpenAI-compatible streaming responses, but some APIs (e.g., Google's Gemini API) that partially emulate OpenAI's chat completion format may omit the index field in tool call deltas within streaming chunks. This leads to a Serde deserialization error like failed deserialization of: {...} when parsing responses where the index is not directly present in the tool call object (it's often at the choice level instead).For example, a Gemini tool call chunk might look like:
json { "choices": [ { "delta": { "tool_calls": [ { "function": { ... }, "id": "function-call-...", "type": "function" } ] }, "index": 0, "finish_reason": "tool_calls" } ], ... }
Here, the index is absent from the inner tool_calls array, causing the required u32 field in ChatCompletionMessageToolCallChunk to fail parsing. This breaks streaming workflows for users integrating with hybrid or third-party providers such as gemini.
SolutionChanged
pub index: u32 to pub index: Option<u32> in ChatCompletionMessageToolCallChunk
.This allows graceful handling of missing index values without altering the struct's behavior for standard OpenAI responses (where index is typically provided and can be unwrapped safely).
No other fields or logic were modified, ensuring minimal impact on existing code.
Impact and BenefitsBackward Compatibility: OpenAI responses remain unaffected—index will still deserialize to Some(value).
Broader Compatibility: Enables seamless use with Gemini and potentially other OpenAI-like APIs (e.g., Anthropic, Mistral) that vary slightly in streaming formats. This reduces the need for user-side workarounds or forks.
Robustness: Makes the crate more resilient to real-world API variations, especially in agentic workflows involving tool calls (e.g., function invocations like get_ticket_types in the example error).
Community Value: As AI providers proliferate with OpenAI-inspired APIs, this change future-proofs the library for multi-provider setups, benefiting developers building cross-compatible chat/streaming apps.