[Feature] Support response transform of gemini-2.0-flash-thinking-exp-1219

[narengogi]

What Would You Like to See with the Gateway?

This new model from google returns multiple text parts in the response. Sample response

{
  "candidates": [
    {
      "content": {
        "role": "model",
        "parts": [
          {
            "text": "The user asked a basic arithmetic question. I need to calculate the result of 2 + 2."
          },
          {
            "text": "Hi there!\n\n2 + 2 = 4\n"
          }
        ]
      },
      "finishReason": "STOP",
      "avgLogprobs": -0.2444594412138968
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 8,
    "candidatesTokenCount": 33,
    "totalTokenCount": 41
  },
  "modelVersion": "gemini-2.0-flash-thinking-exp-1219"
}

OpenAI schema has only one message per generation

  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hi there! How can I assist you today?",
        "refusal": null
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],

Proposed solution:

There's really no 1:1 mapping for this from the schema, but we could merge two content parts.

{
    "id": "portkey-9418aa95-4c0d-4cad-a924-ddcd4e9489a6",
    "object": "chat_completion",
    "created": 1735213212,
    "model": "Unknown",
    "provider": "vertex-ai",
    "choices": [
        {
            "message": {
                "role": "assistant",
                "content": "\nThe user asked a basic arithmetic question. I need to calculate the result of 2 + 2.\nHi there!\n\n2 + 2 = 4\n"
            },
            "index": 0,
            "finish_reason": "STOP"
        }
    ],
    "usage": {
        "prompt_tokens": 8,
        "completion_tokens": 33,
        "total_tokens": 41
    }
}

Context for your Request

No response

Your Twitter/LinkedIn

No response

[manaschakrabortty]

(Google-style)
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"text": "The user asked a basic arithmetic question. I need to calculate the result of 2 + 2."
},
{
"text": "Hi there!\n\n2 + 2 = 4\n"
}
]
},
"finishReason": "STOP",
"avgLogprobs": -0.2444594412138968
}
],
"usageMetadata": {
"promptTokenCount": 8,
"candidatesTokenCount": 33,
"totalTokenCount": 41
},
"modelVersion": "gemini-2.0-flash-thinking-exp-1219"
}
(OpenAI-style)
{
"id": "portkey-9418aa95-4c0d-4cad-a924-ddcd4e9489a6",
"object": "chat_completion",
"created": 1735213212,
"model": "Unknown",
"provider": "vertex-ai",
"choices": [
{
"message": {
"role": "assistant",
"content": "The user asked a basic arithmetic question. I need to calculate the result of 2 + 2.\nHi there!\n\n2 + 2 = 4\n"
},
"index": 0,
"finish_reason": "STOP"
}
],
"usage": {
"prompt_tokens": 8,
"completion_tokens": 33,
"total_tokens": 41
}
}

[narengogi]

@manaschakrabortty this is how I'm thinking about it too

Alternatively I was thinking if we could return content as multiple parts in the same message like this

"message": {
"role": "assistant",
    "content": [
        {
            "type": "text",
            "text": "The user asked a basic arithmetic question. I need to calculate the result of 2 + 2.\nHi there!"
        },
        {
            "type": "text",
            "text": "2 + 2 = 4\n"
        }
    ],

This isn't openai compliant, but I'm thinking we shouldn't really club reasoning and response tokens together