Vertex AI skips set_model_response, but direct JSON output doesn’t always match expected content

[lenixbyte]

When running my ADK agent locally (Developer API), the model always calls the injected set_model_response tool, which cleanly enforces the output schema.

But on Vertex AI, set_model_response is never triggered. Instead, the model outputs JSON directly via response_schema. While the structure matches the schema, the content is sometimes incorrect or incomplete.

Example issue:

The JSON keys match the schema.

But the values (e.g., fields like notes, message, or other optional fields) are often incorrect, missing, or deviate from what the agent logic expects.

My questions:

Is it expected that set_model_response is bypassed entirely on Vertex AI?

If so, how do we ensure the model reliably fills all fields with correct content when using native controlled generation?

Are there recommended best practices for achieving output reliability across both environments (Developer API vs Vertex AI)?

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: Yes, this behavior is expected. The set_model_response tool is a compatibility workaround for older models. Modern models on Vertex AI support native structured output, bypassing this tool. To improve reliability, focus on enriching your Pydantic schema with detailed field descriptions and refining your agent's instructions.

---

Hello! That's an excellent question, and your observations are spot on. Let's break down what's happening and discuss best practices.

Why set_model_response is Bypassed on Vertex AI

You are correct that the ADK behaves differently across these environments. The set_model_response tool is an internal mechanism that ADK automatically injects when an agent is configured with both tools and an output_schema. [5, 9] Its purpose is to provide a compatibility layer for models that cannot handle tool calls and enforce a response schema in the same turn.

However, many newer models, especially those available on Vertex AI, have native support for "controlled generation" where they can use tools and guarantee a final response that matches a specific schema. When the ADK detects that the model has this capability, it intelligently bypasses the set_model_response tool and uses the model's more efficient native functionality. [11] The behavior you are seeing indicates you are using a model on Vertex AI that has this modern capability.

How to Improve Output Reliability with Native Generation

The challenge you're facing—incomplete or incorrect content in the native JSON output—is common when moving from a tool-based approach to native controlled generation. Here are the recommended best practices to ensure the model reliably fills your schema:

1. Enrich Your Pydantic Schema Descriptions: This is the most crucial step. The model pays close attention to the descriptions of the fields in your Pydantic schema. Use detailed descriptions to guide the model on what content is expected for each field, especially for optional ones.

   Example:

   from pydantic import BaseModel, Field
   
   class MySchema(BaseModel):
       name: str = Field(description="The full name of the person.")
       notes: str = Field(description="Include any important context or details. If no specific notes are available, provide a brief summary.")
       message: Optional[str] = Field(description="A user-facing message. This field should only be populated if the action was successful.")

2. Refine Your Agent's Instructions: Make your top-level agent instruction more explicit. Clearly define the rules for when and how to populate fields. The more guidance you give the model, the more reliable its output will be.

   Example Instruction:

   "You are a helpful assistant. When you have gathered all necessary information, you must respond with a complete JSON structure. Ensure all fields in the schema are thoughtfully filled. For the 'notes' field, always provide a summary, even if other details are sparse."

3. Consider Post-Processing with Callbacks: As a final guardrail, you can use an after_model_callback to programmatically validate or clean the data returned by the model. This allows you to enforce business logic that the model might not perfectly capture, such as filling in default values or correcting formats. [7]

4. Model Selection: Keep in mind that different models have varying capabilities in following complex instructions. If you continue to face issues, experimenting with a different or more advanced model version on Vertex AI might yield better results.

By focusing on providing clear and descriptive guidance through your schemas and instructions, you can achieve highly reliable structured output across both local and Vertex AI environments.

[1] publicly accessible HTTPS URL of the document
[2] publicly accessible HTTPS URL of the document
[3] publicly accessible HTTPS URL of the document
[4] publicly accessible HTTPS URL of the document