Is there any way to retry the last tool call or task when LLM call throws error?

[ynaiborlang]

So I'm creating an agent that calls multiple tools in succession, and there are times when the model is Overloaded and throws an error and breaks the workflow. Is there a way to catch the error and resume the rest of the tool call or task.

[adk-bot]

Response from ADK Answering Agent (experimental, answer may be inaccurate)

TLDR: You can handle errors and resume your workflow by using the on_tool_error callback to catch exceptions during tool execution. Additionally, you can enable session resumption for better reliability during streaming conversations and implement error handling within your tool's logic to return structured error messages.

---

Hi @ynaiborlang, that's a great question. Dealing with intermittent errors is a common challenge when building robust agents. The ADK provides several mechanisms to handle these situations:

1. Handling Tool Execution Errors with on_tool_error

The most direct way to catch errors from a specific tool call is to use the on_tool_error callback. This callback is triggered whenever an exception occurs during the execution of a tool. You can use it to log the error, inform the user, or even attempt a retry.

Here’s an example of how you can set it up in your agent's configuration:

from google.adk.services.tool_manager import ToolCode, ToolManager

def my_tool_error_callback(tool_code: ToolCode, error: Exception):
    print(f"Error in tool {tool_code.tool_name}: {error}")
    # You could add retry logic here
    return {"status": "error", "error_message": str(error)}

# In your agent configuration
"tool_config": {
    "callbacks": {
        "on_tool_error": my_tool_error_callback
    }
}

By returning a dictionary with an error message, you can let the LLM know that something went wrong, and it might be able to decide whether to retry or to try a different approach. You can find more about tool callbacks in the documentation. [1]

2. Session Resumption for Streaming

If you are using streaming, for instance with WebSockets, you can enable session resumption. This feature helps to make your agent more reliable, especially in cases of network issues. It allows the agent to resume the conversation from where it left off. [2]

To enable it, you need to configure a ResumptionEngine in your StreamingController. You can see an example of this in the custom streaming with WebSockets documentation. [3]

3. Best Practices for Error Handling in Tools

It's also a good practice to build error handling directly into your tool's implementation. For example, if your tool makes an API call, you should wrap it in a try...except block and handle potential exceptions like network errors or API-specific errors. Instead of letting the exception crash the tool, you can return a dictionary with an error status and a descriptive message. [4]

For example:

def my_api_tool(param1):
    try:
        # Make an API call
        result = some_api_call(param1)
        return {"status": "success", "data": result}
    except Exception as e:
        return {"status": "error", "error_message": f"API call failed: {e}"}

This gives the LLM context about the error and allows it to proceed with the workflow in a more intelligent way.

4. Automatic Retries for Authentication

The ADK also has built-in support for retrying a tool call automatically after a successful authentication flow. This is a specific case, but it shows that the ADK is designed with resilience in mind. [5]

By combining these approaches, you can build a more robust agent that can handle errors gracefully and continue its workflow. I hope this helps you with your project!

[1] https://github.com/google/adk-python/blob/main/contributing/samples/live_tool_callbacks_agent/readme.md
[2] https://google.github.io/adk-docs/streaming/custom-streaming/
[3] https://google.github.io/adk-docs/streaming/custom-streaming-ws/
[4] https://google.github.io/adk-docs/tools/
[5] https://google.github.io/adk-docs/tools/authentication/

[ynaiborlang]

ok now what if the LLM calls throws an error like "the model is overloaded, please try again later" or throws an error while trying to transfer to another agent

[boyangsvl]

You can set the retry config on Gemini model so it retries automatically for you. https://github.com/google/adk-python/blob/main/src/google/adk/models/google_llm.py#L62

[ynaiborlang]

@boyangsvl , Thank you this is exactly what I was looking for