'query-datasource' cannot be bined to llm in langchain interface

[SuleymanSuleymanzade]

Bug with binding 'query-datasource' with llms in Langchain interface

'query-datasource' tool cannot be bounded to the llm models. it shows error (in logs section)
The issue happens only with this tool.

Expected behavior

It must be binded as well as the rest of tools to the llms

Versions

v1.8.0

Steps to reproduce

import asyncio
from datetime import timedelta

import boto3
from langchain_mcp_adapters.client import MultiServerMCPClient
from langchain_aws import BedrockLLM, ChatBedrock, BedrockEmbeddings
from langgraph.prebuilt import create_react_agent

conf = {
    "TableauMCP": {
        "transport": "streamable_http",
        "url": "http://my_ec2_server:3927/tableau-mcp/",
        "timeout": timedelta(seconds=600),
        "headers": None,
        "sse_read_timeout": timedelta(seconds=900)
    }
}

bedrock = boto3.client(
    "bedrock-runtime",
    region_name="us-east-1",
    aws_access_key_id="my_access_key",
    aws_secret_access_key="my_secret_key",
)


llm_haiku = ChatBedrock(
    client=bedrock,
    model_id="us.anthropic.claude-3-haiku-20240307-v1:0",
    model_kwargs={ 
        "max_tokens": 10_000,
        "temperature": 0
    },
    streaming=True
)

client = MultiServerMCPClient(conf)
tools = await client.get_tools() 
llm_haiku.bind_tools([tools[2]]) # only 'query-datasource' tool

Logs

---

KeyError Traceback (most recent call last)
Cell In[167], line 1
----> 1 llm_haiku.bind_tools([tools[2]])

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_aws/chat_models/bedrock.py:669, in ChatBedrock.bind_tools(self, tools, tool_choice, **kwargs)
665 return self._as_converse.bind_tools(
666 tools, tool_choice=tool_choice, **kwargs
667 )
668 if self._get_provider() == "anthropic":
--> 669 formatted_tools = [convert_to_anthropic_tool(tool) for tool in tools]
671 # true if the model is a claude 3 model
672 if "claude-3" in self._get_model():

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_aws/function_calling.py:216, in convert_to_anthropic_tool(tool)
214 return AnthropicTool(tool) # type: ignore
215 else:
--> 216 formatted = convert_to_openai_tool(tool)["function"]
217 return AnthropicTool(
218 name=formatted["name"],
219 description=formatted["description"],
220 input_schema=formatted["parameters"],
221 )

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/function_calling.py:537, in convert_to_openai_tool(tool, strict)
535 if isinstance(tool, dict) and tool.get("type") == "function" and "function" in tool:
536 return tool
--> 537 oai_function = convert_to_openai_function(tool, strict=strict)
538 return {"type": "function", "function": oai_function}

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/function_calling.py:461, in convert_to_openai_function(function, strict)
457 oai_function = cast(
458 dict, _convert_typed_dict_to_openai_function(cast(type, function))
459 )
460 elif isinstance(function, BaseTool):
--> 461 oai_function = cast(dict, _format_tool_to_openai_function(function))
462 elif callable(function):
463 oai_function = cast(dict, _convert_python_function_to_openai_function(function))

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/function_calling.py:319, in _format_tool_to_openai_function(tool)
317 if tool.tool_call_schema and not is_simple_oai_tool:
318 if isinstance(tool.tool_call_schema, dict):
--> 319 return _convert_json_schema_to_openai_function(
320 tool.tool_call_schema, name=tool.name, description=tool.description
321 )
322 elif issubclass(tool.tool_call_schema, (BaseModel, BaseModelV1)):
323 return _convert_pydantic_to_openai_function(
324 tool.tool_call_schema, name=tool.name, description=tool.description
325 )

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/function_calling.py:99, in _convert_json_schema_to_openai_function(schema, name, description, rm_titles)
79 def _convert_json_schema_to_openai_function(
80 schema: dict,
81 *,
(...)
84 rm_titles: bool = True,
85 ) -> FunctionDescription:
86 """Converts a Pydantic model to a function description for the OpenAI API.
87
88 Args:
(...)
97 The function description.
98 """
---> 99 schema = dereference_refs(schema)
100 if "definitions" in schema: # pydantic 1
101 schema.pop("definitions", None)

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/json_schema.py:114, in dereference_refs(schema_obj, full_schema, skip_keys)
100 """Try to substitute $refs in JSON Schema.
101
102 Args:
(...)
108 The dereferenced schema object.
109 """
110 full_schema = full_schema or schema_obj
111 skip_keys = (
112 skip_keys
113 if skip_keys is not None
--> 114 else _infer_skip_keys(schema_obj, full_schema)
115 )
116 return _dereference_refs_helper(schema_obj, full_schema, skip_keys)

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/json_schema.py:87, in _infer_skip_keys(obj, full_schema, processed_refs)
85 keys += _infer_skip_keys(ref, full_schema, processed_refs)
86 elif isinstance(v, (list, dict)):
---> 87 keys += _infer_skip_keys(v, full_schema, processed_refs)
88 elif isinstance(obj, list):
89 for el in obj:

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/json_schema.py:87, in _infer_skip_keys(obj, full_schema, processed_refs)
85 keys += _infer_skip_keys(ref, full_schema, processed_refs)
86 elif isinstance(v, (list, dict)):
---> 87 keys += _infer_skip_keys(v, full_schema, processed_refs)
88 elif isinstance(obj, list):
89 for el in obj:

[... skipping similar frames: _infer_skip_keys at line 87 (3 times)]

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/json_schema.py:87, in _infer_skip_keys(obj, full_schema, processed_refs)
85 keys += _infer_skip_keys(ref, full_schema, processed_refs)
86 elif isinstance(v, (list, dict)):
---> 87 keys += _infer_skip_keys(v, full_schema, processed_refs)
88 elif isinstance(obj, list):
89 for el in obj:

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/json_schema.py:90, in _infer_skip_keys(obj, full_schema, processed_refs)
88 elif isinstance(obj, list):
89 for el in obj:
---> 90 keys += _infer_skip_keys(el, full_schema, processed_refs)
91 return keys

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/json_schema.py:87, in _infer_skip_keys(obj, full_schema, processed_refs)
85 keys += _infer_skip_keys(ref, full_schema, processed_refs)
86 elif isinstance(v, (list, dict)):
---> 87 keys += _infer_skip_keys(v, full_schema, processed_refs)
88 elif isinstance(obj, list):
89 for el in obj:

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/json_schema.py:87, in _infer_skip_keys(obj, full_schema, processed_refs)
85 keys += _infer_skip_keys(ref, full_schema, processed_refs)
86 elif isinstance(v, (list, dict)):
---> 87 keys += _infer_skip_keys(v, full_schema, processed_refs)
88 elif isinstance(obj, list):
89 for el in obj:

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/json_schema.py:83, in _infer_skip_keys(obj, full_schema, processed_refs)
81 continue
82 processed_refs.add(v)
---> 83 ref = _retrieve_ref(v, full_schema)
84 keys.append(v.split("/")[1])
85 keys += _infer_skip_keys(ref, full_schema, processed_refs)

File /opt/conda/envs/scraper/lib/python3.12/site-packages/langchain_core/utils/json_schema.py:26, in _retrieve_ref(path, schema)
24 else:
25 msg = f"Reference '{path}' not found."
---> 26 raise KeyError(msg)
27 return deepcopy(out)

KeyError: "Reference '#/properties/query/properties/fields/items/anyOf/0/properties/fieldCaption' not found."

[anyoung-tableau]

Hey @SuleymanSuleymanzade, thanks for the report. Can you please clarify what the issue is with Tableau MCP? Looks more like an issue in langchain or the Python script.

I see this message in the error:

KeyError: "Reference '#/properties/query/properties/fields/items/anyOf/0/properties/fieldCaption' not found."

The query-datasource tool defines a schema for its parameters that looks like this:

{
  datasourceLuid: string;
  query: {
    fields: Array<Field>;
    filters: <ommitted because irrelevant>
  }
}

where Field is an object that contains a fieldCaption key e.g.

{
  fieldCaption: string;
  fieldAlias: string;
  maxDecimalPlaces: number;
  sortDirection: enum;
  sortPriority: number;
}

I'm not really familiar with langchain, did your script try to call the query-datasource tool but failed to generate an argument that contained a field with a fieldCaption value?

[SuleymanSuleymanzade]

I found out:

when I try to llm.bind_tools([query-datasource]) tool the problem is that LangChain tries to convert your StructuredTool into OpenAI/Anthropic-compatible function schemas, but it chokes while trying to dereference those $ref paths. That’s why there is a failure in:

langchain_core/utils/json_schema.py -> dereference_refs

So, I have to do work around for langchain:

import jsonref
from langchain_core.tools import StructuredTool

# Suppose you already have your tools list
# tools = client.get_tools()   # for example

def flatten_tool_schema(tool):
    """Flatten JSON schema refs inside a tool's args_schema."""
    if hasattr(tool, "args_schema") and tool.args_schema:
        try:
            flat_schema = jsonref.replace_refs(tool.args_schema)
            tool.args_schema = flat_schema
        except Exception as e:
            print(f"⚠️ Could not flatten schema for {tool.name}: {e}")
    return tool

# Flatten all schemas
flattened_tools = [flatten_tool_schema(t) for t in tools]

# Now bind them safely
llm_haiku = llm_haiku.bind_tools(flattened_tools)

# Example: run the model with tool binding
query = "Run a simple query against my Tableau datasource"
response = llm_haiku.invoke(query)

print(response)

jsonref.replace_refs() walks through the schema and replaces all $ref with their actual definition.
That removes paths like "$ref": "#/properties/query/properties/..." which were crashing dereference_refs.
After flattening, the schema is safe for bind_tools.

But it gives error only in the second tool (query-datasource)
If you fix it for more generic solution, especially for Langchain (because the most of AI engineers use this interface) it would be very helpful, thank you.

[anyoung-tableau]

Thanks for digging in on that @SuleymanSuleymanzade and finding a workaround. There may not be much we can do to the schema definitions for the tool arguments since they mirror the actual schemas from the VizQL Data Service API.

Just poking around on StackOverflow this seems like a common problem in Python?
https://stackoverflow.com/questions/47054088/fully-expanding-ref-references-in-a-json-schema-with-python