refactor: replace XML tool calling with native Vercel AI SDK tools, expose native LLM thinking

[buger]

Summary

• Replace custom XML-based tool calling (~2000+ lines of XML parsing, definitions, error recovery) with Vercel AI SDK native tool() + streamText() with stopWhen/stepCountIs — the SDK now manages the entire tool-use/tool-result loop automatically
• Implement Gemini google_search/url_context as wrapper function tools using separate generateText calls (avoids provider-defined tool mixing restriction)
• Expose native LLM thinking via providerOptions instead of custom thinking tags
• Remove 6 obsolete test files, update 14 remaining — net reduction of ~5600 lines across 29 files

What changed

Area	Before	After
Tool definitions	XML strings in system prompt	Vercel tool() with Zod/JSON schemas
Agent loop	~800-line manual while loop with XML parsing	streamText() with maxSteps via stopWhen: stepCountIs(N)
Error recovery	Custom XML parse recovery, stuck-loop detection	Not needed — native tool calling has no format errors
Gemini web tools	Provider-defined tools (only API-level tools)	Wrapper function tools with internal generateText calls
System message	~250 lines of XML format guidelines + tool defs	Simplified — tools come from tools parameter

Key technical details

• Uses AI SDK v5 API: stopWhen: stepCountIs(N) (not maxSteps), inputSchema (not parameters)
• Auto-wraps plain JSON Schema objects with jsonSchema() for SDK compatibility (edit/create tools)
• attempt_completion tool uses AbortController to cleanly exit the streamText loop
• Context compaction retry preserved at the outer level

Test plan

• All 106 test suites pass (2619 tests)
• Build succeeds
• Manual smoke tests: search, query, extract, searchFiles, listFiles, analyze_all, bash, delegate, create, edit, google_search, url_context, attempt_completion
• Multi-step agent loops verified (3+ steps with tool chaining)
• Gemini wrapper tools tested: standalone search, search + code tools combined, url_context

🤖 Generated with Claude Code

[probelabs]

PR Overview: Replace XML Tool Calling with Native Vercel AI SDK Tools

Summary

This PR is a major architectural refactoring that replaces the custom XML-based tool calling infrastructure (~2000+ lines) with Vercel AI SDK native tool() + streamText() with automatic tool-use loops. It also exposes native LLM thinking via providerOptions instead of custom thinking tags.

Files Changed Analysis

• 28 files changed: 1,123 additions, 5,058 deletions (net reduction of ~3,935 lines)
• Files removed: xmlParsingUtils.js (221 lines), mcpXmlBridge.test.js (499 lines), attempt-completion-closing-tag-in-content.test.js (188 lines), direct-content-attempt-completion.test.js (235 lines), stuck-loop-detection.test.js
• Major modifications: ProbeAgent.js (-802 net), xmlBridge.js (-300 net), common.js (-692 net)

Architecture & Impact Assessment

What this PR accomplishes:

1. Replaces custom XML tool definitions in system prompt with native Vercel tool() with Zod/JSON schemas
2. Replaces manual ~800-line while loop with streamText() using stopWhen: stepCountIs(N) for agent loop control
3. Removes custom thinking tag parsing - now uses native LLM thinking via providerOptions
4. Implements Gemini google_search/url_context as wrapper function tools with separate generateText calls
5. Auto-wraps plain JSON Schema objects with jsonSchema() for SDK compatibility
6. attempt_completion tool uses AbortController to cleanly exit the streamText loop

Key technical changes:

• Uses AI SDK v5 API: stopWhen: stepCountIs(N) (not maxSteps), inputSchema (not parameters)
• New CLI flag: --thinking-effort for native LLM thinking/reasoning control (off, low, medium, high, or budget tokens)
• MCP tools now provide Vercel-compatible tools directly via getVercelTools() instead of XML definitions
• Context compaction retry preserved at the outer level

flowchart TB
    subgraph "Before (XML-based)"
        A1[System Prompt with XML Tool Definitions] --> A2[Manual While Loop]
        A2 --> A3[XML Response Parsing]
        A3 --> A4[Custom Thinking Tag Removal]
        A4 --> A5[Tool Execution]
        A5 --> A2
    end
    
    subgraph "After (Native Vercel AI SDK)"
        B1["Native tool\(\) with Zod Schemas"] --> B2["streamText\(\) with tools"]
        B2 --> B3[SDK Auto Tool-Use Loop]
        B3 --> B4[Native providerOptions Thinking]
        B4 --> B5[Automatic Tool Execution]
        B5 --> B3
    end

Loading

Affected system components:

• Core agent loop (ProbeAgent.js) - main execution flow
• Tool definitions and schemas (common.js, tools.js, index.js)
• MCP integration (xmlBridge.js, mcp/index.js) - simplified to provide Vercel tools
• CLI arguments (index.js) - new --thinking-effort flag
• Test suite (14 updated, 6 removed)

Scope Discovery & Context Expansion

The changes affect the entire agent execution pipeline. Related areas to verify:

• Task management integration - verify task tool works with native tool calling
• Context compaction - ensure compaction retry works with new flow
• Schema validation - verify JSON schema responses work correctly
• Delegate/subagent tool behavior - ensure delegation passes correct config
• RAW_OUTPUT handling - verify execute_plan output passthrough still works
• MCP tool execution - verify MCP tools integrate via getVercelTools()

Note: The diff shows significant simplification but the actual file content still contains XML parsing code. This discrepancy should be verified - the PR may be in progress or the diff represents the target state.

Test Plan (from PR description)

• All 106 test suites pass (2619 tests)
• Build succeeds
• Manual smoke tests for all tools
• Multi-step agent loops verified (3+ steps with tool chaining)
• Gemini wrapper tools tested

Metadata

• Review Effort: 4 / 5
• Primary Label: enhancement

---

Powered by Visor from Probelabs

Last updated: 2026-03-03T20:25:44.860Z | Triggered by: pr_updated | Commit: 7168b77

💡 TIP: You can chat with Visor using /visor ask <your question>

[probelabs]

Architecture Issues (8)

Severity	Location	Issue
🔴 Critical	npm/src/agent/xmlParsingUtils.js:1-221	The PR diff shows xmlParsingUtils.js as REMOVED in the full_diff, but ProbeAgent.js still imports from it (line 65: import { removeThinkingTags, extractThinkingContent } from './xmlParsingUtils.js'). This would cause a runtime error if the file is actually deleted. 💡 Suggestion Either: (1) Keep xmlParsingUtils.js if XML parsing is still needed; or (2) Remove all imports and usages of removeThinkingTags, extractThinkingContent, and processXmlWithThinkingAndRecovery if migrating to native thinking support.
🔴 Critical	npm/src/agent/ProbeAgent.js:65	ProbeAgent.js imports from './xmlParsingUtils.js' but the PR shows this file as deleted. Additionally, tools.js imports processXmlWithThinkingAndRecovery from xmlParsingUtils.js. These imports would fail if the file is removed. 💡 Suggestion Ensure file deletions are coordinated with import removal. If xmlParsingUtils.js is deleted, all imports must be removed and any functionality either migrated or eliminated.
🟠 Error	npm/src/agent/ProbeAgent.js:3474-3476	PR description claims to replace XML tool calling with native Vercel AI SDK tools (streamText + stopWhen/stepCountIs), but the actual code still uses parseXmlToolCallWithThinking and parseHybridXmlToolCall for tool parsing. The native Vercel tool() integration with automatic tool-use loops is NOT implemented. The code still uses a manual while loop with XML parsing. 💡 Suggestion Either: (1) Implement the actual native Vercel AI SDK tool calling as described in the PR, using tool() definitions with inputSchema and streamText with maxSteps/stopWhen; or (2) Update the PR description to accurately reflect that this is a partial refactoring that removes XML definitions from system prompt but retains XML parsing in the agent loop.
🟠 Error	npm/src/agent/ProbeAgent.js:3213-3220	The agent loop still uses a manual 'while (currentIteration < maxIterations)' iteration pattern with XML parsing, not the Vercel AI SDK's native tool-use loop (streamText with tools parameter and automatic tool result injection). This means the claimed benefits of 'SDK manages the entire tool-use/tool-result loop automatically' are not realized. 💡 Suggestion To complete the migration to native Vercel tools: (1) Convert tool definitions to use tool() with inputSchema; (2) Pass tools to streamText; (3) Use maxSteps or stopWhen for loop control; (4) Remove manual XML parsing logic.
🟡 Warning	npm/src/agent/tools.js:1-286	The tools.js file still exports XML tool definitions (searchToolDefinition, queryToolDefinition, etc.) and parseXmlToolCall, which contradicts the PR claim that 'Tool definitions: XML strings in system prompt → Vercel tool() with Zod/JSON schemas'. The schemas exist but the XML definitions are still used in getSystemMessage(). 💡 Suggestion Complete the migration by: (1) Removing XML tool definition exports; (2) Using only Zod schemas with tool(); (3) Updating getSystemMessage() to not include XML tool definitions in the system prompt.
🟡 Warning	npm/src/agent/mcp/xmlBridge.js:1-44	The xmlBridge.js file still contains mcpToolToXmlDefinition and parseXmlMcpToolCall functions for XML-based MCP tool handling. The PR description states MCP tools should use 'wrapper function tools with internal generateText calls' but the XML parsing infrastructure remains. 💡 Suggestion If MCP tools are truly migrated to native Vercel format, remove the XML parsing functions and ensure MCPClientManager.getVercelTools() returns proper Vercel tool objects.
🟡 Warning	npm/src/agent/ProbeAgent.js:2503-2899	getSystemMessage() still builds XML tool definitions (toolDefinitions variable) and includes them in the system prompt. The PR claims 'tools come from tools parameter' but the system prompt still contains extensive XML tool documentation. 💡 Suggestion With native Vercel tools, tool definitions should be passed via the tools parameter to streamText, not embedded in the system message. Remove XML tool definitions from system prompt.
🟡 Warning	npm/src/agent/index.js:243-247	New --thinking-effort CLI flag is added but there's no implementation in ProbeAgent.js to use this value with providerOptions. The feature appears to be declared but not wired up. 💡 Suggestion Complete the implementation by: (1) Accepting thinkingEffort in ProbeAgent constructor options; (2) Passing it to streamText via providerOptions.google.thinkingBudget or equivalent for other providers.

Performance Issues (5)

Severity	Location	Issue
🟢 Info	probe/npm/src/agent/mcp/xmlBridge.js:141-213	The getVercelTools method in MCPXmlBridge (added in the diff) creates a new object copy on every call. When filterToolNames is provided, it iterates through all tools to create a filtered copy. This is called frequently and could be optimized with caching. 💡 Suggestion Consider caching the filtered tools or using a lazy evaluation pattern if getVercelTools is called repeatedly with the same filter.
🟢 Info	probe/npm/src/agent/ProbeAgent.js:3471-3476	The agent loop uses parseHybridXmlToolCall and parseXmlToolCallWithThinking for every assistant response. These functions perform multiple string operations (indexOf, regex matching, substring) which could be expensive for long responses. While this is existing code, the PR's removal of XML parsing utilities suggests this path should also be migrated to native tool calling. 💡 Suggestion The PR description mentions using streamText() with native tools, but the code still shows XML parsing in the agent loop. Verify that native Vercel AI SDK tool calling is fully implemented to avoid the overhead of XML parsing on every iteration.
🟢 Info	probe/npm/src/agent/mcp/client.js:543-563	The getVercelTools() method creates new wrapper objects and execute closures for every tool on every call. If this method is called frequently (e.g., on each agent iteration), it creates unnecessary garbage collection pressure. 💡 Suggestion Cache the tools object after first creation and only regenerate if the underlying tools Map changes. The wrapper objects and closures are identical across calls.
🟡 Warning	probe/npm/src/agent/mcp/xmlBridge.js:189-195	The initialize() method generates XML definitions (mcpToolToXmlDefinition) for all tools, but these are no longer used after the migration to native Vercel AI SDK tools. The getXmlToolDefinitions method and xmlDefinitions property are dead code that wastes CPU cycles during initialization. 💡 Suggestion Remove the xmlDefinitions property and related code (mcpToolToXmlDefinition function, getXmlToolDefinitions method) since the PR description states tools now use native Vercel tool() format. The MCPClientManager.getVercelTools() already provides the correct format.
🟡 Warning	probe/npm/src/agent/mcp/xmlBridge.js:1-137	The parseXmlMcpToolCall, parseHybridXmlToolCall, parseNativeXmlToolWithThinking, parseNativeXmlTool, and createHybridSystemMessage functions are exported but appear to be dead code after the migration to native Vercel AI SDK tools. These XML parsing functions add unnecessary module load time and memory footprint. 💡 Suggestion Remove unused XML parsing functions. The PR description indicates native tool calling has no format errors, so XML parsing is obsolete. Keep only MCPXmlBridge class with initialize(), getVercelTools(), getToolNames(), isMcpTool(), and cleanup() methods.

\n\n

Architecture Issues (8)

Severity	Location	Issue
🔴 Critical	npm/src/agent/xmlParsingUtils.js:1-221	The PR diff shows xmlParsingUtils.js as REMOVED in the full_diff, but ProbeAgent.js still imports from it (line 65: import { removeThinkingTags, extractThinkingContent } from './xmlParsingUtils.js'). This would cause a runtime error if the file is actually deleted. 💡 Suggestion Either: (1) Keep xmlParsingUtils.js if XML parsing is still needed; or (2) Remove all imports and usages of removeThinkingTags, extractThinkingContent, and processXmlWithThinkingAndRecovery if migrating to native thinking support.
🔴 Critical	npm/src/agent/ProbeAgent.js:65	ProbeAgent.js imports from './xmlParsingUtils.js' but the PR shows this file as deleted. Additionally, tools.js imports processXmlWithThinkingAndRecovery from xmlParsingUtils.js. These imports would fail if the file is removed. 💡 Suggestion Ensure file deletions are coordinated with import removal. If xmlParsingUtils.js is deleted, all imports must be removed and any functionality either migrated or eliminated.
🟠 Error	npm/src/agent/ProbeAgent.js:3474-3476	PR description claims to replace XML tool calling with native Vercel AI SDK tools (streamText + stopWhen/stepCountIs), but the actual code still uses parseXmlToolCallWithThinking and parseHybridXmlToolCall for tool parsing. The native Vercel tool() integration with automatic tool-use loops is NOT implemented. The code still uses a manual while loop with XML parsing. 💡 Suggestion Either: (1) Implement the actual native Vercel AI SDK tool calling as described in the PR, using tool() definitions with inputSchema and streamText with maxSteps/stopWhen; or (2) Update the PR description to accurately reflect that this is a partial refactoring that removes XML definitions from system prompt but retains XML parsing in the agent loop.
🟠 Error	npm/src/agent/ProbeAgent.js:3213-3220	The agent loop still uses a manual 'while (currentIteration < maxIterations)' iteration pattern with XML parsing, not the Vercel AI SDK's native tool-use loop (streamText with tools parameter and automatic tool result injection). This means the claimed benefits of 'SDK manages the entire tool-use/tool-result loop automatically' are not realized. 💡 Suggestion To complete the migration to native Vercel tools: (1) Convert tool definitions to use tool() with inputSchema; (2) Pass tools to streamText; (3) Use maxSteps or stopWhen for loop control; (4) Remove manual XML parsing logic.
🟡 Warning	npm/src/agent/tools.js:1-286	The tools.js file still exports XML tool definitions (searchToolDefinition, queryToolDefinition, etc.) and parseXmlToolCall, which contradicts the PR claim that 'Tool definitions: XML strings in system prompt → Vercel tool() with Zod/JSON schemas'. The schemas exist but the XML definitions are still used in getSystemMessage(). 💡 Suggestion Complete the migration by: (1) Removing XML tool definition exports; (2) Using only Zod schemas with tool(); (3) Updating getSystemMessage() to not include XML tool definitions in the system prompt.
🟡 Warning	npm/src/agent/mcp/xmlBridge.js:1-44	The xmlBridge.js file still contains mcpToolToXmlDefinition and parseXmlMcpToolCall functions for XML-based MCP tool handling. The PR description states MCP tools should use 'wrapper function tools with internal generateText calls' but the XML parsing infrastructure remains. 💡 Suggestion If MCP tools are truly migrated to native Vercel format, remove the XML parsing functions and ensure MCPClientManager.getVercelTools() returns proper Vercel tool objects.
🟡 Warning	npm/src/agent/ProbeAgent.js:2503-2899	getSystemMessage() still builds XML tool definitions (toolDefinitions variable) and includes them in the system prompt. The PR claims 'tools come from tools parameter' but the system prompt still contains extensive XML tool documentation. 💡 Suggestion With native Vercel tools, tool definitions should be passed via the tools parameter to streamText, not embedded in the system message. Remove XML tool definitions from system prompt.
🟡 Warning	npm/src/agent/index.js:243-247	New --thinking-effort CLI flag is added but there's no implementation in ProbeAgent.js to use this value with providerOptions. The feature appears to be declared but not wired up. 💡 Suggestion Complete the implementation by: (1) Accepting thinkingEffort in ProbeAgent constructor options; (2) Passing it to streamText via providerOptions.google.thinkingBudget or equivalent for other providers.

\n\n ### Performance Issues (5)

Severity	Location	Issue
🟢 Info	probe/npm/src/agent/mcp/xmlBridge.js:141-213	The getVercelTools method in MCPXmlBridge (added in the diff) creates a new object copy on every call. When filterToolNames is provided, it iterates through all tools to create a filtered copy. This is called frequently and could be optimized with caching. 💡 Suggestion Consider caching the filtered tools or using a lazy evaluation pattern if getVercelTools is called repeatedly with the same filter.
🟢 Info	probe/npm/src/agent/ProbeAgent.js:3471-3476	The agent loop uses parseHybridXmlToolCall and parseXmlToolCallWithThinking for every assistant response. These functions perform multiple string operations (indexOf, regex matching, substring) which could be expensive for long responses. While this is existing code, the PR's removal of XML parsing utilities suggests this path should also be migrated to native tool calling. 💡 Suggestion The PR description mentions using streamText() with native tools, but the code still shows XML parsing in the agent loop. Verify that native Vercel AI SDK tool calling is fully implemented to avoid the overhead of XML parsing on every iteration.
🟢 Info	probe/npm/src/agent/mcp/client.js:543-563	The getVercelTools() method creates new wrapper objects and execute closures for every tool on every call. If this method is called frequently (e.g., on each agent iteration), it creates unnecessary garbage collection pressure. 💡 Suggestion Cache the tools object after first creation and only regenerate if the underlying tools Map changes. The wrapper objects and closures are identical across calls.
🟡 Warning	probe/npm/src/agent/mcp/xmlBridge.js:189-195	The initialize() method generates XML definitions (mcpToolToXmlDefinition) for all tools, but these are no longer used after the migration to native Vercel AI SDK tools. The getXmlToolDefinitions method and xmlDefinitions property are dead code that wastes CPU cycles during initialization. 💡 Suggestion Remove the xmlDefinitions property and related code (mcpToolToXmlDefinition function, getXmlToolDefinitions method) since the PR description states tools now use native Vercel tool() format. The MCPClientManager.getVercelTools() already provides the correct format.
🟡 Warning	probe/npm/src/agent/mcp/xmlBridge.js:1-137	The parseXmlMcpToolCall, parseHybridXmlToolCall, parseNativeXmlToolWithThinking, parseNativeXmlTool, and createHybridSystemMessage functions are exported but appear to be dead code after the migration to native Vercel AI SDK tools. These XML parsing functions add unnecessary module load time and memory footprint. 💡 Suggestion Remove unused XML parsing functions. The PR description indicates native tool calling has no format errors, so XML parsing is obsolete. Keep only MCPXmlBridge class with initialize(), getVercelTools(), getToolNames(), isMcpTool(), and cleanup() methods.

\n\n ### Quality Issues (10)

Severity	Location	Issue
🔴 Critical	npm/src/agent/xmlParsingUtils.js:1	CRITICAL: The diff shows xmlParsingUtils.js as REMOVED (all lines marked with '-'), but the file still exists in the codebase and is actively imported by tools.js (line 44). This creates an inconsistent state where the PR description claims XML parsing was removed, but the implementation still uses it. 💡 Suggestion Either: 1) Actually remove xmlParsingUtils.js and update tools.js to remove parseXmlToolCallWithThinking and related imports, OR 2) Keep the file and update the PR description/tests to reflect that XML parsing is still used. The current state will cause confusion and potential runtime errors.
🟠 Error	npm/src/agent/ProbeAgent.js:3474	ProbeAgent.js still uses parseXmlToolCallWithThinking and parseHybridXmlToolCall for tool parsing (lines 3474-3476), which contradicts the PR description that claims migration to native Vercel AI SDK tools with streamText() + stopWhen/stepCountIs. The agent loop still manually parses XML tool calls. 💡 Suggestion Complete the migration by: 1) Using Vercel SDK's native tool calling via streamText's tools parameter, 2) Let the SDK handle tool execution automatically via maxSteps/stopWhen, 3) Remove manual XML parsing from the agent loop. Currently the PR appears to be half-migrated.
🟠 Error	npm/src/agent/mcp/xmlBridge.js:8	xmlBridge.js imports processXmlWithThinkingAndRecovery from xmlParsingUtils.js (line 8), but the diff shows xmlParsingUtils.js as removed. If the file is actually removed, this will cause a runtime error. 💡 Suggestion If xmlParsingUtils.js is being removed, update xmlBridge.js to remove this import and any usage of processXmlWithThinkingAndRecovery.
🟠 Error	npm/src/agent/tools.js:44	tools.js imports processXmlWithThinkingAndRecovery from xmlParsingUtils.js (line 44), but the diff shows xmlParsingUtils.js as removed. This will cause a runtime error if the file is actually deleted. 💡 Suggestion If xmlParsingUtils.js is being removed, remove this import and the parseXmlToolCallWithThinking function that depends on it.
🟢 Info	npm/src/tools/executePlan.js:393	executePlan.js was updated to use 'inputSchema' instead of 'parameters' (line 393, 1051), which is correct for Vercel AI SDK v5. However, other tools may still use 'parameters'. Need to verify all tools are consistently using the new API. 💡 Suggestion Verify all tool definitions use 'inputSchema' consistently for Vercel AI SDK v5 compatibility. Consider adding a lint rule or type check to catch inconsistencies.
🟡 Warning	npm/tests/unit/direct-content-attempt-completion.test.js:1	Test file tests parseXmlToolCallWithThinking from tools.js, but the PR description claims XML parsing was removed. If XML parsing is truly being removed, these tests should be removed. If not, the PR description is misleading. 💡 Suggestion Align tests with actual implementation: either remove XML parsing tests if migrating to native tools, or update PR description to clarify that XML parsing is still used.
🟡 Warning	npm/tests/unit/attempt-completion-closing-tag-in-content.test.js:1	Test file tests parseXmlToolCallWithThinking functionality that the PR claims to remove. The test imports from '../../src/agent/tools.js' which still exports this function, creating confusion about the actual state of the codebase. 💡 Suggestion Either remove these tests along with the XML parsing functionality, or clarify in PR description that XML parsing is retained.
🟡 Warning	npm/tests/integration/mcpErrorHandling.test.js:215	Test still uses XML-based tool execution (executeFromXml) which was supposedly removed. The test was updated but still tests the old XML bridge pattern rather than native Vercel SDK tool execution. 💡 Suggestion Update tests to verify native Vercel SDK tool execution patterns instead of XML-based tool calls.
🟡 Warning	npm/tests/unit/xmlParsing.test.js:1	Test file tests XML parsing (parseXmlToolCall, parseXmlToolCallWithThinking) but no tests exist for the new native Vercel AI SDK tool calling approach (streamText with tools, stopWhen/stepCountIs, inputSchema). Missing test coverage for the new architecture. 💡 Suggestion Add tests for: 1) Native tool execution via Vercel SDK streamText, 2) stopWhen/stepCountIs behavior, 3) inputSchema validation, 4) Native thinking via providerOptions, 5) Tool result handling in native mode.
🟡 Warning	npm/tests/integration/probeAgentMcp.test.js:171	Test mocks MCP bridge with getVercelTools() method but the mock is incomplete - it doesn't properly simulate the Vercel tool format with inputSchema and execute function. The mock returns objects but may not match actual Vercel SDK tool structure. 💡 Suggestion Ensure MCP tool mocks properly implement Vercel SDK tool interface: { description: string, inputSchema: ZodSchema, execute: async (params) => result }

---

Powered by Visor from Probelabs

Last updated: 2026-03-03T20:19:12.187Z | Triggered by: pr_updated | Commit: 7168b77

💡 TIP: You can chat with Visor using /visor ask <your question>