-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Bug: Lieutenant Output Shows Only LLM Text, Drops Tool Execution Progress
Component: vers-lieutenant.ts extension (also affects vers-swarm.ts)
Severity: High
Affects: All lieutenant and swarm orchestration workflows
Summary
vers_lt_read returns stale output while a lieutenant is actively executing tool calls because the event handler in installEventHandler() only captures text_delta events. All tool execution events (tool_execution_start/update/end) and tool call events (toolcall_start/delta/end) are silently dropped. This makes lieutenants appear "stuck" during tool-heavy work, causing the orchestrator to poll aggressively and waste tokens.
Root Cause
File: pi-v/extensions/vers-lieutenant.ts, lines ~717-718
Function: installEventHandler()
// Current code — only captures text_delta
handle.onEvent((event) => {
if (event.type === "agent_start") {
lt.status = "working";
lt.lastOutput = "";
} else if (event.type === "agent_end") {
lt.status = "idle";
} else if (event.type === "message_update" && event.assistantMessageEvent?.type === "text_delta") {
lt.lastOutput += event.assistantMessageEvent.delta; // ← ONLY this is captured
}
// ALL OTHER EVENTS SILENTLY DROPPED
});Pi's RPC mode (rpc-mode.js) emits ALL events via session.subscribe((event) => output(event)) — they are streamed over the SSH tail -f connection and arrive at the extension. They're just ignored.
Same bug exists in vers-swarm.ts at lines ~830-831 and ~1292-1293.
Events Dropped
| Event Type | Description | Captured? |
|---|---|---|
agent_start |
Agent begins | ✅ |
agent_end |
Agent finishes | ✅ |
message_update + text_delta |
LLM text streaming | ✅ |
message_update + toolcall_start |
LLM requests tool | ❌ DROPPED |
message_update + toolcall_delta |
Tool call args streaming | ❌ DROPPED |
message_update + toolcall_end |
Tool call complete | ❌ DROPPED |
tool_execution_start |
Tool begins running | ❌ DROPPED |
tool_execution_update |
Partial tool output | ❌ DROPPED |
tool_execution_end |
Tool finished with result | ❌ DROPPED |
turn_start / turn_end |
Turn lifecycle | ❌ DROPPED |
Incident Data
- Marketing lieutenant sent a task to create 7 files (ad copy, mockups)
- Lieutenant responded: "Let me bang all of this out right now. Creating everything in parallel." (71 chars)
- Orchestrator polled
vers_lt_read22 times over ~2 minutes - Every read returned the same 71-char string
- Lieutenant was actually working — writing files via tool calls
- Output only updated ~3 minutes later when the LLM emitted more text in a new turn
Reproduction Steps
- Create a lieutenant:
vers_lt_create({ name: "test", ... }) - Send a tool-heavy task:
vers_lt_send({ name: "test", message: "Create 5 files with different content" }) - Immediately poll:
vers_lt_read({ name: "test" })repeatedly - Observe: Output freezes after initial text, shows no tool execution progress
Proposed Fix
Extend the event handler to capture tool execution events:
handle.onEvent((event) => {
if (event.type === "agent_start") {
lt.status = "working";
lt.lastOutput = "";
} else if (event.type === "agent_end") {
lt.status = "idle";
} else if (event.type === "message_update") {
const ame = event.assistantMessageEvent;
if (ame?.type === "text_delta") {
lt.lastOutput += ame.delta;
} else if (ame?.type === "toolcall_end") {
lt.lastOutput += `\n[calling ${ame.toolCall?.name}...]`;
}
} else if (event.type === "tool_execution_start") {
lt.lastOutput += `\n[executing ${event.toolName}...]`;
lt.lastActivityAt = new Date().toISOString();
} else if (event.type === "tool_execution_end") {
const status = event.isError ? "ERROR" : "done";
lt.lastOutput += `\n[${event.toolName}: ${status}]`;
lt.lastActivityAt = new Date().toISOString();
}
});Apply the same fix to vers-swarm.ts at both locations.
Impact
- Token waste: ~2,640+ tokens per incident from redundant polling
- Observability: Zero visibility into tool execution progress
- Reliability: Orchestrator may incorrectly assume lieutenant is stuck and take recovery actions
- UX: Operators watching fleet status see frozen output with no indication of progress
Metadata
- Discovered by: orchestrator + investigator lieutenant
- Date: 2026-02-24
- Confirmed via: Source code analysis of vers-lieutenant.ts, vers-swarm.ts, rpc-mode.js, pi-agent-core types
- Full investigation report: Available on request (12KB with architecture diagrams and code refs)