Skip to content

Lieutenant/swarm output streaming drops all tool execution events — only text_delta captured #56

@wwaIII

Description

@wwaIII

Bug: Lieutenant Output Shows Only LLM Text, Drops Tool Execution Progress

Component: vers-lieutenant.ts extension (also affects vers-swarm.ts)
Severity: High
Affects: All lieutenant and swarm orchestration workflows

Summary

vers_lt_read returns stale output while a lieutenant is actively executing tool calls because the event handler in installEventHandler() only captures text_delta events. All tool execution events (tool_execution_start/update/end) and tool call events (toolcall_start/delta/end) are silently dropped. This makes lieutenants appear "stuck" during tool-heavy work, causing the orchestrator to poll aggressively and waste tokens.

Root Cause

File: pi-v/extensions/vers-lieutenant.ts, lines ~717-718
Function: installEventHandler()

// Current code — only captures text_delta
handle.onEvent((event) => {
    if (event.type === "agent_start") {
        lt.status = "working";
        lt.lastOutput = "";
    } else if (event.type === "agent_end") {
        lt.status = "idle";
    } else if (event.type === "message_update" && event.assistantMessageEvent?.type === "text_delta") {
        lt.lastOutput += event.assistantMessageEvent.delta;  // ← ONLY this is captured
    }
    // ALL OTHER EVENTS SILENTLY DROPPED
});

Pi's RPC mode (rpc-mode.js) emits ALL events via session.subscribe((event) => output(event)) — they are streamed over the SSH tail -f connection and arrive at the extension. They're just ignored.

Same bug exists in vers-swarm.ts at lines ~830-831 and ~1292-1293.

Events Dropped

Event Type Description Captured?
agent_start Agent begins
agent_end Agent finishes
message_update + text_delta LLM text streaming
message_update + toolcall_start LLM requests tool ❌ DROPPED
message_update + toolcall_delta Tool call args streaming ❌ DROPPED
message_update + toolcall_end Tool call complete ❌ DROPPED
tool_execution_start Tool begins running ❌ DROPPED
tool_execution_update Partial tool output ❌ DROPPED
tool_execution_end Tool finished with result ❌ DROPPED
turn_start / turn_end Turn lifecycle ❌ DROPPED

Incident Data

  • Marketing lieutenant sent a task to create 7 files (ad copy, mockups)
  • Lieutenant responded: "Let me bang all of this out right now. Creating everything in parallel." (71 chars)
  • Orchestrator polled vers_lt_read 22 times over ~2 minutes
  • Every read returned the same 71-char string
  • Lieutenant was actually working — writing files via tool calls
  • Output only updated ~3 minutes later when the LLM emitted more text in a new turn

Reproduction Steps

  1. Create a lieutenant: vers_lt_create({ name: "test", ... })
  2. Send a tool-heavy task: vers_lt_send({ name: "test", message: "Create 5 files with different content" })
  3. Immediately poll: vers_lt_read({ name: "test" }) repeatedly
  4. Observe: Output freezes after initial text, shows no tool execution progress

Proposed Fix

Extend the event handler to capture tool execution events:

handle.onEvent((event) => {
    if (event.type === "agent_start") {
        lt.status = "working";
        lt.lastOutput = "";
    } else if (event.type === "agent_end") {
        lt.status = "idle";
    } else if (event.type === "message_update") {
        const ame = event.assistantMessageEvent;
        if (ame?.type === "text_delta") {
            lt.lastOutput += ame.delta;
        } else if (ame?.type === "toolcall_end") {
            lt.lastOutput += `\n[calling ${ame.toolCall?.name}...]`;
        }
    } else if (event.type === "tool_execution_start") {
        lt.lastOutput += `\n[executing ${event.toolName}...]`;
        lt.lastActivityAt = new Date().toISOString();
    } else if (event.type === "tool_execution_end") {
        const status = event.isError ? "ERROR" : "done";
        lt.lastOutput += `\n[${event.toolName}: ${status}]`;
        lt.lastActivityAt = new Date().toISOString();
    }
});

Apply the same fix to vers-swarm.ts at both locations.

Impact

  • Token waste: ~2,640+ tokens per incident from redundant polling
  • Observability: Zero visibility into tool execution progress
  • Reliability: Orchestrator may incorrectly assume lieutenant is stuck and take recovery actions
  • UX: Operators watching fleet status see frozen output with no indication of progress

Metadata

  • Discovered by: orchestrator + investigator lieutenant
  • Date: 2026-02-24
  • Confirmed via: Source code analysis of vers-lieutenant.ts, vers-swarm.ts, rpc-mode.js, pi-agent-core types
  • Full investigation report: Available on request (12KB with architecture diagrams and code refs)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions