Skip to content

fix(pipeline): no pipeline-level token budget or cost ceiling #17

@harperreed

Description

@harperreed

Summary

The pipeline engine has no aggregate token budget, cost ceiling, or wall-clock timeout across all nodes. Individual nodes have MaxTurns and retry limits, but the pipeline as a whole can consume unbounded API spend.

Impact

The megaplan.dip example runs 30+ LLM-powered nodes across three providers. Combined with max_restarts (default 5) and max_retries (default 3) on every node, a misconfigured pipeline has multiplicative cost exposure with no circuit breaker.

Recommendation

Add optional pipeline-level limits:

  • max_total_tokens: aggregate token cap across all nodes
  • max_cost: dollar/cent cost ceiling
  • max_wall_time: overall pipeline timeout
  • Circuit breaker that halts execution when any limit is hit

Sources

Found by: OpenAI Researcher (expert panel review)

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1: highSignificant impact on correctness or usabilityarea/enginePipeline execution engineenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions