-
Notifications
You must be signed in to change notification settings - Fork 1
Description
Problem
The local egg-sdlc HITL checkpoint handler (sdlc_hitl.py) uses a single generic 5-option menu for all human input, regardless of what type of input is needed. This creates several problems:
-
No distinction between input types. Phase approval ("approve this analysis?"), multiple-choice decisions ("which database?"), and open-ended feedback all go through the same menu. The human sees the same
[1]-[5]options whether they're approving a plan or choosing between architectures. -
Decisions with discrete options aren't surfaced in the terminal. When the orchestrator queues a decision with specific options (e.g., "PostgreSQL vs MongoDB vs SQLite"), the menu doesn't display those options — it just shows the question text and offers generic approve/feedback choices.
-
No confirmation of what was received. After the human provides feedback or selects an option, there's no summary of what was submitted. The pipeline just continues, leaving the human uncertain whether their input was captured correctly.
-
Feedback is unstructured. Option
[4]collects raw multi-line text with no prompting structure. When the agent asks multiple specific questions, the human sees only the first question and has to answer everything in a single text block. -
No path for structured feedback in local mode. The contract system's
feedbackfield (with multiple questions and individual answers) only flows through GitHub comments. In local mode, there's no way for an agent to request structured multi-question feedback that renders in the terminal. -
No way to redirect approach. The current menu has "Approve" and "Provide feedback" but no explicit option for the human to say "this approach is wrong, try a different one." Feedback text gets resolved as a generic string with no signal to the agent about severity or intent.
Design Decisions
Decision type detection: explicit type field
Add a decision_type field to the orchestrator's HITLDecision model rather than inferring from options content. Values: phase_gate, choice, feedback.
This keeps detection clean and avoids fragile heuristics (e.g., checking if options contain "approve"). The orchestrator sets the type when creating phase gate decisions; agents set it when creating choice or feedback decisions via the API.
Structured feedback: extend HITLDecision
Add a questions field (list[dict]) to HITLDecision so agents can queue multi-question feedback through the orchestrator API. When decision_type is feedback, the terminal renders each question individually and collects answers one at a time. This gives local mode parity with the GitHub-based feedback UX.
General feedback and "change approach" are always available
Every HITL checkpoint — regardless of type — includes two universal options at the bottom:
- General feedback — free-text input that gets included alongside the primary resolution. Available on phase gates, choices, and feedback requests alike. Lets the human add context, caveats, or guidance without rejecting the current output.
- Change approach — signals the agent to re-run the current phase with a different approach. The human provides direction on what to change. The decision is resolved with a structured payload that distinguishes "feedback on current work" from "reject and redo."
Feedback triggers a phase re-run
When the human provides feedback (via "Request Changes" on a phase gate, or "Change approach" on any checkpoint), the agent must re-run the current phase to address the feedback, then present updated results for another round of review. The HITL checkpoint repeats until the human approves. The orchestrator already supports this via the decision resolution → phase restart flow; the terminal UX just needs to make the loop explicit and visible.
Proposed Changes
1. Extend HITLDecision model (orchestrator/models.py)
Add fields:
decision_type: str— one ofphase_gate,choice,feedback(default:choice)questions: list[dict]— for feedback type, each entry hasid,question,answer(default:[])
2. Update decision creation endpoints (orchestrator/routes/decisions.py)
Accept decision_type and questions in the create-decision payload. Pass through to HITLDecision.
3. Update phase gate creation (orchestrator/routes/pipelines.py)
When creating phase gate decisions, set decision_type="phase_gate".
4. Update OrchClient (sandbox/egg_lib/orch_client.py)
Add create_decision() method that supports decision_type and questions fields, so agents can create typed decisions from within containers.
5. Rework handle_hitl_checkpoint() (sandbox/egg_lib/sdlc_hitl.py)
Dispatch to different UX based on decision_type, with universal options always present.
phase_gate — phase approval with draft review:
============================================================
PHASE REVIEW: refine
============================================================
Pipeline: local-a1b2c3d4
--- Draft Preview ---
1 # Analysis Document
2 ## Summary
...
--- End Preview ---
[1] Edit with vim
[2] Start Claude for AI-assisted editing
[3] Approve and advance to next phase
[4] Request changes (re-run this phase with feedback)
─── Always available ───
[f] General feedback (add context without approving/rejecting)
[c] Cancel pipeline
Choose: 3
✓ Approved: advancing from refine → plan
choice — render actual options as numbered choices:
============================================================
DECISION REQUIRED
============================================================
Which database should we use?
[1] PostgreSQL
[2] MongoDB
[3] SQLite
─── Always available ───
[a] Suggest a different approach (re-run phase)
[f] General feedback (add context alongside your choice)
[c] Cancel pipeline
Choose: 2
✓ Selected: MongoDB
feedback — prompt for each question individually:
============================================================
FEEDBACK REQUESTED (2 questions)
============================================================
Q1: What is the expected traffic volume?
> ~10k requests/day
Q2: Any specific performance requirements?
> P95 latency under 200ms
─── Always available ───
[a] Suggest a different approach (re-run phase)
[f] General feedback (add additional context)
[c] Cancel pipeline
Submit answers? [y/n/edit]: y
✓ Feedback submitted (2 answers)
Resolution payloads:
The resolution sent to the orchestrator is a JSON string (not bare text) so the agent can parse the human's intent:
// Approved phase gate
{"action": "approve"}
// Phase gate with general feedback
{"action": "approve", "feedback": "Looks good but watch the error handling"}
// Request changes (triggers phase re-run)
{"action": "request_changes", "feedback": "The analysis missed the auth flow"}
// Multiple choice
{"action": "select", "selected": "MongoDB", "feedback": null}
// Multiple choice with different approach
{"action": "change_approach", "feedback": "None of these — use DynamoDB instead"}
// Structured feedback
{"action": "submit_feedback", "answers": {"Q1": "~10k/day", "Q2": "P95 < 200ms"}, "feedback": null}6. Phase re-run loop
When a decision resolves with action: "request_changes" or action: "change_approach":
- The orchestrator restarts the current phase (existing mechanism)
- The agent re-runs with the feedback injected as context
- On completion, a new HITL decision is queued
- The terminal shows the updated draft/results and the menu again
- This loop repeats until the human approves or cancels
The watch loop in sdlc_cli.py already handles this naturally — it reconnects to the SSE stream after resolving a decision, and the orchestrator emits a new awaiting_human event when the next checkpoint arrives.
Key Files
| Component | Path | Change |
|---|---|---|
| Decision model | orchestrator/models.py |
Add decision_type, questions fields |
| Decision routes | orchestrator/routes/decisions.py |
Accept new fields in create endpoint |
| Phase gate logic | orchestrator/routes/pipelines.py |
Set decision_type="phase_gate" |
| Orchestrator client | sandbox/egg_lib/orch_client.py |
Add create_decision() with type support |
| HITL handler | sandbox/egg_lib/sdlc_hitl.py |
Type-aware rendering, universal options, JSON resolution payloads |
| SDLC CLI | sandbox/egg_lib/sdlc_cli.py |
No changes expected (watch loop already handles re-run) |
Out of Scope
- GitHub issue comment HITL (that's the issue-mode path, handled by contract system)
- Changes to
egg-contractCLI (contract decisions/feedback stay as-is) - Changes to the contract schema or shared models
— Authored by egg