-
Notifications
You must be signed in to change notification settings - Fork 103
Description
Problem
The validator-requirements agent in full-workflow.json consistently crashes with:
error_max_structured_output_retries
"errors": ["Failed to provide valid structured output after 5 attempts"]
Meanwhile, validator-code (which has a simpler schema) works fine.
Root Cause
Claude CLI's --json-schema feature struggles with deeply nested objects inside arrays combined with enum constraints.
validator-code schema (works):
{
"approved": { "type": "boolean" },
"summary": { "type": "string" },
"errors": { "type": "array", "items": { "type": "string" } }
}validator-requirements schema (crashes):
{
"approved": { "type": "boolean" },
"summary": { "type": "string" },
"criteriaResults": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": { "type": "string" },
"status": { "enum": ["PASS", "FAIL", "SKIPPED", "CANNOT_VALIDATE"] },
"evidence": {
"type": "object", // <-- nested object inside array
"properties": { "command", "exitCode", "output" }
}
}
}
}
}The model makes small mistakes (wrong enum case, missing nested field), Claude CLI rejects, retries 5 times internally, then gives up.
Proposed Fix
Simplify criteriaResults from complex nested array → simple string:
"jsonSchema": {
"type": "object",
"properties": {
"approved": { "type": "boolean" },
"summary": { "type": "string" },
"errors": { "type": "array", "items": { "type": "string" } },
"criteriaResults": { "type": "string" }
},
"required": ["approved", "summary"]
}Before (crashes):
{
"criteriaResults": [
{ "id": "AC1", "status": "PASS", "evidence": { "command": "npm test", "exitCode": 0, "output": "..." } }
]
}After (works):
{
"criteriaResults": "AC1 PASS: npm test → 15 tests passed\nAC2 FAIL: grep null → not found"
}Same information, model just outputs formatted text instead of brittle nested JSON.
Files to Change
cluster-templates/base-templates/full-workflow.json- simplify validator-requirements jsonSchema- Update prompt to reflect new output format
Evidence
Cluster flying-quasar-25 burned $5.44 with validator-requirements failing 3 times while validator-code completed successfully on first try.