Skip to content

validator-requirements crashes with error_max_structured_output_retries due to complex nested JSON schema #159

@EivMeyer

Description

@EivMeyer

Problem

The validator-requirements agent in full-workflow.json consistently crashes with:

error_max_structured_output_retries
"errors": ["Failed to provide valid structured output after 5 attempts"]

Meanwhile, validator-code (which has a simpler schema) works fine.

Root Cause

Claude CLI's --json-schema feature struggles with deeply nested objects inside arrays combined with enum constraints.

validator-code schema (works):

{
  "approved": { "type": "boolean" },
  "summary": { "type": "string" },
  "errors": { "type": "array", "items": { "type": "string" } }
}

validator-requirements schema (crashes):

{
  "approved": { "type": "boolean" },
  "summary": { "type": "string" },
  "criteriaResults": {
    "type": "array",
    "items": {
      "type": "object",
      "properties": {
        "id": { "type": "string" },
        "status": { "enum": ["PASS", "FAIL", "SKIPPED", "CANNOT_VALIDATE"] },
        "evidence": {
          "type": "object",  // <-- nested object inside array
          "properties": { "command", "exitCode", "output" }
        }
      }
    }
  }
}

The model makes small mistakes (wrong enum case, missing nested field), Claude CLI rejects, retries 5 times internally, then gives up.

Proposed Fix

Simplify criteriaResults from complex nested array → simple string:

"jsonSchema": {
  "type": "object",
  "properties": {
    "approved": { "type": "boolean" },
    "summary": { "type": "string" },
    "errors": { "type": "array", "items": { "type": "string" } },
    "criteriaResults": { "type": "string" }
  },
  "required": ["approved", "summary"]
}

Before (crashes):

{
  "criteriaResults": [
    { "id": "AC1", "status": "PASS", "evidence": { "command": "npm test", "exitCode": 0, "output": "..." } }
  ]
}

After (works):

{
  "criteriaResults": "AC1 PASS: npm test → 15 tests passed\nAC2 FAIL: grep null → not found"
}

Same information, model just outputs formatted text instead of brittle nested JSON.

Files to Change

  • cluster-templates/base-templates/full-workflow.json - simplify validator-requirements jsonSchema
  • Update prompt to reflect new output format

Evidence

Cluster flying-quasar-25 burned $5.44 with validator-requirements failing 3 times while validator-code completed successfully on first try.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions