[Schema Consistency] Schema Consistency Audit - Deep Type Constraint Validation (Feb 7, 2026)

[github-actions[bot]]

This audit used a new strategy focusing on enum values, const constraints, and pattern regexes to find validation gaps between schema definitions and their enforcement in code.

Summary

• Total Inconsistencies: 7 (2 critical, 2 medium, 3 low priority)
• Analysis Strategy: Deep Type Constraint Validation (new approach)
• Categories: Missing validation logic, documentation gaps, deprecated fields
• Positive Findings: 3 areas with perfect alignment

---

🔴 Critical Issues

1. Missing Validation: engine.runtime Enum

Schema (pkg/parser/schemas/main_workflow_schema.json:2760):

"enum": ["local", "remote"]

Problem: No validation in pkg/workflow/engine.go:74 (ExtractEngineConfig). Invalid values like "invalid" would be accepted.

Impact: Runtime failures with unclear error messages

Fix: Add validation after line 88:

if runtime, ok := engineObj["runtime"].(string); ok {
    if runtime != "local" && runtime != "remote" {
        return "", nil, fmt.Errorf("invalid engine.runtime: %s (must be 'local' or 'remote')", runtime)
    }
}

2. Undocumented Pattern: MCP Domain Validation

Schema (pkg/parser/schemas/main_workflow_schema.json:6540):

"pattern": "^[a-zA-Z0-9]([a-zA-Z0-9\\-]{0,61}[a-zA-Z0-9])?(\\.[a-zA-Z0-9]([a-zA-Z0-9\\-]{0,61}[a-zA-Z0-9])?)*$"

Problem: DNS hostname pattern for MCP domains is not documented in docs/src/content/docs/reference/mcp-gateway.md

Impact: Users get validation errors without understanding format requirements

Fix: Document in MCP Gateway reference:

• Format requirement: DNS-compliant hostnames
• Valid examples: api.example.com, localhost, mcp-server.internal
• Invalid examples: -invalid.com, domain..com, under_score.com

---

🟡 Medium Priority Issues

3. Confusing Runtime Field Definitions

Schema (pkg/parser/schemas/main_workflow_schema.json:2207,2216):

"enum": ["default", "sandbox-runtime", "awf", "srt"]

Problem: Schema defines top-level runtime field, but:

• No workflows use it (all use sandbox.agent instead)
• Docs mention features.sandbox-runtime flag, not runtime field
• Creates confusion about data model

Actual usage in workflows:

sandbox:
  agent: awf  # This is what's actually used

Fix:

1. Clarify if top-level runtime is deprecated
2. Remove from schema or mark as deprecated
3. If active, document properly with examples

4. Undocumented Field: engine.language

Schema (pkg/parser/schemas/main_workflow_schema.json:3152):

"enum": ["go", "typescript", "python", "java", "rust", "csharp"]

Problem: Field exists in schema and code (pkg/workflow/tools_types.go:301) but is completely undocumented

Impact: Users unaware of feature, can't leverage Serena language-specific configs

Fix: Document in docs/src/content/docs/reference/frontmatter.md:

• Purpose: Configures language-specific runtime for Serena tools
• Valid values: go, typescript, python, java, rust, csharp
• Example usage with MCP tools

---

🟢 Low Priority Issues

5. Missing Validation: github-token.access-level

Schema (pkg/parser/schemas/main_workflow_schema.json:5982):

"enum": ["admin", "maintainer", "maintain", "write", "triage"]

Problem: No validation in parser/compiler code

Impact: Low - GitHub API will reject invalid values anyway

Fix: Add compile-time validation for fail-fast with clear error message

6. Pattern Not Enforced: Workflow ID

Schema (pkg/parser/schemas/main_workflow_schema.json:32):

"pattern": "^[a-zA-Z0-9_-]+$"

Problem: No regex validation in pkg/parser/frontmatter.go

Impact: Low - All current workflows comply, but invalid IDs could slip through

Fix: Add regex validation in frontmatter parser

7. Unclear Validation: Permissions Shortcuts

Schema (pkg/parser/schemas/main_workflow_schema.json:1428):

"enum": ["read-all", "write-all"]

Problem: Comment in pkg/workflow/permissions.go:133 mentions validation, but implementation unclear

Impact: Low - Likely handled correctly elsewhere

Fix: Add explicit validation with tests

---

✅ Positive Findings

Perfect Alignment Found

1. Engine IDs: Schema enum ["claude", "codex", "copilot", "custom"] perfectly matches registered engines in code
2. Permission Levels: Schema enum ["read", "write", "none"] matches constants in pkg/workflow/permissions.go:81-83
3. Workflow Compliance: All .github/workflows/*.md files use valid enum values (no violations found)

---

Recommendations Priority

High Priority (Week 1)

1. ✅ Add engine.runtime validation to prevent runtime failures
2. ✅ Document MCP domain pattern in reference docs

Medium Priority (Week 2-3)

3. ✅ Clarify/deprecate confusing runtime field
4. ✅ Document engine.language field and usage

Low Priority (Backlog)

5. Add github-token.access-level validation
6. Add workflow ID pattern enforcement
7. Clarify permissions shortcut validation

---

Strategy Performance

Strategy Used: Deep Type Constraint Validation (NEW)

Approach:

• Extracted all enum/const/pattern definitions from schema
• Cross-referenced with parser/compiler validation logic
• Checked documentation coverage
• Validated against real workflow usage

Effectiveness: HIGH ⭐

• Found 7 distinct inconsistencies
• 2 critical issues that could cause runtime failures
• Discovered completely undocumented field
• Identified confusing schema design

Should Reuse: ✅ Yes - Add to rotation (30% probability in future runs)

---

Workflow Run: §21775830822

AI generated by Schema Consistency Checker

• expires on Feb 14, 2026, 6:43 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-02-14T06:43:17.770Z.

Closed by Workflow