Skip to content

Add strict pre-sample validation gate profile (promote high-risk warnings to errors) #119

@RandomOscillations

Description

@RandomOscillations

Summary

Current sample flow validates merged spec and proceeds when there are warnings. For high-stakes runs, several warning classes are effectively correctness blockers and should fail early.

Why This Matters

Teams are repeatedly discovering issues downstream (DB contradictions, flat dynamics, no-op modifiers) that could have been blocked before sampling.

A strict gate profile would prevent recursive fix-run cycles.

Current Behavior (Code)

In /Users/adithyasrinivasan/Projects/extropy/extropy/cli/commands/sample.py:

  • merged spec validation runs (line ~165-172)
  • only errors block sampling (line ~175)
  • warnings always pass (line ~196-201)

Proposed Fix

Add a strict gate mode for sample (CLI flag and/or config), e.g. --strict-gates.

In strict mode, promote selected warning categories to errors, including at least:

  • condition value mismatches
  • modifier overlap ambiguity for categorical/boolean outcomes
  • critical scenario validation warnings that imply no-op behavior

Design principles:

  • deterministic and documented category list
  • opt-in initially (to avoid breaking current users)
  • printed summary of promoted warnings

Acceptance Criteria

  • extropy sample ... --strict-gates fails when promoted warning categories are present.
  • Default mode behavior remains backward compatible.
  • JSON output reports promoted categories and counts.

Pipeline Impact

Provides a reliable preflight quality gate for steps 1-3 before expensive downstream runs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions