Skip to content

Topology gate checks calibration-phase metrics instead of final network metrics #111

@DeveshParagiri

Description

@DeveshParagiri

Bug

The topology gate rejects networks based on metrics from the calibration phase (similarity-only edges), not the final network (after structural edges, triadic closure, and rewiring). This causes valid networks to be quarantined.

Evidence

ASI scenario, 5,000 agents, avg_degree=10:

Metric Gate Bounds Calibration (checked) Final (ignored)
avg_degree 8.5–11.5 1.53 FAIL 10.4 PASS
clustering ≥0.14 0.01 FAIL 0.26 PASS
modularity 0.30–0.50 0.23 FAIL 0.83 FAIL*
LCC ≥0.95 0.58 FAIL 0.999 PASS
edge_count ≥16,250 3,827 FAIL 25,989 PASS

Calibration only produces ~3,800 edges via similarity threshold. Then ~22,000 structural edges (neighbor, congregation, partner, political_community, etc.) are merged post-calibration — the gate never sees them.

*Modularity is a separate issue — see below.

Root Cause

In generator.py, best_metrics and accepted are set during the calibration loop (after _sample_edges + _triadic_closure). Post-calibration steps (structural edge merge, _apply_rewiring) change the network substantially but the gate never re-evaluates.

The compute_network_metrics() call in generate_network_with_metrics() computes correct final metrics, but they're only informational — never fed back into the gate decision.

Fix

Re-evaluate the topology gate against the final network metrics (after structural edges + rewiring), not the calibration-phase metrics.

Related Issue: LLM-generated targets don't match population structure

The LLM set target_modularity: 0.55 but designed edge rules (state + urban/rural + race blocking) that naturally produce modularity ~0.83. For 5,000 agents across 50 states with strong geographic sorting, high modularity is realistic — the LLM's edge rules are correct but its target numbers are textbook defaults rather than population-specific reasoning. The config generation prompt should instruct the LLM to reason about expected metric ranges for the specific population structure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions