-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Bug
The topology gate rejects networks based on metrics from the calibration phase (similarity-only edges), not the final network (after structural edges, triadic closure, and rewiring). This causes valid networks to be quarantined.
Evidence
ASI scenario, 5,000 agents, avg_degree=10:
| Metric | Gate Bounds | Calibration (checked) | Final (ignored) |
|---|---|---|---|
| avg_degree | 8.5–11.5 | 1.53 FAIL | 10.4 PASS |
| clustering | ≥0.14 | 0.01 FAIL | 0.26 PASS |
| modularity | 0.30–0.50 | 0.23 FAIL | 0.83 FAIL* |
| LCC | ≥0.95 | 0.58 FAIL | 0.999 PASS |
| edge_count | ≥16,250 | 3,827 FAIL | 25,989 PASS |
Calibration only produces ~3,800 edges via similarity threshold. Then ~22,000 structural edges (neighbor, congregation, partner, political_community, etc.) are merged post-calibration — the gate never sees them.
*Modularity is a separate issue — see below.
Root Cause
In generator.py, best_metrics and accepted are set during the calibration loop (after _sample_edges + _triadic_closure). Post-calibration steps (structural edge merge, _apply_rewiring) change the network substantially but the gate never re-evaluates.
The compute_network_metrics() call in generate_network_with_metrics() computes correct final metrics, but they're only informational — never fed back into the gate decision.
Fix
Re-evaluate the topology gate against the final network metrics (after structural edges + rewiring), not the calibration-phase metrics.
Related Issue: LLM-generated targets don't match population structure
The LLM set target_modularity: 0.55 but designed edge rules (state + urban/rural + race blocking) that naturally produce modularity ~0.83. For 5,000 agents across 50 states with strong geographic sorting, high modularity is realistic — the LLM's edge rules are correct but its target numbers are textbook defaults rather than population-specific reasoning. The config generation prompt should instruct the LLM to reason about expected metric ranges for the specific population structure.