-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Bug
Rendered personas have multiple critical problems that would produce garbage simulation results. Sampled 5 random agents from the ASI study (5,000 agents).
Problems
1. Raw floats leaking through
Float attributes like ai_risk_perception render as raw numbers instead of natural language:
"When it comes to new technology like AI, I tend to be 0.40 about where things are headed."
Should be something like "I'm somewhat concerned about where AI is headed."
2. Boolean phrasing broken for homeownership
"I earn around $61,517 a year — I no my home"
Should be "I own my home" or "I rent my home."
3. "About average" for everything — no differentiation
Almost every float/relative attribute collapses to the middle bucket:
"My financial resilience is about average. My level of worry about my financial future is about average. My support for institutional restructuring is about average. My openness is about average. My organization is about average. My sociability is about average. My cooperativeness is about average. My emotional stability is about average."
The z-score bucketing is either miscalibrated or the distributions are too tight. Agents with genuinely different attribute values are all rendering as "about average." This destroys the differentiation that makes simulation meaningful.
4. Self-contradictory attributes from sampling
Agent 2: 18-year-old who is divorced.
Agent 5: 96-year-old with three 17-year-old children, working part-time in management.
Agent 3: Has children (Jackson 6, Noah 12) in household section, then says "I don't have any children under 18 living in my household."
These are sampling/constraint enforcement failures, not persona rendering bugs, but they compound the problem — the persona faithfully renders nonsensical profiles.
5. Contradictory employment rendering
"I work Unemployed in the Management/Professional field"
employment_status=Unemployed + occupation=Management/Professional should render as "I'm currently unemployed; my background is in management/professional work" — not mashed together.
6. Duplicate section headers
"Who I Am" appears twice in the rendered output.
Impact
These personas are what LLM agents reason from. Flat "about average" personality + self-contradictory facts = undifferentiated, incoherent reasoning. The simulation will converge on bland, generic responses because the agents can't distinguish themselves.
Sample output
===== AGENT 1: 28yo Black Female, Urban AZ, Lean Democrat =====
I'm a 28-year-old Female living in AZ... I work Unemployed in the
Management/Professional field... I no my home... I tend to be 0.40
about where things are headed.
My financial resilience is about average.
My level of worry is about average.
My support for institutional restructuring is about average.
My openness is about average.
My cooperativeness is about average.
[...every trait "about average"]
===== AGENT 5: 96yo Hispanic Male, Suburban WI, Lean Republican =====
I'm a 96-year-old Male... I work Part-time in the Management/Professional
field. Our children are Samantha (17), Michael (17) and Ashley (17).
I don't have any children under 18 living in my household.