Skip to content

FABRICATION_WATCHLIST substring match causes false positives (e.g. "rust" in "TrustSec") #10

@tearl42

Description

@tearl42

Description of the bug:
The fabrication detection in validator.py uses a simple in substring check against the skills text, which causes false positives when a watchlist term appears as part of a longer word.

To Reproduce:
Add TrustSec to your skills boundary. Run the tailor stage. Every attempt will fail with Fabricated skill: 'rust' even though "rust" was never added — it's being matched inside "TrustSec".

Expected behavior:
The check should use word boundary matching so "rust" only matches the standalone word "rust", not substrings within other words.

Fix
In validator.py around line 132, change:
if fake in skills_text:
to
if re.search(r"\b" + re.escape(fake) + r"\b", skills_text):

Environment:
ApplyPilot version: (your version)
Model: gemini-2.5-flash
Resume: Long-career networking professional with TrustSec in skills boundary

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions