Skip to content

Improve system prompt to reduce false positives #70

@felipefernandes

Description

@felipefernandes

🎯 Goal

Reduce false positives in code reviews by improving the system prompt with specific guidelines.

📊 Complexity

Quick Win (1-2 hours)

🔍 Problem

The LLM is generating false positives like:

  • Reporting ${{ secrets.X }} as hardcoded secrets (it's correct GitHub Actions syntax)
  • Flagging os.chmod on config files as performance issues (it's security best practice)
  • Reporting existing try-except blocks as "missing error handling"
  • Suggesting premature optimizations for small lists (< 10 items)

✅ Solution

Add specific guidelines to the system prompt in iara/prompt.py:

IMPORTANT GUIDELINES:
- GitHub Actions: ${{ secrets.X }} is CORRECT, NOT hardcoded
- os.chmod on config files: Security best practice, NOT performance issue
- Existing try-except blocks: NOT missing error handling
- Small lists (< 10 items): O(n) vs O(1) is negligible, NOT worth optimizing
- Only report REAL problems that would cause bugs, security issues, or significant performance degradation
- When in doubt, DON'T report

📝 Implementation Steps

  1. Read current system prompt in iara/prompt.py
  2. Add new section with false positive guidelines
  3. Test with known false positive cases
  4. Verify reduction in false positive rate

🎁 Expected Impact

  • 50-70% reduction in false positives
  • More focused and actionable reviews
  • Better user trust in the tool

Related to Groq provider integration (#67) where false positives were identified.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Status

    Ready

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions