Spam Detector Thinks I'm a First-Time Commenter

[someCatInTheWorld]

Apparently, even though I have contributed, the spam detector thinks I haven't. I suspect it might be because when it checks for the contribution API, it doesn't call for high enough array objects to check properly.

There could be more problems, but I don't know Bash and GitHub Actions well enough to check.

See:

• https://github.com/OmniBlocks/scratch-gui/blob/main/.github/workflows/spam-guard.yml
• let's add AI to omnibloxks #757 (comment)

@coderabbitai

[traycerai]

Plan

Observations

The spam-guard workflow in file:.github/workflows/spam-guard.yml has a critical bug in the first-time commenter detection logic. The comment count check (lines 52-54) fetches only the first 100 comments from the repository using /repos/{owner}/{repo}/issues/comments?per_page=100, then filters client-side by username. If the repository has more than 100 total comments and the user's previous comments fall outside this window, they won't be counted, causing the workflow to incorrectly flag returning contributors as first-time commenters.

Additionally, the current contribution detection logic (lines 32-50) has limitations:

• The commits check only verifies the existence of at least one commit (using per_page=1), which is correct but doesn't account for commits where the author email doesn't match the GitHub username
• The PR and issue checks only verify existence (using --limit 1), which is correct
• The comment check is fundamentally broken due to the pagination issue

Approach

The fix involves replacing the inefficient comment counting method with GitHub's Search API, which provides a commenter: filter to directly query issues/PRs where a specific user has commented. This eliminates the pagination limitation and ensures accurate detection of returning contributors. The solution maintains the existing workflow structure while only modifying the comment count logic to use a more reliable API endpoint.

Additionally, the workflow should include enhanced logging to help diagnose future false positives and provide better visibility into the contribution detection process.

Implementation Steps

1. Update Comment Count Logic in Spam Guard Workflow

File: file:.github/workflows/spam-guard.yml

Location: Lines 52-54 (the COMMENT_COUNT calculation)

Current problematic code:

COMMENT_COUNT=$(gh api \
  "/repos/${{ github.repository }}/issues/comments?per_page=100" \
  --jq "[.[] | select(.user.login==\"$COMMENTER\")] | length" 2>/dev/null || echo "0")

Replace with GitHub Search API approach:

Use the gh api command to call the GitHub Search API with the commenter: qualifier. The search endpoint is /search/issues and supports filtering by commenter and repository.

Implementation details:

• Use the search query: repo:${{ github.repository }} commenter:$COMMENTER
• The search API returns a total_count field that gives the exact number of issues/PRs where the user has commented
• This approach is not limited by pagination since it returns the total count directly
• Add error handling to default to "0" if the API call fails
• The search counts both issue comments and PR comments, providing comprehensive coverage

Implementation:

COMMENT_COUNT=$(gh api \
  "/search/issues?q=repo:${{ github.repository }}+commenter:$COMMENTER" \
  --jq '.total_count' 2>/dev/null || echo "0")

This single API call replaces the problematic pagination-limited approach with a direct count of all issues/PRs where the user has commented.

2. Add Enhanced Logging for Debugging

File: file:.github/workflows/spam-guard.yml

Location: Lines 56-64 (after calculating TOTAL)

Enhancement:
Add detailed logging to help diagnose future issues and provide visibility into the contribution detection process:

• Log individual counts: CONTRIBUTION_COUNT, PR_COUNT, ISSUE_COUNT, COMMENT_COUNT
• Log the total count before the comparison
• This will make it easier to debug false positives/negatives in the future

Implementation:

echo "📊 Contribution breakdown for $COMMENTER:"
echo "  - Commits: $CONTRIBUTION_COUNT"
echo "  - PRs: $PR_COUNT"
echo "  - Issues: $ISSUE_COUNT"
echo "  - Comments: $COMMENT_COUNT"
echo "  - Total: $TOTAL"

Insert this logging block immediately after the TOTAL calculation and before the if [ "$TOTAL" -eq 1 ] check.

3. Update Documentation

File: file:.github/workflows/spam-guard.yml

Location: Lines 22-24 (comment block for step 1)

Enhancement:
Add a clarifying comment explaining the contribution detection logic and the use of the Search API for comment counting:

# Check if user has ever contributed to this repo
# Note: We use the Search API for comment counting to avoid pagination limits
# This ensures we accurately detect all previous contributions

4. Testing Recommendations

After implementing the fix:

• Test with a known returning contributor (like the issue reporter) to verify they're not flagged as first-time
• Test with an actual first-time commenter to ensure the detection still works correctly
• Monitor the workflow runs for a few days to catch any edge cases
• Check the audit logs (lines 242-263) to verify correct classification
• Verify that the Search API call doesn't exceed rate limits in high-activity repositories

Technical Notes

GitHub Search API Advantages:

• The Search API has a rate limit of 30 requests per minute for authenticated requests, which is sufficient for this use case
• The total_count field provides an accurate count without pagination
• The search counts both issue comments and PR comments in a single query
• No need to implement pagination logic

Why This Fix Works:

• The original code fetched only the first 100 comments from the repository and filtered client-side
• In active repositories, a user's previous comments could be beyond the first 100, causing them to be missed
• The Search API directly queries for comments by a specific user, eliminating the pagination problem
• This is the recommended approach for user-specific queries in GitHub's API documentation

Import In IDE

🤖 Prompt for AI Agents

## Observations

The spam-guard workflow in `file:.github/workflows/spam-guard.yml` has a critical bug in the first-time commenter detection logic. The comment count check (lines 52-54) fetches only the first 100 comments from the repository using `/repos/{owner}/{repo}/issues/comments?per_page=100`, then filters client-side by username. If the repository has more than 100 total comments and the user's previous comments fall outside this window, they won't be counted, causing the workflow to incorrectly flag returning contributors as first-time commenters.

Additionally, the current contribution detection logic (lines 32-50) has limitations:
- The commits check only verifies the existence of at least one commit (using `per_page=1`), which is correct but doesn't account for commits where the author email doesn't match the GitHub username
- The PR and issue checks only verify existence (using `--limit 1`), which is correct
- The comment check is fundamentally broken due to the pagination issue

## Approach

The fix involves replacing the inefficient comment counting method with GitHub's Search API, which provides a `commenter:` filter to directly query issues/PRs where a specific user has commented. This eliminates the pagination limitation and ensures accurate detection of returning contributors. The solution maintains the existing workflow structure while only modifying the comment count logic to use a more reliable API endpoint.

Additionally, the workflow should include enhanced logging to help diagnose future false positives and provide better visibility into the contribution detection process.

## Implementation Steps

### 1. Update Comment Count Logic in Spam Guard Workflow

**File:** `file:.github/workflows/spam-guard.yml`

**Location:** Lines 52-54 (the `COMMENT_COUNT` calculation)

**Current problematic code:**
```bash
COMMENT_COUNT=$(gh api \
  "/repos/${{ github.repository }}/issues/comments?per_page=100" \
  --jq "[.[] | select(.user.login==\"$COMMENTER\")] | length" 2>/dev/null || echo "0")
```

**Replace with GitHub Search API approach:**

Use the `gh api` command to call the GitHub Search API with the `commenter:` qualifier. The search endpoint is `/search/issues` and supports filtering by commenter and repository.

**Implementation details:**
- Use the search query: `repo:${{ github.repository }} commenter:$COMMENTER`
- The search API returns a `total_count` field that gives the exact number of issues/PRs where the user has commented
- This approach is not limited by pagination since it returns the total count directly
- Add error handling to default to "0" if the API call fails
- The search counts both issue comments and PR comments, providing comprehensive coverage

**Implementation:**
```bash
COMMENT_COUNT=$(gh api \
  "/search/issues?q=repo:${{ github.repository }}+commenter:$COMMENTER" \
  --jq '.total_count' 2>/dev/null || echo "0")
```

This single API call replaces the problematic pagination-limited approach with a direct count of all issues/PRs where the user has commented.

### 2. Add Enhanced Logging for Debugging

**File:** `file:.github/workflows/spam-guard.yml`

**Location:** Lines 56-64 (after calculating TOTAL)

**Enhancement:**
Add detailed logging to help diagnose future issues and provide visibility into the contribution detection process:
- Log individual counts: `CONTRIBUTION_COUNT`, `PR_COUNT`, `ISSUE_COUNT`, `COMMENT_COUNT`
- Log the total count before the comparison
- This will make it easier to debug false positives/negatives in the future

**Implementation:**
```bash
echo "📊 Contribution breakdown for $COMMENTER:"
echo "  - Commits: $CONTRIBUTION_COUNT"
echo "  - PRs: $PR_COUNT"
echo "  - Issues: $ISSUE_COUNT"
echo "  - Comments: $COMMENT_COUNT"
echo "  - Total: $TOTAL"
```

Insert this logging block immediately after the `TOTAL` calculation and before the `if [ "$TOTAL" -eq 1 ]` check.

### 3. Update Documentation

**File:** `file:.github/workflows/spam-guard.yml`

**Location:** Lines 22-24 (comment block for step 1)

**Enhancement:**
Add a clarifying comment explaining the contribution detection logic and the use of the Search API for comment counting:

```bash
# Check if user has ever contributed to this repo
# Note: We use the Search API for comment counting to avoid pagination limits
# This ensures we accurately detect all previous contributions
```

### 4. Testing Recommendations

After implementing the fix:
- Test with a known returning contributor (like the issue reporter) to verify they're not flagged as first-time
- Test with an actual first-time commenter to ensure the detection still works correctly
- Monitor the workflow runs for a few days to catch any edge cases
- Check the audit logs (lines 242-263) to verify correct classification
- Verify that the Search API call doesn't exceed rate limits in high-activity repositories

## Technical Notes

**GitHub Search API Advantages:**
- The Search API has a rate limit of 30 requests per minute for authenticated requests, which is sufficient for this use case
- The `total_count` field provides an accurate count without pagination
- The search counts both issue comments and PR comments in a single query
- No need to implement pagination logic

**Why This Fix Works:**
- The original code fetched only the first 100 comments from the repository and filtered client-side
- In active repositories, a user's previous comments could be beyond the first 100, causing them to be missed
- The Search API directly queries for comments by a specific user, eliminating the pagination problem
- This is the recommended approach for user-specific queries in GitHub's API documentation

---

Execution Information

Branch: main
Commit: 2c93b63

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[someCatInTheWorld]

@coderabbitai Please see this.

[github-actions]

Hi @someCatInTheWorld! Thanks for reporting this issue with the can of pork detector not properly recognizing contributors.

It looks like you’ve identified a potential problem with how the spam guard checks contributions via the GitHub API. You suspect it might not be fetching enough array objects to correctly determine whether a user has contributed. Since this involves GitHub Actions and Bash, diving into the workflow configuration file () and testing its array handling would likely be the next step.

If you’re not familiar with the tech stack, perhaps someone in the community or maintainers can help narrow this down further! Let us know if you have more details or observations. 😊

[github-actions]

🎭 Issue Poem

In the realm of code, a glitch we detect,
The can of pork thinks, “You’re a newbie, correct?”
But fear not, dear friend, with skill you possess,
We’ll raise that array, and put it to rest!

With Bash and GitHub, we’ll tackle the quest,
Fixing detectors, yes, we’ll give it our best!
So let’s rally together, with laughter and cheer,
For every contribution, your voice is so near! 🐷✨

---

Generated with ❤️ by the OmniBlocks Poetry Bot

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

I'll investigate the spam detector issue for you. Let me examine the workflow file and the referenced issue comment.

---

🏁 Script executed:

#!/bin/bash

# Read the spam-guard.yml workflow file
echo "=== Spam Guard Workflow ==="
cat .github/workflows/spam-guard.yml

echo -e "\n\n=== Checking referenced issue comment ==="
# Get the specific comment from issue `#757`
gh api /repos/OmniBlocks/scratch-gui/issues/comments/3814522194 --jq '.body'

echo -e "\n\n=== Checking someCatInTheWorld's contributions to this repo ==="
# Check user's contributions
gh api /repos/OmniBlocks/scratch-gui/commits?author=someCatInTheWorld --jq 'length'
gh api /repos/OmniBlocks/scratch-gui/commits?author=someCatInTheWorld --jq '.[0:3] | .[] | {sha: .sha, date: .commit.author.date, message: .commit.message}'

Length of output: 12037

---

@someCatInTheWorld I found the bug! You're absolutely right about the root cause. 🎯

The Problem

The spam detector has a logic flaw on line 31 of the workflow. Here's what's happening:

Current Logic:

# Each check returns 0 or 1 (not the actual count!)
CONTRIBUTION_COUNT=$(gh api "/repos/.../commits?author=$COMMENTER&per_page=1" --jq 'length')  # Returns 1 if you have commits
PR_COUNT=$(gh pr list --author "$COMMENTER" --limit 1 --jq 'length')  # Returns 1 if you have PRs
ISSUE_COUNT=$(gh issue list --author "$COMMENTER" --limit 1 --jq 'length')  # Returns 1 if you have issues
COMMENT_COUNT=$(gh api ".../comments?per_page=100" --jq "[.[] | select(.user.login==\"$COMMENTER\")] | length")

TOTAL=$((CONTRIBUTION_COUNT + PR_COUNT + ISSUE_COUNT + COMMENT_COUNT))

if [ "$TOTAL" -eq 1 ]; then  # ⚠️ BUG IS HERE
  echo "is_first_time=true"

Why It Flags You:
Even though you have 4+ commits, the workflow only checks if at least one commit exists (per_page=1). So for you:

• ✅ CONTRIBUTION_COUNT = 1 (you have commits)
• ❓ PR_COUNT = 0 or 1 (depends on if you've opened PRs)
• ❓ ISSUE_COUNT = 0 or 1 (depends on if you've opened issues)
• ❓ COMMENT_COUNT = variable (only checks first 100 comments)

If TOTAL happens to equal exactly 1 (e.g., you have commits but no PRs, no issues, and your comment isn't in the first 100), you get flagged as first-time! 🤦

The Fix

Change line 31 from:

if [ "$TOTAL" -eq 1 ]; then

To:

if [ "$TOTAL" -eq 0 ]; then

This way, only users with zero prior contributions get spam-checked, not users with exactly one type of contribution.

Alternative (Better) Fix

For more reliable detection, the workflow should either:

1. Use higher per_page values to get actual counts
2. Or simplify: if ANY contribution type > 0, skip spam check

Would you like me to create a GitHub issue to track this fix? I can also help draft a PR if needed.

---

✏️ Learnings added

Learnt from: someCatInTheWorld
URL: https://github.com/OmniBlocks/scratch-gui/issues/759

Timestamp: 2026-01-29T02:15:03.351Z
Learning: OmniBlocks/scratch-gui spam-guard.yml has a bug on line 31: the condition `if [ "$TOTAL" -eq 1 ]` incorrectly flags contributors with commits as first-time commenters because each API call uses `per_page=1` or `limit 1` (returning 0 or 1, not actual counts). Users with commits but no PRs/issues can have TOTAL=1 and get spam-filtered. The fix is to change the condition to `if [ "$TOTAL" -eq 0 ]` to only flag truly first-time users.
```

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

---

🧠 Learnings used

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-27T22:15:17.804Z
Learning: In OmniBlocks/scratch-gui, CodeRabbit should only respond to comments that directly address coderabbitai. If a comment is directed at another bot (e.g., copilot) but mentions CodeRabbit in passing, CodeRabbit should not respond - the comment is meant for the other bot to handle.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:34:24.309Z
Learning: CodeRabbit experienced a spam bug on PR `#309` in OmniBlocks/scratch-gui where it posted 11 duplicate "no user input to reply to" messages. The spam was triggered by bot comments from github-actions and wellcode-ai. The comments were successfully minimized using GitHub GraphQL API mutations with the OFF_TOPIC classifier.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-12-05T16:55:31.105Z
Learning: OmniBlocks/scratch-gui: supervoidcoder wants automated comment moderation to detect spam/phishing from first-time commenters using GitHub Models AI (similar to ulti.yml workflow). Should analyze comments and hide/delete based on severity levels.

---

Learnt from: someCatInTheWorld
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-28T23:50:49.812Z
Learning: OmniBlocks/scratch-gui moderation: the spam rule currently flags the exact pattern “coderabbitai is this true”, causing comment jobs to fail, while “grok is this true” is not caught. Recommend avoiding that exact phrasing or adding punctuation/line breaks; consider relaxing the rule to reduce false positives.

---

Learnt from: github-actions[bot]
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-07T15:45:54.850Z
Learning: In OmniBlocks/scratch-gui repository, the GitHub Actions workflow that deploys PR test builds triggers on ANY comment containing the specific trigger phrase (including explanations, code blocks, or casual mentions), not just standalone commands. This causes multiple unintended workflow runs if the phrase is mentioned repeatedly in comments.

---

Learnt from: github-actions[bot]
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:58:09.061Z
Learning: In OmniBlocks/scratch-gui, GitHub author.login for CodeRabbit appears as "coderabbitai" (without the UI “[bot]” suffix). Use a case-insensitive ^coderabbit regex when filtering bot comments for cleanup.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-10-10T15:23:20.794Z
Learning: OmniBlocks/scratch-gui PR test workflow now ignores bot/AI comments containing the trigger phrase; only human users are allowed to trigger the deployment.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:55:33.086Z
Learning: OmniBlocks/scratch-gui: Never reply to CodeRabbit’s own auto-generated comments (status, tips, rate-limit); when a bot loop is detected on a PR, post a single apology, minimize CodeRabbit auto-noise, and then remain silent unless a human asks for more.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T16:28:02.495Z
Learning: OmniBlocks/scratch-gui: On closed pull requests, CodeRabbit must remain completely silent even if new CI/bot comments appear; do not reply to github-actions or any automated posts.

---

Learnt from: supervoidcoder
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2026-01-10T16:28:02.495Z
Learning: OmniBlocks/scratch-gui: To avoid bot loops, never reply to automated CI status updates or bot comments; only respond to human users. Prefer pausing reviews when noisy automation is detected.

---

Learnt from: github-actions[bot]
Repo: OmniBlocks/scratch-gui PR: 0
File: :0-0
Timestamp: 2025-11-05T01:50:28.065Z
Learning: OmniBlocks/scratch-gui: When cleaning PR noise, prefer minimizing CodeRabbit auto-generated comments matching headers like "This is an auto-generated reply by CodeRabbit", "Rate Limit Exceeded", and "automated status update", while preserving human-directed conversation.

[someCatInTheWorld]

@OmniBlocks/ai