Skip to content

feat: add GitHub issue ingestion with admin #177

Open
tusharshah21 wants to merge 8 commits intomainfrom
feature/github-issues-ingestion
Open

feat: add GitHub issue ingestion with admin #177
tusharshah21 wants to merge 8 commits intomainfrom
feature/github-issues-ingestion

Conversation

@tusharshah21
Copy link
Collaborator

Summary

Implements GitHub issue ingestion into the backend database per issue #74: adds the github_issues table with composite PK (repo_id, github_issue_id), a POST /admin/github/sync endpoint behind admin auth, and idempotent upsert logic that derives points from labels, normalizes label names, persists assignees, and ignores PRs.

Changes

  • Added migration 006_create_github_issues_table.sql with composite PK and indexes
  • Added GithubIssue domain entity, GithubIssueRepository trait, and PostgresGithubIssueRepository with ON CONFLICT upsert
  • Added GithubService trait and RestGithubService implementation for GitHub REST API
  • Added sync_github_issues command with derive_points (label pattern points:N) and transform_issue (lower-case labels, assignee logins, PR filtering)
  • Added GithubSyncRequest/GithubSyncResponse DTOs
  • Added github_sync_handler and wired POST /admin/github/sync under existing admin auth middleware
  • Added reqwest 0.12 dependency for HTTP client
  • Added 10 unit tests covering points derivation, label normalization, PR ignoring, idempotent upsert, closed issue reflection, assignee persistence, and title update on re-sync

Scope

  • This PR intentionally avoids refactors, cleanups, and unrelated changes.
  • All modifications are limited to what is required by the issue.
  • Pagination, retry/backoff, and custom endpoint auth are explicitly out of scope per the ticket.

Verification

  • cargo check --tests compiles with zero errors
  • 10 new unit tests pass (labels→points, idempotent upsert, closed issue, PR filtering, assignees, upsert updates)
  • All existing tests continue passing

Closes #74

@tusharshah21
Copy link
Collaborator Author

GitHub issues ingestion with idempotent upsert (composite PK), admin-only sync endpoint, PR filtering, label→points derivation, and tests. Ready for review.

Copy link
Contributor

@joelamouche joelamouche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a first round of comments

Copy link
Contributor

@joelamouche joelamouche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another round of comments.

Don't forget to test this with a real repo, not just the unit tests

@tusharshah21
Copy link
Collaborator Author

Done reverted unrelated edits, renamed fields, added GITHUB_* + Bearer auth, switched to repo-name sync, added GET /github/issues and README examples
please review.

@joelamouche
Copy link
Contributor

pls fix CI

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not how point labels are forma ttedb in our repo

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of points:3 using 3pts

was tacking the label part first time, didn't have that propper clue,

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated chabges

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okays

@joelamouche
Copy link
Contributor

I ran the sync locally with the guild repo and all I got was this

backend-1 | 2026-02-18T10:36:49.500225Z DEBUG request{method=POST uri=/admin/github/sync version=HTTP/1.1}: tower_http::trace::on_request: started processing request backend-1 | 2026-02-18T10:36:49.500390Z DEBUG request{method=POST uri=/admin/github/sync version=HTTP/1.1}: tower_http::trace::on_response: finished processing request latency=0 ms status=401

And the git issue db is empty

You tried this locally with a git repo and it worked?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ingest GitHub Issues into Backend Database

2 participants

Comments