Proposal: Extend Evaluation Beyond GitHub (LeetCode, HackerRank, CodeForces, CodeChef, etc)

[singla-rhydham]

Hi everyone

I had a brief exchange with Hari over email regarding improving the fairness of evaluations in Hiring Agent. I’m moving the discussion here.

Currently, while the system extracts multiple profiles (LeetCode, HackerRank, Codeforces, Codechef etc.), only GitHub is actively evaluated.
My proposal outlines how we can extend the same architecture used for GitHub to these platforms using a modular and cache-driven design.

📁 Proposal & Resources: [Google Drive Folder Link]

The folder contains:

• Proposal.docx – detailed approach
• Flow Diagram – visual of the integration path
• Mermaid Code – for reproducibility

Would love to hear thoughts or feedback from maintainers or contributors before I start implementing.

Thanks,
Rhydham Singla

[Pranjalbharat]

Hi @singla-rhydham, I completely align with your idea. I’ve already implemented a similar integration for LeetCode and raised a PR for it.
The same approach can definitely be extended to other platforms like CodeChef or Codeforces as well. You can check it out.

[AkshayJKulkarni]

Really nice proposal @singla-rhydham — the modular and cache-driven approach makes a lot of sense, especially since many of these platforms share similar data patterns.

One thing that might be useful while extending this is handling the different API limitations across platforms. Some (like Codeforces) have stable public APIs, while others rely more on scraping. Maybe having a common interface with fallback options could make scoring more consistent.

Excited to see how this progresses — I can help with testing or smaller parts of the integration if needed.

[trishanu-init]

@singla-rhydham you have an interesting proposal, but the evaluation criteria that you have proposed seems to focus heavily towards leetcode contributions and the dependency on public APIs and webscraping could be overcome, i have been trying out Firecrawl and looks like it can satisfy the rquirements. What's your opinion?