feat: Kalshi TikTok feed — extension + scraper backend

[felarof99]

Summary

Chrome Extension (apps/kalshi-feed)

• TikTok-style vertical swipe feed of Kalshi prediction markets as a Chrome side panel
• Built with WXT, React 19, Tailwind CSS v4
• 12 fake markets for development, category tabs, "Buy Yes/No" deep-links to kalshi.com
• SCHEMA.md documents the exact API contract for the backend

Backend Service (apps/kalshi-scraper)

• Kalshi API + Browserbase scraping — Ingests prediction markets from Kalshi's public REST API and scrapes trending/top movers lists via Browserbase cloud browser automation
• Supabase storage via Drizzle ORM — Stores markets in PostgreSQL with upsert-on-ticker, matching SCHEMA.md
• REST API with ranked feed — Hono server: GET /feed with category filtering + cursor pagination, POST /scrape for cron-triggered refresh
• Algorithmic feed ranking — Scores by volume (30%), controversy (25%), recency (20%), closing urgency (15%), trending boost (10%)
• Category mapping — Maps Kalshi event categories + crypto keyword detection to 8 frontend categories

Design

Hybrid API + Browserbase Scraper. A Bun/Hono service combines Kalshi's public API for structured data with Browserbase + Playwright scraping for curated rankings the API doesn't expose. Both pipelines run in parallel, merge results, compute feed scores, and upsert to Supabase.

New files

apps/kalshi-scraper/
├── src/
│   ├── index.ts                    # Hono app with cors, logger, routes
│   ├── config.ts                   # Zod env validation
│   ├── routes/
│   │   ├── feed.ts                 # GET /feed — ranked market feed
│   │   └── scrape.ts               # POST /scrape — triggers ingestion
│   ├── services/
│   │   ├── kalshi-api.ts           # Kalshi REST API client with pagination & retry
│   │   ├── browserbase-scraper.ts  # Browserbase + Playwright DOM scraper
│   │   ├── ranking.ts              # Feed scoring algorithm
│   │   └── category-mapper.ts      # Kalshi → frontend category mapping
│   └── db/
│       ├── index.ts                # Drizzle client setup
│       └── schema.ts               # Markets table (matches SCHEMA.md)
├── scripts/test-api.ts
├── drizzle.config.ts
├── package.json
└── tsconfig.json

Test plan

• Load extension in Chrome → side panel opens with TikTok-style feed
• Run bun run apps/kalshi-scraper/scripts/test-api.ts to verify API fetching
• Set up .env, run bun run apps/kalshi-scraper/src/index.ts
• POST /scrape with x-scrape-secret header triggers ingestion
• GET /feed?category=trending returns ranked markets
• GET /feed?category=politics&limit=5 returns filtered results
• Verify cursor pagination works across pages

🤖 Generated with Claude Code

[greptile-apps]

Greptile Summary

Adds TikTok-style Kalshi prediction market feed with Chrome extension + backend scraper

Chrome Extension (apps/kalshi-feed):

• Vertical swipe feed using WXT + React 19 + Tailwind CSS v4
• Currently uses 12 fake markets for development
• Deep-links to kalshi.com for YES/NO betting

Backend Service (apps/kalshi-scraper):

• Combines Kalshi REST API + Browserbase/Playwright scraping
• Stores markets in Supabase PostgreSQL via Drizzle ORM
• GET /feed endpoint with category filtering + cursor pagination
• POST /scrape endpoint for cron-triggered ingestion
• Algorithmic ranking: volume (30%), controversy (25%), recency (20%), urgency (15%), trending boost (10%)

Issues Found:

• Magic constants in kalshi-api.ts should use @browseros/shared/constants per CLAUDE.md
• Inconsistent logging — uses console.log/warn/error instead of structured logger
• SCHEMA.md documentation missing 8 columns that exist in actual implementation
• Weak default SCRAPE_SECRET needs verification for production deployment

Confidence Score: 4/5

• Safe to merge with minor style improvements recommended
• Well-architected feature with clean separation of concerns. Backend uses proper error handling, retry logic, and batch operations. Frontend follows React best practices. Main issues are style-related (magic constants, logging) rather than functional bugs. No security vulnerabilities or logic errors found.
• Pay attention to apps/kalshi-scraper/src/services/kalshi-api.ts for magic constants and apps/kalshi-feed/SCHEMA.md for documentation completeness

Important Files Changed

Filename	Overview
apps/kalshi-scraper/src/services/kalshi-api.ts	Kalshi API client with pagination, retry logic, and rate limiting — uses magic constants instead of @browseros/shared
apps/kalshi-scraper/src/routes/scrape.ts	POST /scrape endpoint with parallel API + scraping, batch upserts — minor logging consistency issues
apps/kalshi-scraper/src/index.ts	Hono server with CORS, logging, feed and scrape routes — uses console.error instead of structured logger
apps/kalshi-scraper/src/config.ts	Zod environment validation — weak default secret for scrape endpoint needs strong value in production
apps/kalshi-feed/SCHEMA.md	API contract documentation — missing several columns that exist in actual backend schema

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Chrome Extension<br/>kalshi-feed] -->|GET /feed| B[Hono Server<br/>kalshi-scraper]
    C[Cron Job] -->|POST /scrape| B
    
    B --> D{Scrape Endpoint}
    D -->|Parallel Fetch| E[Kalshi API<br/>fetchAllMarkets]
    D -->|Parallel Fetch| F[Kalshi API<br/>fetchEvents]
    D -->|Parallel Scrape| G[Browserbase<br/>Playwright DOM Scraping]
    
    E --> H[Transform Markets]
    F --> H
    G -->|Trending Titles| H
    
    H --> I[Compute Feed Scores<br/>Volume + Controversy<br/>+ Recency + Urgency]
    I --> J[Category Mapping<br/>8 Categories]
    J --> K[Batch Upsert<br/>Drizzle ORM]
    K --> L[(PostgreSQL<br/>Supabase)]
    
    B --> M{Feed Endpoint}
    M --> N[Query Markets<br/>Category Filter]
    L --> N
    N --> O[Cursor Pagination<br/>by feed_score]
    O --> P[JSON Response]
    P --> A

Loading

_{Last reviewed commit: 0b04c1f}

[greptile-apps]

_{35 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

conspiracy category not supported by backend — category-mapper.ts only maps to 8 categories listed in SCHEMA.md (politics, sports, crypto, economics, entertainment, science, weather, tech)

Prompt To Fix With AI

This is a comment left during a code review.
Path: apps/kalshi-feed/lib/types/market.ts
Line: 10

Comment:
`conspiracy` category not supported by backend — `category-mapper.ts` only maps to 8 categories listed in `SCHEMA.md` (politics, sports, crypto, economics, entertainment, science, weather, tech)

How can I resolve this? If you propose a fix, please make it concise.