Switch static site generation to use S3 data instead of API

[khaliqgant]

Summary

Resolves static page generation failures by switching from API calls to S3-hosted data during build time. This eliminates rate limiting issues and enables successful generation of 1,862+ package pages.

Problem

Current implementation makes individual API calls for each package during next build:

• 1,862+ packages × individual /api/v1/packages/{name} calls
• Registry API limit: 100 requests/minute
• Result: 429 errors → null responses → all pages render as 404s
• Build time: Fails after ~30 seconds due to rate limiting

Solution

Lambda function pre-fetches all data and uploads to S3. Next.js reads from S3 instead of API:

• Before: 1,862+ API calls during build → rate limited → fails
• After: 2 S3 reads (packages.json, collections.json) → no rate limits → succeeds

Changes

Package Pages (packages/[author]/[...package]/page.tsx)

• generateStaticParams(): Fetches from ${S3_SEO_DATA_URL}/packages.json
• getPackage(): Finds package in S3 JSON array instead of API call
• generateMetadata(): Reuses getPackage() (eliminates duplicate API call)

Collection Pages (collections/[slug]/page.tsx)

• generateStaticParams(): Fetches from ${S3_SEO_DATA_URL}/collections.json
• getCollection(): Finds collection in S3 JSON array instead of API call
• generateMetadata(): Reuses getCollection() (eliminates duplicate API call)

Configuration

• Environment Variable: NEXT_PUBLIC_S3_SEO_DATA_URL
• Default: https://prpm-prod-packages.s3.amazonaws.com/seo-data
• S3 Files Required:
  • seo-data/packages.json - Array of all public, non-deprecated packages
  • seo-data/collections.json - Array of all public collections

Diff Summary

packages/[author]/[...package]/page.tsx: -92 lines, +59 lines
collections/[slug]/page.tsx: -100 lines, +60 lines
Total: -192 lines, +119 lines (37% reduction)

Deployment Workflow

1. Lambda invocation (uploads S3 data):

   aws lambda invoke \
     --function-name prpm-prod-seo-data-fetcher \
     --payload '{"bucketName":"prpm-prod-packages","keyPrefix":"seo-data"}' \
     response.json

2. Next.js build (reads from S3):

   cd packages/webapp
   npm run build  # Now reads from S3 instead of API

3. Deploy:

   npm run deploy  # S3 + CloudFront

Testing Plan

• Deploy Lambda infrastructure (infrastructure PR V2 #2)
• Invoke Lambda to upload S3 data
• Verify S3 files exist: s3://prpm-prod-packages/seo-data/packages.json
• Run npm run build - should complete without 429 errors
• Verify static pages contain actual package data (not 404s)
• Check build logs show S3 URLs, not API URLs

Related PRs

• Infrastructure PR V2 #2: SEO Lambda function
• Depends on Lambda deployment completing first

Rollback Plan

If issues occur:

1. Set SKIP_SSG=true in build env (generates only mock pages)
2. Or revert this PR and use dynamic rendering (slower, but works)

🤖 Generated with Claude Code
via Happy