From d4f391a3ec9c26d95fe0f3b7bc1583c5750941d2 Mon Sep 17 00:00:00 2001 From: ruben-cytonic Date: Fri, 6 Feb 2026 13:50:17 +0000 Subject: [PATCH 1/2] feat: add SEO & AEO toolkit templates Add new "seo" category with two framework-agnostic templates: - AEO Toolkit: llms.txt, AI crawler management, structured data, citation tracking - SEO Toolkit: metadata, sitemaps, Core Web Vitals, audit dashboard Both use a monorepo architecture with adapters for Next.js, Nuxt, Astro, Remix, and Express. Co-Authored-By: Claude Opus 4.6 --- README.md | 6 + specs/seo/aeo-toolkit.md | 337 +++++++++++++++++++++++++++++++++++++ specs/seo/seo-toolkit.md | 351 +++++++++++++++++++++++++++++++++++++++ templates.json | 23 +++ 4 files changed, 717 insertions(+) create mode 100644 specs/seo/aeo-toolkit.md create mode 100644 specs/seo/seo-toolkit.md diff --git a/README.md b/README.md index 5dc1323..df44968 100644 --- a/README.md +++ b/README.md @@ -49,6 +49,12 @@ Ralph reads these specs and builds the entire project autonomously. |----------|-------------|------------| | [react-native-app](specs/mobile/react-native-app.md) | Cross-platform mobile app | Intermediate | +### SEO & AEO +| Template | Description | Difficulty | +|----------|-------------|------------| +| [aeo-toolkit](specs/seo/aeo-toolkit.md) | Answer Engine Optimization with llms.txt, AI crawlers, citations | Advanced | +| [seo-toolkit](specs/seo/seo-toolkit.md) | Technical SEO with metadata, sitemaps, Core Web Vitals | Intermediate | + ### Tools | Template | Description | Difficulty | |----------|-------------|------------| diff --git a/specs/seo/aeo-toolkit.md b/specs/seo/aeo-toolkit.md new file mode 100644 index 0000000..c675f76 --- /dev/null +++ b/specs/seo/aeo-toolkit.md @@ -0,0 +1,337 @@ +# AEO Toolkit — Answer Engine Optimization + +Build a framework-agnostic Answer Engine Optimization toolkit that makes any website fully discoverable by AI crawlers (GPTBot, ClaudeBot, PerplexityBot) and optimized for citation in AI-generated answers. + +## Overview + +A production-ready AEO (Answer Engine Optimization) toolkit shipped as a standalone Node.js package with framework adapters for Next.js, Nuxt, Astro, Remix, and Express. As AI-powered answer engines like ChatGPT, Perplexity, and Claude increasingly replace traditional search, websites need a new layer of optimization beyond classic SEO. This toolkit generates `llms.txt` / `llms-full.txt`, configures `robots.txt` with AI crawler directives, produces structured data (JSON-LD) tuned for answer extraction, ensures all critical content is server-rendered in HTML (AI crawlers don't execute JavaScript), and provides a monitoring dashboard to track AI bot traffic and citation performance. + +The core library is framework-agnostic — it generates plain text, markdown, XML, and JSON-LD strings. Framework adapters wire these generators into each framework's routing and middleware system. Inspired by the `llms.txt` standard (llmstxt.org), Firecrawl's crawling patterns, and Conductor's AI crawlability research. + +## Features + +- `llms.txt` and `llms-full.txt` auto-generation from site content map +- `robots.txt` with granular AI crawler directives (GPTBot, ChatGPT-User, ClaudeBot, Claude-Web, PerplexityBot, Google-Extended, CCBot) +- AI-optimized XML sitemap with priority scoring and `lastmod` timestamps +- Server-side rendered (SSR) content layer — ensures critical content is in raw HTML, not behind JavaScript +- Structured data engine: FAQ, Article, HowTo, Product, Organization, and BreadcrumbList schemas (JSON-LD) +- Answer-first content formatting helpers (concise answer block above the fold, then detail) +- Entity consistency checker — validates that brand names, product names, and key entities are consistent across pages +- Authorship and freshness signals (author schema, datePublished, dateModified) +- Markdown endpoint generator — serves `.md` versions of pages for LLM consumption (per llms.txt spec) +- AI bot traffic monitoring dashboard — parses server/access logs to track GPTBot, ClaudeBot, PerplexityBot visits +- Citation tracker — monitors when your content appears in AI answer engine results +- Core Web Vitals monitoring integration +- Framework adapters: Next.js, Nuxt, Astro, Remix, Express +- CLI commands for auditing, generating, and validating all AEO artifacts + +## Tasks + +### Task 1: Core Library Setup + +- [ ] Initialize TypeScript monorepo with `packages/core`, `packages/cli`, and `packages/adapters/*` +- [ ] Configure Turborepo or npm workspaces for package management +- [ ] Set up `packages/core` with zero framework dependencies +- [ ] Create `aeo.config.ts` schema with Zod (site metadata, crawler policies, content map) +- [ ] Implement config loader that reads from project root +- [ ] Install core dependencies (unified/remark for markdown, xml2js for XML) +- [ ] Set up ESLint, Prettier, and Vitest + +### Task 2: robots.txt Generator + +- [ ] Create `generateRobotsTxt(config)` function in `packages/core/src/robots.ts` +- [ ] Implement AI crawler directive builder with per-bot Allow/Disallow rules +- [ ] Support configurable policies: allow-all, block-training-only, selective-access +- [ ] Add known AI bot user-agents: GPTBot, ChatGPT-User, ClaudeBot, Claude-Web, PerplexityBot, Google-Extended, CCBot, Meta-ExternalAgent, Bytespider, Applebot-Extended +- [ ] Include Sitemap directives pointing to both standard and AI sitemaps +- [ ] Add Crawl-delay directives for aggressive bots +- [ ] Write unit tests for all policy modes + +### Task 3: llms.txt and llms-full.txt Generator + +- [ ] Create `generateLlmsTxt(config)` function in `packages/core/src/llms-txt.ts` +- [ ] Implement auto-generation from config site content map +- [ ] Structure output per llmstxt.org spec: H1 title, blockquote summary, H2 sections with link lists +- [ ] Create `generateLlmsFullTxt(config)` with comprehensive site documentation +- [ ] Include company overview, product descriptions, target audience, competitive advantages +- [ ] Add "Optional" section for secondary/supplementary pages +- [ ] Add `lastUpdated` timestamp to both files +- [ ] Write unit tests for output format validation + +### Task 4: AI-Optimized Sitemap + +- [ ] Create `generateSitemap(pages, config)` function in `packages/core/src/sitemap.ts` +- [ ] Create `generateAISitemap(pages, config)` with AI-specific priority scoring +- [ ] Implement priority calculation based on content type (landing pages > blog > archives) +- [ ] Add `lastmod` timestamps from content metadata or git history +- [ ] Add `changefreq` hints based on content volatility +- [ ] Reference both sitemaps in robots.txt output +- [ ] Write tests for XML output validity + +### Task 5: Structured Data Engine (JSON-LD) + +- [ ] Create `generateJsonLd(type, data)` function in `packages/core/src/structured-data.ts` +- [ ] Implement Article schema (headline, author, datePublished, dateModified, publisher) +- [ ] Implement FAQ schema (Question/Answer pairs for featured snippet targeting) +- [ ] Implement HowTo schema (step-by-step content) +- [ ] Implement Product schema (name, description, price, availability, reviews) +- [ ] Implement Organization schema (name, logo, sameAs social profiles, contactPoint) +- [ ] Implement BreadcrumbList schema for navigation hierarchy +- [ ] Add schema validator that checks output against schema.org specs +- [ ] Export framework-agnostic `