diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..ca5714b --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,58 @@ +# Changelog + +All notable changes to the Codex file format specification. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). + +## [Unreleased] + +### Added +- Legal extension example document (`examples/legal-document/`) +- Precise layout example (`examples/presentation-document/presentation/layouts/letter.json`) +- Highlight annotation documentation in collaboration extension (Section 4.7) +- Example coverage checking capability in sync checker + +### Fixed +- Broken cross-reference in presentation layers spec (Section 5.1.2 → 5.4) +- Sync checker false positives for MIME types, token types, and enum values + +### Changed +- Content schema now references legal extension marks (`legal:cite`) + +## [0.1] - 2025-01 (Draft) + +### Added + +#### Core Specification +- Document manifest with state machine (draft → review → frozen → published) +- Content block model with 22 block types +- Dublin Core metadata support +- Asset embedding and management +- Provenance and lineage tracking +- Anchors and references system +- Presentation layers (paginated, continuous, responsive, precise) + +#### Extensions +- **Academic** (`codex.academic`) - Theorems, proofs, exercises, algorithms, equation groups +- **Collaboration** (`codex.collaboration` v0.2) - CRDT integration, comments, change tracking, presence +- **Forms** (`codex.forms`) - Interactive form fields with validation +- **Legal** (`codex.legal`) - Citations, Table of Authorities, court captions, signature blocks +- **Phantoms** (`codex.phantoms`) - Invisible structural elements for complex layouts +- **Presentation** (`codex.presentation`) - Advanced typography, master pages, print features +- **Security** (`codex.security`) - Digital signatures, encryption, redaction, scoped signatures +- **Semantic** (`codex.semantic`) - Bibliography, footnotes, glossary, entity markup, JSON-LD + +#### Profiles +- Simple Documents profile for recreational reading + +#### Tooling +- JSON Schema validation for all specification components +- Example document validation +- Cross-reference validation +- Spec-schema synchronization checking +- Template generation script + +### Notes +- This is an initial draft specification +- The collaboration extension migrated from v0.1 to v0.2 (anchor-based addressing) +- Feedback welcome via GitHub issues diff --git a/examples/legal-document/content/document.json b/examples/legal-document/content/document.json new file mode 100644 index 0000000..3080049 --- /dev/null +++ b/examples/legal-document/content/document.json @@ -0,0 +1,218 @@ +{ + "version": "0.1", + "blocks": [ + { + "type": "legal:caption", + "id": "caption", + "court": "United States District Court for the Northern District of California", + "caseNumber": "No. 3:24-cv-01234", + "parties": { + "plaintiff": "Jane Doe", + "defendant": "Acme Corporation" + }, + "docket": "Hon. Sarah Johnson" + }, + { + "type": "heading", + "id": "doc-title", + "level": 1, + "children": [ + { "type": "text", "value": "Brief in Support of Motion for Summary Judgment" } + ] + }, + { + "type": "legal:tableOfAuthorities", + "id": "toa", + "title": "Table of Authorities", + "categories": [ + { "name": "Cases", "key": "cases", "format": "bluebook" }, + { "name": "Statutes", "key": "statutes", "format": "bluebook" }, + { "name": "Regulations", "key": "regulations", "format": "bluebook" } + ], + "pageReferences": true, + "passimThreshold": 5 + }, + { + "type": "heading", + "id": "intro-heading", + "level": 2, + "children": [ + { "type": "text", "value": "I. Introduction" } + ] + }, + { + "type": "paragraph", + "id": "intro-para-1", + "children": [ + { "type": "text", "value": "Plaintiff Jane Doe respectfully submits this brief in support of her Motion for Summary Judgment. As demonstrated below, the undisputed facts establish that Defendant violated " }, + { + "type": "text", + "value": "Title VII of the Civil Rights Act of 1964", + "marks": [ + { + "type": "legal:cite", + "citation": "42 U.S.C. \u00a7 2000e et seq.", + "category": "statutes" + } + ] + }, + { "type": "text", "value": " by terminating Plaintiff's employment based on her gender." } + ] + }, + { + "type": "heading", + "id": "facts-heading", + "level": 2, + "children": [ + { "type": "text", "value": "II. Statement of Undisputed Facts" } + ] + }, + { + "type": "paragraph", + "id": "facts-para-1", + "children": [ + { "type": "text", "value": "Plaintiff was employed by Defendant from January 2020 until her termination on June 15, 2024. During her employment, Plaintiff received consistently positive performance reviews and was promoted twice." } + ] + }, + { + "type": "heading", + "id": "argument-heading", + "level": 2, + "children": [ + { "type": "text", "value": "III. Argument" } + ] + }, + { + "type": "heading", + "id": "argument-a-heading", + "level": 3, + "children": [ + { "type": "text", "value": "A. Standard of Review" } + ] + }, + { + "type": "paragraph", + "id": "standard-para-1", + "children": [ + { "type": "text", "value": "Summary judgment is appropriate when there is no genuine dispute as to any material fact and the movant is entitled to judgment as a matter of law. " }, + { + "type": "text", + "value": "Celotex Corp. v. Catrett", + "marks": [ + { + "type": "legal:cite", + "citation": "477 U.S. 317 (1986)", + "category": "cases", + "shortForm": "Celotex" + } + ] + }, + { "type": "text", "value": ". The moving party bears the initial burden of demonstrating the absence of a genuine issue of material fact. " }, + { + "type": "text", + "value": "Id.", + "marks": [ + { + "type": "legal:cite", + "citation": "477 U.S. 317 (1986)", + "category": "cases", + "shortForm": "Celotex", + "pinpoint": "at 323" + } + ] + }, + { "type": "text", "value": "" } + ] + }, + { + "type": "heading", + "id": "argument-b-heading", + "level": 3, + "children": [ + { "type": "text", "value": "B. Plaintiff Has Established a Prima Facie Case of Discrimination" } + ] + }, + { + "type": "paragraph", + "id": "prima-facie-para-1", + "children": [ + { "type": "text", "value": "Under the burden-shifting framework established in " }, + { + "type": "text", + "value": "McDonnell Douglas Corp. v. Green", + "marks": [ + { + "type": "legal:cite", + "citation": "411 U.S. 792 (1973)", + "category": "cases", + "shortForm": "McDonnell Douglas" + } + ] + }, + { "type": "text", "value": ", a plaintiff establishes a prima facie case of discrimination by showing: (1) membership in a protected class; (2) satisfactory job performance; (3) an adverse employment action; and (4) circumstances giving rise to an inference of discrimination." } + ] + }, + { + "type": "paragraph", + "id": "prima-facie-para-2", + "children": [ + { "type": "text", "value": "The Ninth Circuit has consistently applied this framework. See " }, + { + "type": "text", + "value": "Chuang v. Univ. of Cal. Davis", + "marks": [ + { + "type": "legal:cite", + "citation": "225 F.3d 1115, 1123 (9th Cir. 2000)", + "category": "cases", + "shortForm": "Chuang" + } + ] + }, + { "type": "text", "value": ". Furthermore, EEOC regulations provide that employers must maintain records of employment decisions. " }, + { + "type": "text", + "value": "29 C.F.R. \u00a7 1602.14", + "marks": [ + { + "type": "legal:cite", + "citation": "29 C.F.R. \u00a7 1602.14", + "category": "regulations" + } + ] + }, + { "type": "text", "value": "." } + ] + }, + { + "type": "heading", + "id": "conclusion-heading", + "level": 2, + "children": [ + { "type": "text", "value": "IV. Conclusion" } + ] + }, + { + "type": "paragraph", + "id": "conclusion-para-1", + "children": [ + { "type": "text", "value": "For the foregoing reasons, Plaintiff respectfully requests that this Court grant summary judgment in her favor on her Title VII claim." } + ] + }, + { + "type": "legal:signatureBlock", + "id": "sig-block", + "role": "counsel", + "signer": { + "name": "Jane Smith", + "title": "Counsel for Plaintiff", + "barNumber": "CA 123456", + "firm": "Smith & Associates LLP", + "address": "100 Market Street, Suite 500, San Francisco, CA 94105", + "telephone": "(415) 555-1234", + "email": "jsmith@smithlaw.com" + }, + "date": "2025-01-30" + } + ] +} diff --git a/examples/legal-document/manifest.json b/examples/legal-document/manifest.json new file mode 100644 index 0000000..f3a1418 --- /dev/null +++ b/examples/legal-document/manifest.json @@ -0,0 +1,25 @@ +{ + "codex": "0.1", + "id": "pending", + "state": "draft", + "created": "2025-01-30T10:00:00Z", + "modified": "2025-01-30T10:00:00Z", + "extensions": [ + { + "id": "codex.legal", + "version": "0.1", + "required": false + } + ], + "content": { + "path": "content/document.json", + "hash": "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + }, + "metadata": { + "dublinCore": "metadata/dublin-core.json" + }, + "legal": { + "citationStyle": "bluebook", + "jurisdiction": "US" + } +} diff --git a/examples/legal-document/metadata/dublin-core.json b/examples/legal-document/metadata/dublin-core.json new file mode 100644 index 0000000..aa1df9c --- /dev/null +++ b/examples/legal-document/metadata/dublin-core.json @@ -0,0 +1,14 @@ +{ + "version": "1.1", + "terms": { + "title": "Brief in Support of Motion for Summary Judgment", + "creator": ["Jane Smith, Esq."], + "subject": ["civil rights", "employment discrimination"], + "description": "A demonstration of the legal extension features including caption, Table of Authorities, legal citations, and signature block.", + "date": "2025-01-30", + "type": "Text", + "format": "application/vnd.codex+zip", + "language": "en", + "rights": "Confidential - Attorney Work Product" + } +} diff --git a/examples/presentation-document/presentation/layouts/letter.json b/examples/presentation-document/presentation/layouts/letter.json new file mode 100644 index 0000000..79c8235 --- /dev/null +++ b/examples/presentation-document/presentation/layouts/letter.json @@ -0,0 +1,166 @@ +{ + "version": "0.1", + "presentationType": "precise", + "targetFormat": "letter", + "pageSize": { + "width": "8.5in", + "height": "11in" + }, + "contentHash": "sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "generatedAt": "2025-01-30T14:00:00Z", + "pageTemplate": { + "margins": { + "top": "1in", + "bottom": "1in", + "left": "1in", + "right": "1in" + }, + "header": { + "content": "Presentation Features Demo", + "style": "header", + "y": "0.5in" + }, + "footer": { + "content": "Page {pageNumber} of {totalPages}", + "style": "footer", + "y": "10.5in" + } + }, + "pages": [ + { + "number": 1, + "elements": [ + { + "blockId": "chapter-1", + "x": "1in", + "y": "1in", + "width": "6.5in", + "height": "0.5in" + }, + { + "blockId": "p-intro-1", + "x": "1in", + "y": "1.75in", + "width": "6.5in", + "height": "0.8in" + }, + { + "blockId": "p-intro-2", + "x": "1in", + "y": "2.75in", + "width": "6.5in", + "height": "0.9in" + }, + { + "blockId": "section-1-1", + "x": "1in", + "y": "3.85in", + "width": "6.5in", + "height": "0.4in" + }, + { + "blockId": "p-typo-1", + "x": "1in", + "y": "4.45in", + "width": "6.5in", + "height": "0.8in" + }, + { + "blockId": "p-typo-2", + "x": "1in", + "y": "5.45in", + "width": "6.5in", + "height": "0.8in" + }, + { + "blockId": "fig-layout", + "x": "1in", + "y": "6.45in", + "width": "6.5in", + "height": "2.5in", + "continues": true + } + ] + }, + { + "number": 2, + "elements": [ + { + "blockId": "fig-layout", + "x": "1in", + "y": "1in", + "width": "6.5in", + "height": "0.5in", + "continuation": true + }, + { + "blockId": "section-1-2", + "x": "1in", + "y": "1.75in", + "width": "6.5in", + "height": "0.4in" + }, + { + "blockId": "p-layout-1", + "x": "1in", + "y": "2.35in", + "width": "6.5in", + "height": "0.8in" + }, + { + "blockId": "p-layout-2", + "x": "1in", + "y": "3.35in", + "width": "6.5in", + "height": "0.9in" + }, + { + "blockId": "chapter-2", + "x": "1in", + "y": "4.5in", + "width": "6.5in", + "height": "0.5in" + }, + { + "blockId": "p-print-1", + "x": "1in", + "y": "5.25in", + "width": "6.5in", + "height": "0.8in" + }, + { + "blockId": "p-print-2", + "x": "1in", + "y": "6.25in", + "width": "6.5in", + "height": "0.9in" + }, + { + "blockId": "table-features", + "x": "1in", + "y": "7.35in", + "width": "6.5in", + "height": "2in" + } + ] + } + ], + "fonts": { + "heading": { + "family": "Helvetica", + "style": "normal", + "weight": 700, + "unitsPerEm": 1000, + "ascender": 718, + "descender": -207 + }, + "body": { + "family": "Times New Roman", + "style": "normal", + "weight": 400, + "unitsPerEm": 2048, + "ascender": 1825, + "descender": -443 + } + } +} diff --git a/package.json b/package.json index f7c496e..01c621e 100644 --- a/package.json +++ b/package.json @@ -9,6 +9,7 @@ "validate:examples": "npx tsx scripts/validate-examples.ts", "check:sync": "npx tsx scripts/check-spec-schema-sync.ts", "check:refs": "npx tsx scripts/validate-cross-refs.ts", + "check:coverage": "npx tsx scripts/check-example-coverage.ts", "generate:template": "npx tsx scripts/generate-template.ts" }, "devDependencies": { diff --git a/schemas/content.schema.json b/schemas/content.schema.json index 42470ab..6425217 100644 --- a/schemas/content.schema.json +++ b/schemas/content.schema.json @@ -187,7 +187,8 @@ { "$ref": "https://codex.document/schemas/academic.schema.json#/$defs/theoremRefMark" }, { "$ref": "https://codex.document/schemas/academic.schema.json#/$defs/equationRefMark" }, { "$ref": "https://codex.document/schemas/academic.schema.json#/$defs/algorithmRefMark" }, - { "$ref": "https://codex.document/schemas/presentation.schema.json#/$defs/indexMark" } + { "$ref": "https://codex.document/schemas/presentation.schema.json#/$defs/indexMark" }, + { "$ref": "https://codex.document/schemas/legal.schema.json#/$defs/legalCiteMark" } ] } } diff --git a/scripts/check-example-coverage.ts b/scripts/check-example-coverage.ts new file mode 100644 index 0000000..da7f00e --- /dev/null +++ b/scripts/check-example-coverage.ts @@ -0,0 +1,200 @@ +#!/usr/bin/env npx tsx + +/** + * Checks example coverage for extensions and schemas. + * + * This script reports which extensions and schemas have example documents + * and which are missing coverage. + */ + +import * as fs from 'fs'; +import * as path from 'path'; + +const rootDir = path.join(__dirname, '..'); +const specExtensionsDir = path.join(rootDir, 'spec', 'extensions'); +const examplesDir = path.join(rootDir, 'examples'); +const schemasDir = path.join(rootDir, 'schemas'); + +interface CoverageResult { + name: string; + hasExample: boolean; + examplePath?: string; +} + +// Get all extension directories +function getExtensions(): string[] { + if (!fs.existsSync(specExtensionsDir)) { + return []; + } + + return fs.readdirSync(specExtensionsDir, { withFileTypes: true }) + .filter(dirent => dirent.isDirectory()) + .map(dirent => dirent.name); +} + +// Get all example directories +function getExamples(): string[] { + if (!fs.existsSync(examplesDir)) { + return []; + } + + return fs.readdirSync(examplesDir, { withFileTypes: true }) + .filter(dirent => dirent.isDirectory()) + .map(dirent => dirent.name); +} + +// Get all schema files +function getSchemas(): string[] { + if (!fs.existsSync(schemasDir)) { + return []; + } + + return fs.readdirSync(schemasDir) + .filter(file => file.endsWith('.schema.json')); +} + +// Check if an example uses a specific extension +function exampleUsesExtension(exampleDir: string, extensionId: string): boolean { + const manifestPath = path.join(examplesDir, exampleDir, 'manifest.json'); + + if (!fs.existsSync(manifestPath)) { + return false; + } + + try { + const manifest = JSON.parse(fs.readFileSync(manifestPath, 'utf8')); + const extensions = manifest.extensions || []; + return extensions.some((ext: { id: string }) => + ext.id === `codex.${extensionId}` || ext.id === extensionId + ); + } catch { + return false; + } +} + +// Check if an example uses a specific schema (by having a matching file) +function exampleUsesSchema(exampleDir: string, schemaName: string): boolean { + const examplePath = path.join(examplesDir, exampleDir); + + // Map schema names to expected file locations + const schemaFileMap: Record = { + 'manifest.schema.json': ['manifest.json'], + 'content.schema.json': ['content/document.json'], + 'dublin-core.schema.json': ['metadata/dublin-core.json'], + 'collaboration.schema.json': ['collaboration/comments.json', 'collaboration/changes.json'], + 'forms.schema.json': ['forms/data.json'], + 'phantoms.schema.json': ['phantoms/clusters.json'], + 'security.schema.json': ['security/signatures.json', 'security/annotations.json'], + 'provenance.schema.json': ['provenance/lineage.json'], + 'asset-index.schema.json': ['assets/index.json'], + 'precise-layout.schema.json': ['presentation/layouts/letter.json', 'presentation/layouts/a4.json'], + 'presentation.schema.json': ['presentation/paginated.json', 'presentation/continuous.json', 'presentation/responsive.json'], + 'academic.schema.json': ['academic/numbering.json'], + 'semantic.schema.json': ['semantic/bibliography.json'], + 'annotations.schema.json': ['security/annotations.json'], + }; + + const expectedFiles = schemaFileMap[schemaName] || []; + + for (const file of expectedFiles) { + if (fs.existsSync(path.join(examplePath, file))) { + return true; + } + } + + // Also check if manifest references this extension + if (schemaName.match(/^(academic|collaboration|forms|legal|phantoms|presentation|security|semantic)\.schema\.json$/)) { + const extName = schemaName.replace('.schema.json', ''); + return exampleUsesExtension(exampleDir, extName); + } + + return false; +} + +// Main +console.log('Checking example coverage...\n'); + +const extensions = getExtensions(); +const examples = getExamples(); +const schemas = getSchemas(); + +console.log(`Found ${extensions.length} extensions`); +console.log(`Found ${examples.length} example documents`); +console.log(`Found ${schemas.length} schemas\n`); + +// Check extension coverage +console.log('Extension Coverage:'); +console.log('='.repeat(50)); + +const extensionResults: CoverageResult[] = []; + +for (const ext of extensions) { + const matchingExamples = examples.filter(ex => exampleUsesExtension(ex, ext)); + + extensionResults.push({ + name: ext, + hasExample: matchingExamples.length > 0, + examplePath: matchingExamples.length > 0 ? `examples/${matchingExamples[0]}` : undefined + }); +} + +const coveredExtensions = extensionResults.filter(r => r.hasExample); +const uncoveredExtensions = extensionResults.filter(r => !r.hasExample); + +for (const result of coveredExtensions.sort((a, b) => a.name.localeCompare(b.name))) { + console.log(` ✓ ${result.name} - ${result.examplePath}`); +} + +for (const result of uncoveredExtensions.sort((a, b) => a.name.localeCompare(b.name))) { + console.log(` ✗ ${result.name} - NO EXAMPLE`); +} + +// Check schema coverage +console.log('\nSchema Coverage:'); +console.log('='.repeat(50)); + +const schemaResults: CoverageResult[] = []; + +// Schemas that are always used (core schemas) +const coreSchemas = ['manifest.schema.json', 'content.schema.json', 'dublin-core.schema.json']; + +// Schemas that don't need direct examples (they define shared types) +const sharedSchemas = ['anchor.schema.json']; + +for (const schema of schemas) { + if (sharedSchemas.includes(schema)) { + continue; // Skip shared type schemas + } + + const matchingExamples = examples.filter(ex => exampleUsesSchema(ex, schema)); + + schemaResults.push({ + name: schema, + hasExample: matchingExamples.length > 0, + examplePath: matchingExamples.length > 0 ? `${matchingExamples.length} example(s)` : undefined + }); +} + +const coveredSchemas = schemaResults.filter(r => r.hasExample); +const uncoveredSchemas = schemaResults.filter(r => !r.hasExample); + +for (const result of coveredSchemas.sort((a, b) => a.name.localeCompare(b.name))) { + console.log(` ✓ ${result.name} - ${result.examplePath}`); +} + +for (const result of uncoveredSchemas.sort((a, b) => a.name.localeCompare(b.name))) { + console.log(` ✗ ${result.name} - NO EXAMPLE`); +} + +// Summary +console.log('\n' + '='.repeat(50)); +console.log('\nSummary:'); +console.log(` Extensions: ${coveredExtensions.length}/${extensionResults.length} covered`); +console.log(` Schemas: ${coveredSchemas.length}/${schemaResults.length} covered`); + +if (uncoveredExtensions.length > 0 || uncoveredSchemas.length > 0) { + console.log('\nMissing coverage detected.'); + // Don't exit with error - this is informational +} else { + console.log('\nAll extensions and schemas have example coverage.'); +} diff --git a/scripts/check-spec-schema-sync.ts b/scripts/check-spec-schema-sync.ts index 81fd7d7..0065340 100644 --- a/scripts/check-spec-schema-sync.ts +++ b/scripts/check-spec-schema-sync.ts @@ -28,6 +28,46 @@ interface SyncReport { synced: string[]; } +// Patterns for types that appear in spec JSON examples but are NOT block/mark types +// These are excluded from sync checking as they are: +// - MIME types (asset types, not content blocks) +// - Presentation layer types (layout types, not content blocks) +// - Syntax highlighting token types (code formatting, not blocks) +// - State/enum values (document states, annotation types, etc.) +const EXCLUDED_PATTERNS: RegExp[] = [ + // MIME types (assets, not blocks) + /^image\//, /^font\//, /^application\//, /^text\//, /^audio\//, /^video\//, + // Presentation layer types (not content blocks) + /^(paginated|continuous|responsive|flow|precise)$/, + // Syntax highlighting token types (note: 'comment' is NOT excluded as it's also a collaboration annotation type) + /^(keyword|function|class|variable|parameter|string|number|boolean|null|docstring|operator|punctuation|delimiter|type|namespace|decorator|plain)$/, + // Document state values + /^(draft|review|frozen|published)$/, + // Collaboration change types (modification action types, not annotation blocks) + /^(insert|modify|delete)$/, + // Form field types that appear in examples as values (not block types themselves) + /^(textInput|checkbox|select|date|number|radio|email)$/, + // Dublin Core metadata values (not block types) + /^(Text|Image|Sound|Collection|Dataset|Event|InteractiveResource|MovingImage|PhysicalObject|Service|Software|StillImage)$/, + // JSON Schema type values + /^(object|array|string|integer|number|boolean|null)$/, + // Provenance evidence types (enum values) + /^(inclusion|exclusion|rfc3161|blockchain|aggregated|reference)$/, + // Presentation layout types (not content blocks) + /^(columns|grid|spot)$/, + // Citation format types (CSL types, not blocks) + /^(article-journal|article|book|chapter|paper-conference|thesis|report|webpage|entry-encyclopedia)$/, + // HTML/markdown styling (not block types) + /^(strong|em|code|sub|sup)$/, + // Other enum values that aren't block types + /^(attachment|embedded|external|required|optional)$/, +]; + +// Check if a type name should be excluded from sync checking +function isExcludedType(typeName: string): boolean { + return EXCLUDED_PATTERNS.some(pattern => pattern.test(typeName)); +} + // Extract block types from spec markdown files function extractTypesFromSpec(filePath: string): BlockType[] { const types: BlockType[] = []; @@ -48,7 +88,8 @@ function extractTypesFromSpec(filePath: string): BlockType[] { while ((match = typePattern.exec(line)) !== null) { const typeName = match[1]; // Skip "text" as it's always present and not a block type - if (typeName !== 'text' && !types.find(t => t.type === typeName)) { + // Also skip excluded patterns (MIME types, presentation layer types, etc.) + if (typeName !== 'text' && !isExcludedType(typeName) && !types.find(t => t.type === typeName)) { types.push({ type: typeName, source: filename, @@ -60,7 +101,7 @@ function extractTypesFromSpec(filePath: string): BlockType[] { // Match inline code references while ((match = inlinePattern.exec(line)) !== null) { const typeName = match[1]; - if (typeName !== 'text' && !types.find(t => t.type === typeName)) { + if (typeName !== 'text' && !isExcludedType(typeName) && !types.find(t => t.type === typeName)) { types.push({ type: typeName, source: filename, @@ -73,33 +114,58 @@ function extractTypesFromSpec(filePath: string): BlockType[] { return types; } +// Recursively extract type constants from a schema object +function extractTypeConstFromObject(obj: unknown, types: string[]): void { + if (!obj || typeof obj !== 'object') return; + + const record = obj as Record; + + // Check if this object has properties.type.const + if (record.properties && typeof record.properties === 'object') { + const props = record.properties as Record; + if (props.type && typeof props.type === 'object') { + const typeProp = props.type as Record; + if (typeProp.const && typeof typeProp.const === 'string') { + const typeName = typeProp.const; + // Skip 'text' as it's ubiquitous (matches spec extraction behavior) + if (typeName !== 'text' && !types.includes(typeName)) { + types.push(typeName); + } + } + } + } + + // Recurse into allOf, anyOf, oneOf arrays + for (const key of ['allOf', 'anyOf', 'oneOf']) { + if (Array.isArray(record[key])) { + for (const item of record[key] as unknown[]) { + extractTypeConstFromObject(item, types); + } + } + } + + // Recurse into if/then/else + for (const key of ['if', 'then', 'else']) { + if (record[key] && typeof record[key] === 'object') { + extractTypeConstFromObject(record[key], types); + } + } +} + // Extract block types from JSON schema files function extractTypesFromSchema(filePath: string): BlockType[] { const types: BlockType[] = []; const content = fs.readFileSync(filePath, 'utf8'); const filename = path.relative(rootDir, filePath); + const extractedTypes: string[] = []; try { const schema = JSON.parse(content); // Look for const types in $defs if (schema.$defs) { - for (const [defName, defValue] of Object.entries(schema.$defs)) { - const def = defValue as Record; - - // Check for type const in properties - if (def.properties && typeof def.properties === 'object') { - const props = def.properties as Record; - if (props.type && typeof props.type === 'object') { - const typeProp = props.type as Record; - if (typeProp.const && typeof typeProp.const === 'string') { - types.push({ - type: typeProp.const, - source: filename - }); - } - } - } + for (const [_defName, defValue] of Object.entries(schema.$defs)) { + extractTypeConstFromObject(defValue, extractedTypes); } } @@ -109,15 +175,21 @@ function extractTypesFromSchema(filePath: string): BlockType[] { for (const condition of allOf) { if (condition.if?.properties?.type?.const) { const typeName = condition.if.properties.type.const as string; - if (!types.find(t => t.type === typeName)) { - types.push({ - type: typeName, - source: filename - }); + // Skip 'text' as it's ubiquitous (matches spec extraction behavior) + if (typeName !== 'text' && !extractedTypes.includes(typeName)) { + extractedTypes.push(typeName); } } } } + + // Convert to BlockType objects + for (const typeName of extractedTypes) { + types.push({ + type: typeName, + source: filename + }); + } } catch (err) { console.error(`Error parsing ${filename}: ${err}`); } diff --git a/scripts/validate-examples.ts b/scripts/validate-examples.ts index 50eb040..d5c15cc 100644 --- a/scripts/validate-examples.ts +++ b/scripts/validate-examples.ts @@ -42,7 +42,7 @@ function loadJson(filepath: string): unknown { // Schema dependencies (schemas that need other schemas loaded first) const schemaDependencies: Record = { - 'content.schema.json': ['semantic.schema.json', 'academic.schema.json', 'presentation.schema.json'], + 'content.schema.json': ['semantic.schema.json', 'academic.schema.json', 'presentation.schema.json', 'legal.schema.json'], 'collaboration.schema.json': ['anchor.schema.json'], 'phantoms.schema.json': ['anchor.schema.json'], 'security.schema.json': ['anchor.schema.json'], diff --git a/scripts/validate-schemas.ts b/scripts/validate-schemas.ts index 599dfaa..dcddbd3 100644 --- a/scripts/validate-schemas.ts +++ b/scripts/validate-schemas.ts @@ -35,7 +35,7 @@ const standaloneSchemas: string[] = [ const dependentSchemas: DependentSchema[] = [ { schema: 'annotations.schema.json', refs: ['anchor.schema.json'] }, { schema: 'collaboration.schema.json', refs: ['anchor.schema.json'] }, - { schema: 'content.schema.json', refs: ['semantic.schema.json', 'academic.schema.json', 'presentation.schema.json'] }, + { schema: 'content.schema.json', refs: ['semantic.schema.json', 'academic.schema.json', 'presentation.schema.json', 'legal.schema.json'] }, { schema: 'phantoms.schema.json', refs: ['anchor.schema.json'] }, { schema: 'security.schema.json', refs: ['anchor.schema.json'] }, ]; diff --git a/spec/core/00-introduction.md b/spec/core/00-introduction.md index 7cd8d24..6b37996 100644 --- a/spec/core/00-introduction.md +++ b/spec/core/00-introduction.md @@ -149,6 +149,8 @@ Codex provides three annotation storage locations, each serving different purpos All annotation layers are **outside the content hash boundary** — adding annotations never changes the document's identity or invalidates signatures. +**Implementation note**: Core annotations live in `security/annotations.json` because they share the security directory's "outside content hash" semantics. When migrating from core annotations to the collaboration extension, implementations SHOULD convert existing annotations to the collaboration format and remove the core annotations file. The two formats SHOULD NOT coexist in the same document to avoid confusion. + ### 1.6 Specification Organization This specification is organized into the following sections: diff --git a/spec/core/04-presentation-layers.md b/spec/core/04-presentation-layers.md index c8271e7..a09dd2a 100644 --- a/spec/core/04-presentation-layers.md +++ b/spec/core/04-presentation-layers.md @@ -162,7 +162,7 @@ The `writingMode` property controls the direction of text flow and block progres **Background:** - `backgroundColor` -- `backgroundImage` - URL/path to image (see Section 5.1.2) +- `backgroundImage` - URL/path to image (see Section 5.4) - `backgroundSize` - `cover`, `contain`, or dimensions - `backgroundPosition` - Position keywords or values - `backgroundRepeat` - `repeat`, `no-repeat`, `repeat-x`, `repeat-y` diff --git a/spec/extensions/collaboration/README.md b/spec/extensions/collaboration/README.md index 4253588..c51a1a3 100644 --- a/spec/extensions/collaboration/README.md +++ b/spec/extensions/collaboration/README.md @@ -231,6 +231,25 @@ Suggestion statuses: `pending`, `accepted`, `rejected` Standard emoji identifiers using Unicode CLDR short names (without colons). Examples: `thumbsup`, `heart`, `thinking`, `rocket`. See the [Unicode CLDR annotations](https://cldr.unicode.org/translation/characters-emoji-symbols/short-names-and-keywords) for the canonical list. +### 4.7 Highlights + +```json +{ + "id": "highlight-1", + "type": "highlight", + "anchor": { "blockId": "block-456", "start": 20, "end": 45 }, + "author": { "name": "Reviewer" }, + "created": "2025-01-15T14:00:00Z", + "color": "#ffeb3b", + "note": "Important passage to revisit" +} +``` + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `color` | string | No | Highlight color (CSS color value, defaults to yellow) | +| `note` | string | No | Optional note attached to the highlight | + ## 5. Change Tracking ### 5.1 Overview