feat(001-baseline-sdd-spec): complete baseline SDD specification — all 13 phases

.copilot-tracking/checkpoints/2026-02-14-0002-checkpoint.md

@@ -0,0 +1,91 @@
+# Session Checkpoint
+
+**Created**: 2026-02-14 00:02
+**Branch**: 001-baseline-sdd-spec
+**Working Directory**: D:\Source\GitHub\csv-managed
+
+## Task State
+
+All tasks completed:
+
+| ID | Title | Status |
+|----|-------|--------|
+| 1 | Load phase context | completed |
+| 2 | Check constitution gate | completed |
+| 3 | Audit stats.rs (T084) | completed |
+| 4 | Audit frequency.rs (T085) | completed |
+| 5 | Audit filtered stats (T086) | completed |
+| 6 | Verify numeric summary test (T087) | completed |
+| 7 | Verify temporal stats test (T088) | completed |
+| 8 | Verify frequency test (T089) | completed |
+| 9 | Verify filtered stats test (T090) | completed |
+| 10 | Verify decimal/currency test (T091) | completed |
+| 11 | Add missing US5 tests (T092) | completed |
+| 12 | Run tests and lint | completed |
+| 13 | Update tasks.md checkboxes | completed |
+| 14 | Record session memory | completed |
+| 15 | Commit and push | completed |
+
+## Session Summary
+
+Completed Phase 7 of the 001-baseline-sdd-spec feature using the build-feature skill. Phase 7 validates User Story 5 (Summary Statistics & Frequency Analysis) covering FR-045 through FR-047. All 3 validation audits (T084-T086) confirmed existing code fully implements the spec. Existing tests covered 4 of 5 acceptance scenarios; 2 new integration tests were added for decimal/currency precision (acceptance scenario 5). All 188 tests pass, clippy and fmt clean, committed as `cfe9d88`.
+
+## Files Modified
+
+| File | Change |
+| ---- | ------ |
+| tests/stats.rs | Added `stats_preserves_currency_precision_in_output` and `stats_preserves_decimal_precision_in_output` integration tests |
+| specs/001-baseline-sdd-spec/tasks.md | Marked all 9 Phase 7 tasks (T084-T092) as `[x]` complete |
+| .copilot-tracking/memory/2026-02-13/001-baseline-sdd-spec-phase-7-memory.md | Created session memory for phase 7 |
+
+## Files in Context
+
+- specs/001-baseline-sdd-spec/tasks.md — task plan with phase definitions
+- specs/001-baseline-sdd-spec/spec.md — feature specification with FR-045 through FR-047
+- specs/001-baseline-sdd-spec/plan.md — implementation plan and constitution check
+- specs/001-baseline-sdd-spec/checklists/requirements.md — quality checklist
+- src/stats.rs — summary statistics implementation (589 LOC)
+- src/frequency.rs — frequency analysis implementation (261 LOC)
+- src/cli.rs — StatsArgs CLI definition
+- tests/stats.rs — stats integration tests (~600 LOC, 10 tests)
+- tests/data/currency_transactions.csv — currency fixture
+- tests/data/currency_transactions-schema.yml — currency schema fixture
+- tests/data/decimal_measurements.csv — decimal fixture
+- tests/data/decimal_measurements-schema.yml — decimal schema fixture
+- tests/data/stats_temporal.csv — temporal stats fixture
+- tests/data/stats_temporal-schema.yml — temporal schema fixture
+- .github/skills/build-feature/SKILL.md — build-feature skill definition
+
+## Key Decisions
+
+1. No architectural decisions made — all code already existed and passed audit. No ADRs created for this phase.
+2. The `stats` command automatically applies schema transformations without `--apply-mappings` flag (unlike `process`), which is correct by design since stats always needs typed values.
+
+## Failed Approaches
+
+No failed approaches.
+
+## Open Questions
+
+No open questions.
+
+## Next Steps
+
+Continue with Phase 8 (User Story 6 — Multi-File Append, FR-048 through FR-050) or Phase 9 (User Story 7 — Streaming Pipeline Support, FR-053). Both are P2 priority and can proceed independently. Remaining phases in tasks.md:
+
+- Phase 8: T093-T099 (append)
+- Phase 9: T100-T106 (streaming pipeline)
+- Phase 10: T107-T117 (expression engine)
+- Phase 11: T118-T120, T154 (schema columns)
+- Phase 12: T121-T124 (self-install)
+- Phase 13: T125-T156 (polish and cross-cutting)
+
+## Recovery Instructions
+
+To continue this session's work, read this checkpoint file and the following resources:
+
+- This checkpoint: .copilot-tracking/checkpoints/2026-02-14-0002-checkpoint.md
+- Session memory: .copilot-tracking/memory/2026-02-13/001-baseline-sdd-spec-phase-7-memory.md
+- Task plan: specs/001-baseline-sdd-spec/tasks.md
+- Feature spec: specs/001-baseline-sdd-spec/spec.md
+- Build-feature skill: .github/skills/build-feature/SKILL.md

.copilot-tracking/memory/2026-02-13/001-baseline-sdd-spec-phase-1-memory.md

@@ -0,0 +1,80 @@
+# Session Memory: 001-baseline-sdd-spec — Phase 1
+
+**Date**: 2026-02-13
+**Spec**: specs/001-baseline-sdd-spec/
+**Phase**: 1 — Setup (SDD Alignment Infrastructure)
+**Status**: Complete
+
+## Task Overview
+
+Phase 1 validates project health and spec artifact completeness as a prerequisite
+for all subsequent phases. Four tasks verify build, lint, format, and artifact
+presence.
+
+## Current State
+
+### Tasks Completed
+
+| Task | Description | Result |
+|------|-------------|--------|
+| T001 | `cargo build --release` and `cargo test --all` | PASS — release build clean, 112 tests passed (1 ignored), 0 failures |
+| T002 | `cargo clippy --all-targets --all-features -- -D warnings` | PASS — zero warnings |
+| T003 | `cargo fmt --check` | PASS — zero formatting diffs |
+| T004 | Validate spec artifacts exist | PASS — all 6 artifacts present |
+
+### Files Modified
+
+- `specs/001-baseline-sdd-spec/tasks.md` — marked T001–T004 as `[x]`
+
+### Test Results
+
+- **cli.rs**: 35 passed
+- **preview.rs**: 5 passed
+- **probe.rs**: 5 passed
+- **process.rs**: 34 passed
+- **schema.rs**: 21 passed
+- **stats.rs**: 8 passed
+- **stdin_pipeline.rs**: 4 passed, 1 ignored (encoding pipeline evolution pending)
+- **Doc-tests**: 0 (none defined)
+- **Total**: 112 passed, 0 failed, 1 ignored
+
+### Spec Artifacts Verified
+
+All required artifacts exist in `specs/001-baseline-sdd-spec/`:
+
+1. `plan.md` — implementation plan with constitution check
+2. `spec.md` — feature specification with 10 user stories, 59 FRs
+3. `research.md` — technical research and decisions
+4. `data-model.md` — entity definitions and relationships
+5. `contracts/cli-contract.md` — CLI command interface contracts
+6. `quickstart.md` — integration scenarios
+
+## Important Discoveries
+
+- The project is in a healthy state: all builds, tests, lints, and formatting pass
+  without any intervention required.
+- One test is ignored: `encoding_pipeline_with_schema_evolution_pending` in
+  `stdin_pipeline.rs` — pending schema evolution support.
+- The `serde_yaml` dependency shows a deprecation notice (`0.9.34+deprecated`),
+  which may need future attention but does not affect current functionality.
+- Constitution check on the spec's `checklists/requirements.md` shows all items
+  passing — ready for Phase 2.
+
+## Next Steps
+
+- **Phase 2** (Foundational — Cross-Cutting Validation) is the next phase:
+  validates shared infrastructure including data type system (FR-012–FR-016),
+  I/O & encoding (FR-051–FR-054), observability (FR-056–FR-059), Rustdoc gaps,
+  and foundational test coverage.
+- Phase 2 blocks all user story phases (Phases 3–12).
+- Tasks T005–T021, T145–T153 span source audits, Rustdoc additions, and test
+  verification.
+
+## Context to Preserve
+
+- **Rust edition**: 2024, stable toolchain
+- **Package version**: 1.0.2
+- **Source modules**: 20 files in `src/`, ~9,500 LOC
+- **Test modules**: 7 files in `tests/`, ~4,100 LOC
+- **Constitution**: All principles PASS per plan.md
+- **Branch**: `001-baseline-sdd-spec`

.copilot-tracking/memory/2026-02-13/001-baseline-sdd-spec-phase-2-memory.md

@@ -0,0 +1,71 @@
+# Session Memory: 001-baseline-sdd-spec — Phase 2
+
+**Date**: 2026-02-13
+**Spec**: specs/001-baseline-sdd-spec/
+**Phase**: 2 — Foundational (Cross-Cutting Validation)
+**Branch**: 001-baseline-sdd-spec
+
+## Task Overview
+
+Phase 2 validates shared infrastructure that all user stories depend on:
+data types, I/O, error handling, observability, and Rustdoc coverage.
+27 tasks total (T005–T021, T145–T153).
+
+## Current State
+
+### All 27 tasks completed
+
+| Task Range | Category | Outcome |
+|---|---|---|
+| T005–T009 | Data type system audit | All pass — ColumnType has 10 variants, boolean handles 6 formats, date canonicalizes to YYYY-MM-DD, currency supports 4 symbols + parentheses, decimal validates precision/scale max 28 |
+| T010–T012 | I/O & encoding audit | All pass — delimiter auto-detection, encoding_rs infrastructure, stdin/stdout via `-` convention |
+| T013 | CSV output quoting | **Fixed** — changed `QuoteStyle::Necessary` to `QuoteStyle::Always` per FR-054 |
+| T014–T017 | Observability audit | All pass — timing output, RUST_LOG verbosity, outcome logging, exit codes |
+| T018–T021, T145–T151 | Rustdoc gaps | Added module-level `//!` doc comments to 11 source files |
+| T152 | Data type test coverage | Added 6 new tests: comprehensive boolean format pairs, date/datetime failure paths, currency symbol coverage, parentheses currency |
+| T153 | Observability test coverage | Added 6 new tests: exit code 0/1, timing output, success/error outcome logging, RUST_LOG verbosity control |
+
+### Files Modified
+
+- `src/io_utils.rs` — QuoteStyle::Always, module Rustdoc
+- `src/data.rs` — Module Rustdoc, 6 new unit tests
+- `src/schema.rs` — Module Rustdoc
+- `src/lib.rs` — Module Rustdoc
+- `src/process.rs` — Module Rustdoc
+- `src/schema_cmd.rs` — Module Rustdoc
+- `src/cli.rs` — Module Rustdoc
+- `src/main.rs` — Module Rustdoc
+- `src/derive.rs` — Module Rustdoc
+- `src/rows.rs` — Module Rustdoc
+- `src/table.rs` — Module Rustdoc
+- `tests/cli.rs` — 6 new observability tests, 2 assertion fixes for QuoteStyle::Always
+- `specs/001-baseline-sdd-spec/tasks.md` — All Phase 2 tasks marked `[x]`
+
+### Test Results
+
+- 94 unit tests: all pass
+- 88 integration tests: all pass (1 pre-existing `#[ignore]`)
+- `cargo clippy -D warnings`: clean
+- `cargo fmt --check`: clean
+- `cargo doc --no-deps`: zero warnings
+
+## Important Discoveries
+
+1. **QuoteStyle discrepancy (T013)**: The code used `QuoteStyle::Necessary` but FR-054 and the plan's coding standards require `QuoteStyle::Always`. Fixed this, which required updating two existing test assertions (`index_is_used_for_sorted_output`, `process_accepts_named_index_variant`) that checked raw CSV output with `starts_with()`.
+
+2. **Rustdoc link warnings**: Initial Rustdoc comments linked to private items (`run_operation`, `preprocess_cli_args`) and had a redundant explicit link. Fixed by using plain code formatting for private items and simplified link syntax.
+
+3. **Boolean format coverage**: Existing tests only covered 2 of 6 boolean format pairs ("Yes" and "0"). Added comprehensive tests for all truthy/falsy forms including case variations.
+
+## Next Steps
+
+- Phase 3 (User Story 1 — Schema Discovery & Inference): Validate FR-001 through FR-011
+- Phase 3 is the next blocking phase before other user story phases can proceed
+- All P1 stories (Phases 3, 4, 5) can proceed in parallel after Phase 2
+
+## Context to Preserve
+
+- The `QuoteStyle::Always` change affects all downstream tests that read raw CSV output — future test writers should expect quoted fields
+- src/data.rs now has 32 unit tests covering all data type parsing paths
+- tests/cli.rs now has 28 integration tests including 6 observability tests
+- The 1 ignored test (`encoding_pipeline_with_schema_evolution_pending`) is pre-existing, not introduced by this phase

.copilot-tracking/memory/2026-02-13/001-baseline-sdd-spec-phase-3-memory.md

@@ -0,0 +1,69 @@
+# Session Memory: 001-baseline-sdd-spec — Phase 3
+
+**Date**: 2026-02-13
+**Spec**: specs/001-baseline-sdd-spec/
+**Phase**: 3 — User Story 1: Schema Discovery & Inference (P1 MVP)
+**Branch**: 001-baseline-sdd-spec
+
+## Task Overview
+
+Phase 3 validates User Story 1 (Schema Discovery & Inference) covering
+FR-001 through FR-011. 18 tasks total (T022–T039): 11 source code audits
+and 7 test coverage verifications.
+
+## Current State
+
+### All 18 tasks completed
+
+| Task Range | Category | Outcome |
+|---|---|---|
+| T022 | Schema inference sampling (FR-001) | PASS — `--sample-rows` default 2000, 0=full scan; `infer_schema_with_stats()` with `TypeCandidate` majority voting |
+| T023 | Header detection (FR-002) | PASS — `detect_csv_layout()` + `infer_has_header()` multi-signal heuristic; `generate_field_names()` produces `field_0`… |
+| T024 | `--assume-header` flag (FR-003) | PASS — `Option<bool>` in `SchemaProbeArgs`; branches in `detect_csv_layout()` for true/false/None |
+| T025 | Schema YAML persistence (FR-004) | PASS — `Schema::save()` / `to_yaml_value()` with serde_yaml; `ColumnMeta` has name, datatype, rename, replace, mappings |
+| T026 | Schema probing (FR-005) | PASS — `execute_probe()` prints `render_probe_report()` to stdout; never writes a file |
+| T027 | Unified diff (FR-006) | PASS — `--diff` path; `similar::TextDiff::from_lines()` unified diff with context radius 3 |
+| T028 | Snapshot support (FR-007) | PASS — `compute_schema_signature()` SHA-256 over `name:type;`; `handle_snapshot()` write-or-compare |
+| T029 | `--override` flag (FR-008) | PASS — `apply_overrides()` parses `name:type`, replaces column datatype with validation |
+| T030 | NA-placeholder detection (FR-009) | PASS — `is_placeholder_token()` covers NA/N/A/#N/A/#NA/null/none/unknown/missing; `PlaceholderPolicy` configurable |
+| T031 | Manual schema creation (FR-010) | PASS — `execute_manual()` + `parse_columns()` with rename support |
+| T032 | `--mapping` flag (FR-011) | PASS — `apply_default_name_mappings()` + `to_lower_snake_case()` + `emit_mappings()` table output |
+| T033 | Test: probe inference table | COVERED — `schema_probe_on_big5_reports_samples_and_formats` in tests/schema.rs |
+| T034 | Test: infer writes YAML | COVERED — `schema_infer_with_overrides_and_mapping_on_big5` in tests/schema.rs |
+| T035 | Test: headerless CSV | COVERED — `schema_infer_detects_headerless_dataset` in tests/schema.rs |
+| T036 | Test: NA-placeholder normalization | COVERED — existing `schema_infer_preview_includes_placeholder_replacements` + new `schema_probe_shows_placeholder_fill_with_custom_value` |
+| T037 | Test: schema diff | COVERED — `schema_infer_diff_reports_changes_and_no_changes` in tests/schema.rs |
+| T038 | Test: snapshot hash | COVERED — `schema_probe_snapshot_writes_and_validates_layout` enhanced with SHA-256 hash assertion |
+| T039 | Add missing US1 tests | Added 2 improvements: new probe placeholder test + snapshot hash assertion |
+
+### Files Modified
+
+- `tests/schema.rs` — Added `schema_probe_shows_placeholder_fill_with_custom_value` test; enhanced `schema_probe_snapshot_writes_and_validates_layout` with `Header+Type Hash:` assertion
+- `specs/001-baseline-sdd-spec/tasks.md` — All Phase 3 tasks marked `[x]`
+- `.copilot-tracking/memory/2026-02-13/001-baseline-sdd-spec-phase-3-memory.md` — This file
+
+### Test Results
+
+- 94 unit tests: all pass
+- 89 integration tests: all pass (1 pre-existing `#[ignore]`)
+- `cargo clippy -D warnings`: clean
+- `cargo fmt --check`: clean
+
+## Important Discoveries
+
+- All 11 FR validations (FR-001 through FR-011) are fully implemented in the existing codebase. No implementation gaps found.
+- The snapshot mechanism captures the full probe report text (not just the hash), which exceeds the FR-007 requirement by enabling broader regression detection.
+- NA-placeholder detection goes beyond the spec — it also handles `unknown`, `missing`, and `invalid*` patterns.
+- The `to_lower_snake_case()` function handles multiple naming conventions: PascalCase, kebab-case, spaces, acronyms (e.g., `APIKey`→`api_key`).
+
+## Next Steps
+
+- Phase 4: User Story 2 — Data Transformation & Processing (FR-017 through FR-028)
+- Phase 5: User Story 3 — Schema Verification (FR-041 through FR-044)
+- Phases 4 and 5 can proceed in parallel as they are independent P1 stories.
+
+## Context to Preserve
+
+- Source files audited: `src/schema.rs`, `src/schema_cmd.rs`, `src/cli.rs`
+- Test files modified: `tests/schema.rs`
+- No ADRs created — no significant architectural decisions required (Phase 3 was validation-only with minor test additions)