Skip to content

Add tested examples to documentation #17

@titusz

Description

@titusz

Problem

Documentation code examples are not tested. When the API changes, examples can silently become outdated or broken, eroding user trust.

Goal

Ensure code examples in documentation are executable and verified by CI. This catches API drift and guarantees that users can copy-paste examples that actually work.

Current State

  • 100+ code examples across 16 documentation files (tutorials, how-to guides, index page)
  • Examples are well-written but untested
  • Many tutorial examples are sequential (build state across multiple code blocks)
  • Some examples involve file I/O (save/load/restore)
  • Print output in comments uses # result style, not assertion-based

Considerations

  • Sequential code blocks: Tutorials build state across blocks — need a strategy for self-contained vs. chained examples
  • File I/O: Examples writing to disk need temp directory handling
  • Non-deterministic output: HNSW results (key ordering, distances) may vary slightly
  • MkDocs tab syntax: === "Tab name" indented code blocks may need special handling
  • Tooling options: pytest-examples, doctest, plain pytest with extracted snippets, or other approaches — to be evaluated

Acceptance Criteria

  • Documentation examples are executed and validated in CI
  • Approach chosen balances test coverage with documentation readability
  • No degradation of the current documentation quality

Metadata

Metadata

Assignees

No one assigned

    Labels

    Priority: MediumMedium priority issuedocumentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions