Skip to content

Add visual regression testing to CI #338

@mmcky

Description

@mmcky

Summary

Add automated visual regression testing to catch unintended styling changes during theme development.

Context

During review of PR #335 (theme refactor), we performed a structural/content check comparing preview builds against production. However, this only verified HTML content parity - not actual CSS styling, colors, layout, or spacing.

True visual regression testing would catch:

  • CSS styling changes
  • Font rendering differences
  • Layout/spacing issues
  • Dark mode toggle behavior
  • Responsive design breakpoints

Suggested Tools

Tool Type Notes
Percy SaaS GitHub integration, visual diffs
Playwright OSS Screenshot comparisons in CI
BackstopJS OSS Config-driven visual regression
Chromatic SaaS Works well with component libraries

Implementation Ideas

  1. Build a test site using the theme (or use an existing lecture site build)
  2. Capture baseline screenshots of key page types:
    • Homepage/intro
    • Lecture with code blocks
    • Lecture with math equations
    • Lecture with figures
    • Mobile viewport
  3. Run visual diff on PRs before merge
  4. Store baseline snapshots in repo or CI cache

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions