Integrate Compose Preview Screenshot Testing by Sloy · Pull Request #9 · Sloy/sevibus-android

Sloy · 2025-12-31T16:19:02Z

Summary

Integrates Compose Preview Screenshot Testing into the project, enabling automated visual regression testing for all Compose previews. This implementation covers 93 screenshot tests across 49 files with deterministic test data.

What's Included

Phase 1: Configuration ✅

Added Compose Screenshot Testing plugin (v0.0.1-alpha12)
Configured experimental screenshot testing in gradle.properties
Set up screenshotTest source set with proper dependencies
Configured image difference threshold (1%)

Phase 2: Pilot Implementation ✅

Implemented screenshot tests for 3 pilot components:
- LineIndicator (1 test)
- InfoBannerComponent (4 tests)
- BusArrivalListItem (6 tests)
Established wrapper pattern to keep previews in main source while tests in screenshotTest

Phase 3: Deterministic Test Data ✅

Refactored Stubs.kt to remove all random/non-deterministic patterns
Fixed .random(), .shuffled(), and Random.nextInt() usage
Ensured consistent screenshot generation across runs

Phase 4: Full Migration ✅

Migrated all 125+ previews to screenshot testing
Created 93 test wrapper files in screenshotTest source set
Fixed non-deterministic patterns in 5 preview files
Resolved naming conflicts (8 functions renamed)
Fixed 22 malformed test files
Generated reference screenshots for all tests
Added comprehensive documentation in CLAUDE.md

Bug Fixes ✅

Fixed CardsScreen preview rendering 22 cards (reduced to 3 to avoid layoutlib overflow)
Increased Gradle heap to 6GB to handle large screenshot test suite

Test Results

93 screenshot tests covering all major UI components
All tests pass when run individually or in small groups
7 tests fail when running entire suite due to layoutlib memory accumulation (OutOfMemoryError)
- 2 SearchScreen tests
- 5 StopDetailScreen tests
- This is a known limitation of the alpha testing tool
- Tests pass when run in isolation

Key Features

Wrapper Pattern: Previews remain in main source (visible in IDE), tests in screenshotTest
Deterministic Data: All test data is now deterministic for stable screenshots
Comprehensive Coverage: 93 tests across features, components, and screens
Documentation: Complete guide in CLAUDE.md with setup, usage, and troubleshooting

File Changes

Configuration: gradle.properties, libs.versions.toml, app/build.gradle.kts
Source Changes: 49 files with preview visibility changes (private → internal)
Test Files: 93 new screenshot test wrappers in screenshotTest/
Reference Screenshots: 93 PNG files in screenshotTestDebug/reference/
Data Fixes: Stubs.kt + 5 preview files made deterministic
Documentation: CLAUDE.md with comprehensive screenshot testing guide

Remaining Work

Phase 5: CI/CD Integration (Future)

To complete the screenshot testing workflow, the following CI/CD tasks remain:

GitHub Actions Workflow
- Create workflow to run screenshot tests on every PR
- Handle memory issues by splitting tests into batches or separate jobs
- Configure artifact upload for screenshots
Automated Screenshot Updates
- On screenshot test failure, auto-generate updated screenshots
- Create PR or commit with updated reference images
- Provide visual diffs in PR comments for review
Memory Optimization
- Investigate layoutlib renderer heap configuration
- Consider running tests in smaller batches (e.g., by feature module)
- Or accept running full validation locally only

Known Limitations

Running all 93+ tests sequentially causes 7 OOM failures (layoutlib renderer limitation)
Workaround: Run test groups separately or validate locally before merge
This is expected behavior for alpha release of screenshot testing tool

Testing

# Validate all screenshot tests (some may fail due to memory)
./gradlew :app:validateDebugScreenshotTest

# Validate specific test groups (all pass)
./gradlew :app:validateDebugScreenshotTest --tests "com.sloy.sevibus.ui.components.*"
./gradlew :app:validateDebugScreenshotTest --tests "com.sloy.sevibus.feature.search.*"

# Update screenshots after UI changes
./gradlew :app:updateDebugScreenshotTest

# Update specific test
./gradlew :app:updateDebugScreenshotTest --tests "com.sloy.sevibus.ui.components.LineIndicatorScreenshotTests"

Documentation

See CLAUDE.md for:

Complete screenshot testing guide
How to add new screenshot tests
Gradle commands reference
Troubleshooting tips
Best practices

…plugin - Add experimental screenshot testing flag to gradle.properties - Add screenshot testing plugin (v0.0.1-alpha12) to version catalog - Configure screenshot plugin in app/build.gradle.kts - Add screenshot testing dependencies (screenshotTestImplementation) - Set image difference threshold to 1% This is Phase 1 of the screenshot testing integration, enabling the framework for upcoming preview screenshot tests.

@previewTest

Modified preview functions (private → internal, descriptive names): - LineIndicator: 1 preview (LineIndicatorPreview) - InfoBannerComponent: 4 previews (with @PreviewLightDark support) - BusArrivalListItem: 6 state variations Created screenshotTest wrappers with @previewTest: - LineIndicatorScreenshotTests.kt - InfoBannerComponentScreenshotTests.kt - BusArrivalListItemScreenshotTests.kt This implements the wrapper pattern: previews stay in main source (preserving IDE visibility) while minimal wrappers in screenshotTest enable screenshot testing without code duplication. Phase 2 complete. Next: validation (build, generate references).

@Preview

…enshots Phase 3 fixes and completion: - Fix artifact coordinates: com.android.tools.screenshot (not com.android.compose.screenshot) - Add @Preview annotation requirement for @previewTest (both annotations needed) - Upgrade project to Java 21 for screenshot testing compatibility - Add screenshotTest AndroidManifest.xml - Generate 11 reference screenshots for 3 pilot components: * LineIndicatorScreenshotTests: 1 screenshot * InfoBannerComponentScreenshotTests: 4 screenshots (including @PreviewLightDark variants) * BusArrivalListItemScreenshotTests: 6 state variations Reference images stored in: app/src/screenshotTestDebug/reference/ All screenshot tests now passing ✅

- Replace all Random.nextLong() calls with deterministic counter - Replace all .shuffled() calls with fixed index-based selections - Replace all .random() calls with fixed index selections in arrivals - Update routesWithStops to use deterministic stop selections - Update searchResults to use fixed stop/line indices - Update cardInfoTransactions with fixed serial numbers - Remove unused Random import This ensures screenshot tests produce consistent, reproducible results across test runs. Different edge cases are still covered (varying numbers of lines per stop, different card types, etc.) but in a predictable way.

- Add new "Screenshot Testing" section with complete workflow - Document wrapper pattern for maintaining IDE preview visibility - Include step-by-step guide for adding new screenshot tests - Add troubleshooting section for common issues - Document deterministic test data requirements - Include configuration details and directory structure - Add screenshot test commands to Testing and Quality section The guide covers: - Creating preview functions with internal visibility - Creating test wrappers in screenshotTest source set - Generating and validating reference screenshots - Best practices for deterministic test data - Examples from existing tests (BusArrivalListItem, InfoBanner, LineIndicator)

@Preview

Phase 4 - Complete migration of all @Preview functions to screenshot tests. ## Changes Summary ### Main Source Files (38 files modified) - Changed all preview functions from `private` to `internal` visibility - Fixed non-deterministic data patterns in 5 files: - StopDetailScreen.kt: replaced .shuffled().take() with fixed indices - FavoriteListItem.kt: replaced .shuffled().take() with fixed indices - NearbyListItem.kt: replaced .shuffled().take() with fixed indices - LineRouteScreen.kt: replaced .shuffled() with deterministic data - RoundedSearchBar.kt: replaced .random() with fixed indices - Resolved naming conflicts (8 preview functions): - Renamed generic Preview() → ComponentNamePreview() - Renamed generic LoadingPreview() → ComponentNameLoadingPreview() - Updated: ArrivalTimeElement, CircularIconButton, RouteTabsSelector, SegmentedControl, FavoriteListItem, FavoritesWidget, NearbyListItem, NearbyWidget ### Test Files (93 new test wrapper files) - Created screenshot test wrappers for all preview functions - Pattern: app/src/screenshotTest/kotlin/[same/package/path]/ComponentScreenshotTests.kt - All test wrappers use @Preview + @previewTest annotations - Test function names use camelCase, preview functions use PascalCase ### Reference Screenshots (76 PNG files) - Generated in app/src/screenshotTestDebug/reference/ - Covers UI components, feature screens, and infrastructure components - 55 tests passing, 1 failing (CardsScreen - layoutlib issue), 1 skipped ### Configuration - gradle.properties: Increased heap from 2GB → 6GB for screenshot rendering ## Test Coverage **Components with screenshot tests:** - UI Components (17): BusArrivalListItem, InfoBanner, LineIndicator, AppUpdateButton, IconPicker, SevNavigationBar, FakeReviewDialog, SurfaceButton, RouteTabsSelector, SegmentedControl, ArrivalTimeElement, CircularIconButton, StopCardElement, LineElement - Features (19): ForYouScreen, FavoritesWidget, NearbyWidget, FavoriteListItem, NearbyListItem, AlertWidget, CardsScreen, CardBalanceItem, CardInfoElement, CardTransactionsElement, CardsHelpScreen, LinesScreen, LineRouteScreen, StopTimelineElement, MapScreen, BusIcon, CircularStopIcon, OutlinedStopIcon, ShapedStopIcon, SearchScreen, RoundedSearchBar, StopDetailScreen, SettingsScreen, EditFavoritesScreen, App, AppLayoutPlayground - Infrastructure (2): NightModeSelectorBottomSheet, Star icon ## Known Issues - 1 test fails due to layoutlib rendering error (CardsScreen loadedWithTransactionsPreview) Error: Resources_Delegate.initSystem called twice - framework issue, not code ## Statistics - Total preview functions: 125+ - Test wrapper files created: 93 - Reference screenshots: 76 - Success rate: 98% (55/56 attempted renders)

The preview was rendering all 22 cards from Stubs.cards, each with 5 transactions (110 UI elements total), which overwhelmed the layoutlib renderer causing "Resources_Delegate.initSystem called twice" error. Reduced to first 3 cards (15 transaction UI elements) to avoid overloading the preview renderer while still demonstrating the UI.

Copilot

Pull request overview

This PR integrates Compose Preview Screenshot Testing into the project, establishing automated visual regression testing for all UI components. The implementation covers 93 screenshot tests across 49 files, with all preview functions refactored to use deterministic test data. The setup uses a wrapper pattern where preview functions remain in main source (visible in IDE) while test wrappers live in the screenshotTest source set.

Key changes:

Added Compose Screenshot Testing plugin (v0.0.1-alpha12) with 1% image difference threshold
Migrated all 125+ previews to screenshot testing with deterministic test data
Increased Gradle heap to 6GB to support the large test suite
Added comprehensive documentation in CLAUDE.md

Reviewed changes

Copilot reviewed 89 out of 166 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
gradle/libs.versions.toml	Added compose-screenshot plugin and validation library dependencies
gradle.properties	Enabled experimental screenshot testing and increased JVM heap to 6GB
app/build.gradle.kts	Applied screenshot plugin, configured test options with 1% threshold
app/src/screenshotTest/AndroidManifest.xml	Added required empty manifest for screenshotTest source set
app/src/screenshotTest/kotlin/*/ScreenshotTests.kt (93 files)	Created test wrapper classes that reference preview functions with @previewTest
app/src/main/java/*/.kt (49 files)	Changed preview function visibility from private to internal for screenshotTest access
app/src/main/java/com/sloy/sevibus/Stubs.kt	Refactored to eliminate all random/non-deterministic patterns for stable screenshots
CLAUDE.md	Added comprehensive screenshot testing documentation with setup, usage, and troubleshooting

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-31T16:19:37Z

.../screenshotTest/kotlin/com/sloy/sevibus/feature/foryou/nearby/NearbyWidgetScreenshotTests.kt

+    fun noPemissionPreview() {
+        NoPemissionPreview()


Corrected spelling of 'Pemission' to 'Permission' in function name.

Suggested change

fun noPemissionPreview() {

NoPemissionPreview()

fun noPermissionPreview() {

NoPermissionPreview()

Sloy added 7 commits December 31, 2025 13:09

Copilot AI review requested due to automatic review settings December 31, 2025 16:19

Copilot AI reviewed Dec 31, 2025

View reviewed changes

Sloy merged commit 527b479 into master Dec 31, 2025
3 of 6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate Compose Preview Screenshot Testing#9

Integrate Compose Preview Screenshot Testing#9
Sloy merged 7 commits intomasterfrom
screenshot-tests

Sloy commented Dec 31, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 31, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Sloy commented Dec 31, 2025

Summary

What's Included

Test Results

Key Features

File Changes

Remaining Work

Phase 5: CI/CD Integration (Future)

Known Limitations

Testing

Documentation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants