Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions gradle/libs.versions.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ junit-jupiter-api = { module = "org.junit.jupiter:junit-jupiter-api", version.re
junit-jupiter-engine = { module = "org.junit.jupiter:junit-jupiter-engine", version.ref = "junit" }
junit-jupiter-params = { module = "org.junit.jupiter:junit-jupiter-params", version.ref = "junit" }
junit-platform-launcher = { module = "org.junit.platform:junit-platform-launcher", version = "6.0.2" }
playwright = { module = "com.microsoft.playwright:playwright", version = "1.51.0" }
selenium = { module = "org.seleniumhq.selenium:selenium-java", version = "4.28.1" }
spock = { module = "org.spockframework:spock-core", version = "2.4-M7-groovy-5.0" }
testng = { module = "org.testng:testng", version = "7.12.0" }
wiremock = { module = "org.wiremock:wiremock", version.ref = "wiremock" }
Expand Down
4 changes: 4 additions & 0 deletions manual/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,14 @@ testing {
useJUnitJupiter()
dependencies {
implementation(project(":modules:core"))
implementation(project(":modules:image"))
implementation(project(":modules:json-jackson"))
implementation(project(":modules:yaml-jackson"))
implementation(project(":modules:http"))

implementation(libs.playwright)
implementation(libs.selenium)

implementation(libs.jackson2.databind)
implementation(libs.jackson2.dataformat.yaml)
implementation(libs.jackson2.jsr310)
Expand Down
283 changes: 283 additions & 0 deletions manual/src/docs/asciidoc/chapters/10-images.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,283 @@
= Image Approval

ApproveJ supports approval testing of images, which is particularly useful for visual regression testing.
This is commonly used with browser automation tools like Playwright to capture screenshots and ensure UI consistency across changes.


[id="image_basics"]
== Basic Image Approval

To approve an image, use the `approveImage` static method from `ImageApprovalBuilder`.
It accepts either a `BufferedImage` or a `byte[]` (for direct use with screenshot APIs) and provides a fluent API similar to text approval.

The `byte[]` overload works seamlessly with popular browser automation tools:

[source,java,indent=0,role="primary"]
.Playwright
----
include::../../../test/java/examples/java/ImageDocTest.java[tag=approve_screenshot]
----
<1> Creates an `ImageApprovalBuilder` directly from the screenshot bytes
<2> Compares result to a previously approved image file next to the test

[source,java,indent=0,role="secondary"]
.Selenium
----
include::../../../test/java/examples/java/ImageDocTest.java[tag=approve_screenshot_selenium]
----
<1> Get screenshot as byte array using `OutputType.BYTES`
<2> Pass bytes directly to `approveImage`

This will create files named `<TestClass>-<testMethod>-received.png` and `<TestClass>-<testMethod>-approved.png` next to your test.

When the test runs for the first time, a blank approved file is created.
Copy the received file to the approved file (or use a diff tool) to establish the baseline.
Subsequent runs compare the new screenshot against this approved baseline.


[id="image_comparators"]
== Image Comparators

ApproveJ provides two image comparison strategies, each with different characteristics and use cases.


[id="perceptual_hash"]
=== Perceptual Hash (Default)

Perceptual hashing (pHash) is the default comparison method.
It is robust to minor visual differences that are imperceptible to humans.

[source,java,indent=0]
----
include::../../../test/java/examples/java/ImageDocTest.java[tag=approve_screenshot_phash]
----
<1> Explicitly use perceptual hash comparison (this is the default)

*How it works:*

1. Both images are resized to 32×32 pixels
2. Converted to grayscale
3. A Discrete Cosine Transform (DCT) is applied
4. The low-frequency components are extracted
5. A 64-bit hash is generated based on whether each DCT value exceeds the mean
6. The hashes are compared using Hamming distance

*Strengths:*

* Robust to antialiasing differences across browsers/platforms
* Tolerant of minor font rendering variations
* Handles slight color shifts from compression
* Ignores subpixel rendering differences
* Fast comparison (comparing two 64-bit numbers)

*Weaknesses:*

* May miss small, localized changes (e.g., a single button color change)
* Less precise for pixel-perfect requirements
* Sensitive to significant layout shifts

*Best for:*

* Cross-browser visual testing
* CI/CD environments with different rendering engines
* Testing on multiple operating systems
* General UI regression testing


[id="pixel_comparison"]
=== Pixel Comparison

Pixel-by-pixel comparison calculates the exact difference between each pixel.

[source,java,indent=0]
----
include::../../../test/java/examples/java/ImageDocTest.java[tag=approve_screenshot_pixel]
----
<1> Use pixel comparison with 1% threshold (99% similarity required)

*How it works:*

1. Each pixel's RGB values are compared
2. Differences are weighted by alpha (transparency)
3. Total difference is calculated as a percentage

*Strengths:*

* Precise detection of any visual change
* Good for pixel-perfect design requirements
* Detects small, localized changes

*Weaknesses:*

* Sensitive to antialiasing differences
* Fails on subpixel rendering variations
* Different browsers/platforms may produce slightly different results
* Font rendering differences often cause false failures

*Best for:*

* Same-browser, same-platform testing
* Pixel-perfect design verification
* Detecting subtle changes like icons or colors


[id="thresholds"]
== Configuring Thresholds

Both comparators accept a threshold to control how much difference is acceptable.

[source,java,indent=0]
----
include::../../../test/java/examples/java/ImageDocTest.java[tag=approve_screenshot_threshold]
----
<1> Require at least 95% similarity

*Perceptual Hash Threshold:*

* Default: 0.90 (90% similarity)
* Range: 0.0 to 1.0
* Higher values = stricter matching
* Recommended: 0.85-0.95 for most use cases

*Pixel Comparison Threshold:*

* Default: 0.01 (1% difference allowed)
* Range: 0.0 to 1.0
* Lower values = stricter matching (threshold represents allowed difference)
* Recommended: 0.01-0.05 depending on stability needs


[id="visual_testing_challenges"]
== Common Challenges in Visual Testing

Visual approval testing can be tricky due to various sources of non-determinism.
Here are common problems and solutions.


[id="animations"]
=== Animations and Transitions

CSS animations and transitions can cause screenshots to capture intermediate states, leading to flaky tests.

[source,java,indent=0]
----
include::../../../test/java/examples/java/ImageDocTest.java[tag=approve_screenshot_animations]
----
<1> Wait for all resources to load
<2> Inject CSS to disable all animations and transitions


[id="dynamic_content"]
=== Dynamic Content

Content that changes between runs (timestamps, ads, user-specific data) will cause failures.

*Solutions:*

* Mock or stub dynamic data sources
* Screenshot specific elements instead of full pages
* Use CSS to hide dynamic elements before capturing
* Scrub dynamic content with `display: none`

[source,java,indent=0]
----
include::../../../test/java/examples/java/ImageDocTest.java[tag=approve_screenshot_element]
----
<1> Screenshot only the h1 element


[id="font_rendering"]
=== Font Rendering Differences

Different operating systems render fonts differently, causing pixel-level variations.

*Solutions:*

* Use perceptual hash comparison (default) -- it's designed for this
* Run tests in containerized environments with consistent fonts
* Use web fonts that render consistently across platforms
* Lower pixel comparison threshold if you must use pixel comparison


[id="viewport_consistency"]
=== Viewport and Resolution

Different screen sizes and DPI settings affect rendering.

*Solutions:*

* Always set explicit viewport size: `page.setViewportSize(1280, 720)`
* Use consistent device scale factor
* Document and enforce the expected resolution in CI


[id="network_timing"]
=== Network and Loading Timing

Images or external resources may not be fully loaded when the screenshot is taken.

*Solutions:*

* Use `page.waitForLoadState()` to wait for network idle
* Wait for specific elements: `page.waitForSelector(".hero-image")`
* Add explicit waits for lazy-loaded content
* Consider using `page.waitForTimeout()` as a last resort (not recommended for production)


[id="scrolling_and_lazy_loading"]
=== Scrolling and Lazy Loading

Content below the fold may not be rendered or may trigger lazy loading.

*Solutions:*

* Scroll to the element before screenshotting
* Use full-page screenshots: `page.screenshot(new Page.ScreenshotOptions().setFullPage(true))`
* Trigger lazy loading by scrolling, then scroll back


[id="cursor_and_focus"]
=== Cursor and Focus States

The cursor position or focused elements can vary between runs.

*Solutions:*

* Click elsewhere before taking screenshot: `page.click("body")`
* Move mouse to consistent position: `page.mouse().move(0, 0)`
* Remove focus rings via CSS injection


[id="choosing_comparator"]
== Choosing the Right Comparator

|===
| Scenario | Recommended Comparator | Threshold

| Cross-browser testing
| Perceptual Hash
| 0.85-0.90

| Same browser, CI/CD
| Perceptual Hash
| 0.90-0.95

| Pixel-perfect design
| Pixel
| 0.01-0.02

| Component screenshots
| Perceptual Hash
| 0.90

| Icon/logo verification
| Pixel
| 0.005-0.01

| Full page screenshots
| Perceptual Hash
| 0.85-0.90
|===

When in doubt, start with perceptual hash comparison.
It provides a good balance between catching real regressions and avoiding false positives from rendering differences.
3 changes: 2 additions & 1 deletion manual/src/docs/asciidoc/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,5 @@ include::chapters/06-reviewing.adoc[leveloffset=1]
include::chapters/07-json-jackson.adoc[leveloffset=1]
include::chapters/08-yaml-jackson.adoc[leveloffset=1]
include::chapters/09-http.adoc[leveloffset=1]
include::chapters/10-configuration.adoc[leveloffset=1]
include::chapters/10-images.adoc[leveloffset=1]
include::chapters/11-configuration.adoc[leveloffset=1]
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading