diff --git a/.claude/skills/ai-test-runner/SKILL.md b/.claude/skills/ai-test-runner/SKILL.md new file mode 100644 index 000000000000..c8f611c1dd9d --- /dev/null +++ b/.claude/skills/ai-test-runner/SKILL.md @@ -0,0 +1,248 @@ +--- +name: ai-test-runner +description: >- + Run a suite of AI-driven test cases against the WordPress and Jetpack iOS + app in a simulator. Use when asked to run a test suite, run AI tests, or + execute test cases in a directory. +--- + +# AI Test Runner + +Run plain-language test cases against the WordPress or Jetpack iOS app on an +iOS Simulator. Each test case is a markdown file with Prerequisites, Steps, +and Expected Outcome. Claude Code navigates the app UI autonomously using +WebDriverAgent. + +## Phase 1: Collect Credentials + +Before running any tests, ask the user for the following using AskUserQuestion. + +- **App**: Which app to test — WordPress or Jetpack +- **Site URL**: The WordPress site URL (e.g., `https://example.com`) +- **Username**: The WordPress username or email +- **Application Password**: A WordPress application password for REST API access +- **Test directory**: Path to the directory containing test case markdown files + +Here are the app bundle IDs: +- **WordPress**: `org.wordpress` +- **Jetpack**: `com.automattic.jetpack` + +## Phase 2: Discover Tests + +1. Use Glob to find all `*.md` files in the directory the user specified. +2. Sort them alphabetically by filename. +3. Print the list of discovered tests to the terminal: + ``` + Discovered N test(s) in : + - view-media-library.md + - view-posts-list.md + - view-site-settings.md + ``` + +If no `.md` files are found, tell the user and stop. + +## Phase 3: Start WDA + +1. Run the WDA start script, which locates at `scripts/wda-start.rb` in the + `ios-sim-navigation` skill directory. This may take up to 60 seconds the first time. + +2. Create a WDA session: + ```bash + curl -s -X POST http://localhost:8100/session \ + -H 'Content-Type: application/json' \ + -d '{"capabilities":{"alwaysMatch":{}}}' + ``` + Extract the session ID from `value.sessionId` in the response. + +3. Get the booted simulator UDID for screenshots: + ```bash + xcrun simctl list devices booted -j | jq -r '.devices | to_entries[].value[] | select(.state == "Booted") | .udid' + ``` + +If WDA fails to start or no simulator is booted, tell the user and stop. + +## Phase 4: Initialize Results Directory + +1. Compute the timestamp as `YYYY-MM-DD-HHmm` from the current date and time. +2. Determine the suite name from the test directory's last path component (e.g., `ui-tests`). +3. Derive the base directory as the **parent** of the test directory (e.g., if the test + directory is `ai-tests/ui-tests`, the base directory is `ai-tests/`). +4. Create the per-test results directory: `mkdir -p /results/-` +5. Create the screenshots directory: `mkdir -p /results/screenshots` +6. Store the results directory path, screenshots directory path, timestamp, and suite name + in context for use in later phases. + +## Phase 5: Run Tests + +Run each test case **sequentially**. Tests share one simulator so they must not +run in parallel. + +Track pass/fail/remaining counts in-context (incrementing counters). + +### For each test case: + +#### Step 1: Dispatch subagent + +Call the Agent tool with `subagent_type: general-purpose` and a prompt +constructed from the template below. + +Build the prompt by filling in the `` with actual values: + +```` +You are running a single test case against the iOS app in a simulator +using WebDriverAgent (WDA). + +Use the ios-sim-navigation skill for WDA interaction reference. + +## Context + +- App Bundle ID: +- WDA Session ID: +- Simulator UDID: +- Test file: (absolute path) +- Per-test results directory: (absolute path) +- Site URL: +- Username: +- Application Password: +- Screenshots directory: (absolute path) + +## Instructions + +1. **Read the test file** at ``. It contains the information + needed to execute the test: prerequisites, steps, expected outcome, etc. + + Derive the test filename (without extension) from the file path for use + in result files and screenshots. + +2. **Relaunch the app** for a clean state: + + ```bash + xcrun simctl launch --terminate-running-process \ + -ui-test-site-url \ + -ui-test-site-user \ + -ui-test-site-pass + ``` + + Wait 2-3 seconds for the app to finish loading. + + The app may already be logged in to the site. Check the accessibility tree + to determine if login is required. If the app is already showing the + logged-in state (e.g., My Site screen), skip login. + + If the app shows a login/signup screen, log in using these steps: + + 1. Tap the **"Enter your existing site address"** button. + 2. Type the exact site URL value into the site address text field. + 3. Tap **Continue**. The app will auto-login after this. + + Wait 2-3 seconds for the app to finish loading after login. + +3. **Fulfill prerequisites** from the test file. + + For REST API prerequisites (e.g., creating tags, categories, or posts), + make the API calls using the site URL, username, and application password. + For UI prerequisites like "Logged in to the app with the test account", + the app relaunch in step 2 handles this automatically. + + If a prerequisite cannot be fulfilled, mark the test as FAIL with reason + "Prerequisite not met:
" and skip to the result writing step. + +4. **Execute the test case** following the steps, expected outcome, and any + verification/cleanup sections in the test file. Use WDA for all UI + interactions (refer to the ios-sim-navigation skill). Perform any REST API + cleanup regardless of pass/fail. + +5. **Write per-test result file** at + `/.md`: + + On pass — write: + ``` + ### PASS + Passed. + ``` + + On fail — take a failure screenshot, save it, then write: + ```bash + xcrun simctl io screenshot /-failure.png + ``` + ``` + ### FAIL + **Failure reason:** + **Screenshot:** screenshots/-failure.png + ``` + +6. **End your response** with exactly one of these lines as the very last line: + ``` + RESULT: PASS + ``` + or: + ``` + RESULT: FAIL: + ``` + +IMPORTANT: Prefer the accessibility tree over screenshots. After every tap or +swipe, wait 0.5-1 seconds then re-fetch the tree to see the updated UI state. +```` + +#### Step 2: Parse subagent response + +After the subagent returns, parse its response: + +- Extract the last line for `RESULT: PASS` or `RESULT: FAIL: `. +- Update the in-context counters accordingly. + +#### Step 3: Print status update + +``` +[2/5] PASS: create-blank-page +``` +or: +``` +[2/5] FAIL: create-blank-page — +``` + +## Phase 6: Cleanup and Assemble Results + +1. Stop WDA: + ```bash + ruby ~/.claude/skills/ios-sim-navigation/scripts/wda-stop.rb + ``` + +2. **Assemble the final results file** at `/results/-.md`: + - Read all per-test result files from `/results/-/` + - Sort them alphabetically by filename + - Write the assembled file with this structure: + ``` + # Test Results: + + - **Date:** + - **Site:** + - **Total:** N | **Passed:** P | **Failed:** F + + ## Results + + + ``` + +3. Print the final summary to the terminal: + ``` + Test run complete. + Total: N | Passed: P | Failed: F + Results: /results/-.md + ``` + +## Important Notes + +- The app MUST already be built and installed on a booted simulator. The app + is relaunched and logged in if needed at the start of each test. +- Each test case runs in its own subagent to keep the main context lean. + The subagent relaunches the app for a clean state before each test. +- Prefer the accessibility tree over screenshots for all simulator interactions. +- NEVER stop on a test failure. Always continue to the next test. +- After every tap or swipe, wait 0.5-1 seconds then re-fetch the accessibility + tree to see the updated UI state. +- For scrolling, swipe from `(screen_width - 30, screen_height / 2)` upward + to avoid accidentally tapping interactive elements in the center. +- Save failure screenshots to the derived screenshots directory (`/results/screenshots/`). +- Each subagent writes its own per-test result file. The final results file is + assembled in Phase 6 after all tests complete. diff --git a/.claude/skills/ios-sim-navigation/SKILL.md b/.claude/skills/ios-sim-navigation/SKILL.md new file mode 100644 index 000000000000..0ec4ab7efab1 --- /dev/null +++ b/.claude/skills/ios-sim-navigation/SKILL.md @@ -0,0 +1,372 @@ +--- +name: ios-sim-navigation +description: General-purpose skill for navigating and interacting with an iOS app running in a Simulator using WebDriverAgent (WDA). Use when the user asks to tap buttons, swipe, scroll, type text, check what's on screen, go to a tab or screen, automate a flow, or verify UI state in a simulator app. Also use when the user wants to take screenshots, inspect the accessibility tree, explore screen hierarchy, or test a UI flow end-to-end on a simulator. Even if the user says something casual like "open settings in the app", "click that button", or "what's showing on the simulator" — this skill applies. +--- + +# iOS Simulator Navigation with WebDriverAgent + +## Prerequisites + +- Xcode with iOS Simulators installed +- WebDriverAgent built for simulator use (see Setup below) +- The app must be built and installed on the target simulator + +### First-Time Setup + +Clone and build WebDriverAgent: + +```bash +mkdir -p .build +git clone https://github.com/appium/WebDriverAgent.git .build/WebDriverAgent +cd .build/WebDriverAgent +xcodebuild build-for-testing \ + -project WebDriverAgent.xcodeproj \ + -scheme WebDriverAgentRunner \ + -destination "platform=iOS Simulator,name=iPhone 17" \ + CODE_SIGNING_ALLOWED=NO +``` + +## WDA Lifecycle + +Start and stop WDA using the lifecycle scripts. **WDA must be running before using any curl commands below.** + +```bash +# Start WDA (waits until ready, ~60s first time) +ruby scripts/wda-start.rb [--udid ] [--port ] + +# Check if WDA is running +curl -s http://localhost:8100/status | head -c 200 + +# Stop WDA +ruby scripts/wda-stop.rb [--port ] +``` + +Both scripts auto-detect the first booted simulator. Use `--udid` to target a specific one. + +## Strategy: Tree-First Navigation + +**Always prefer the accessibility tree over screenshots.** The tree is text-based, faster to process, and doesn't require viewing an image. + +1. Fetch the tree with `GET /source?format=description` +2. Make decisions from the tree alone +3. Only take a screenshot when the tree doesn't contain enough info (e.g., verifying visual layout) + +## Accessibility Tree + +WDA offers two tree formats via `GET /source?format=`: + +### `format=description` -- compact plaintext (~25 KB) + +```bash +curl -s 'http://localhost:8100/source?format=description' | jq -r .value +``` + +Returns a human-readable indented tree. Each line shows an element with its type, memory address, frame as `{{x, y}, {width, height}}`, and optional attributes (identifier, label, Selected, etc.): + +``` +NavigationBar, 0x105351660, {{0.0, 62.0}, {402.0, 54.0}}, identifier: 'my-site-navigation-bar' + Button, 0x105351a20, {{16.0, 62.0}, {44.0, 44.0}}, identifier: 'BackButton', label: 'Site Name' + StaticText, 0x105351b40, {{178.7, 73.7}, {44.7, 20.7}}, label: 'Posts' +``` + +**Use this format by default.** It's ~15x smaller than JSON, easy to reason about, and contains all the information needed for navigation (types, labels, identifiers, and coordinates). + +### `format=json` -- structured data (~375 KB) + +```bash +curl -s 'http://localhost:8100/source?format=json' > /tmp/wda-tree.json +``` + +Returns deeply nested JSON. Use this when you need to programmatically extract coordinates or search for elements with `jq`. The response has the structure `{"value": , "sessionId": "..."}`. Each node has: + +| Field | Description | +|-------|-------------| +| `type` | Element type (e.g., `Button`, `StaticText`, `NavigationBar`) | +| `label` | Accessibility label (user-visible text) | +| `name` | Accessibility identifier (developer-assigned ID) | +| `value` | Current value (e.g., text field contents, switch state) | +| `rect` | `{"x": N, "y": N, "width": N, "height": N}` -- structured, use for tap coordinates | +| `frame` | Same as rect but as a string: `"{{x, y}, {w, h}}"` | +| `isEnabled` | Whether the element is interactive | +| `children` | Array of child nodes | + +Search example with `jq`: + +```bash +cat /tmp/wda-tree.json | jq '.. | objects | select(.label == "Settings")' +``` + +### Computing Tap Coordinates + +From the description format, parse the frame `{{x, y}, {width, height}}` and compute: + +``` +tap_x = x + width / 2 +tap_y = y + height / 2 +``` + +From the JSON format, use the `rect` object: + +``` +tap_x = rect.x + rect.width / 2 +tap_y = rect.y + rect.height / 2 +``` + +### Finding Elements + +Use this priority order when locating elements in the tree: + +1. **`identifier` / `name`** -- most stable; developer-assigned, unlikely to change across locales +2. **`label`** -- accessibility label; user-visible text, may change with localization +3. **`type` + context** -- e.g., "Button inside NavigationBar" or "Cell inside Table" +4. **Partial matching** -- element label *contains* the target text (useful for dynamic labels like "3 Posts") +5. **Positional heuristics** -- last resort; fragile across screen sizes + +In the description format, search the text output for labels or identifiers. In the JSON format, use `jq`: + +```bash +# Exact match by identifier +cat /tmp/wda-tree.json | jq '.. | objects | select(.name == "settings-button")' + +# Exact match by label +cat /tmp/wda-tree.json | jq '.. | objects | select(.label == "Settings")' + +# Partial match by label +cat /tmp/wda-tree.json | jq '.. | objects | select(.label? // "" | contains("Settings"))' + +# Type + context: find Buttons inside NavigationBar +cat /tmp/wda-tree.json | jq '.. | objects | select(.type == "NavigationBar") | .. | objects | select(.type == "Button")' +``` + +### Screen Size + +The root node's `rect` gives the screen dimensions (e.g., `width: 393, height: 852`). + +## Session Management + +Most action endpoints require a session ID. Create one if `/status` doesn't return a `sessionId`: + +```bash +# Create session +curl -s -X POST http://localhost:8100/session \ + -H 'Content-Type: application/json' \ + -d '{"capabilities":{"alwaysMatch":{}}}' | jq . +``` + +The session ID is at `value.sessionId` in the response. Use it in subsequent action URLs as `SESSION_ID`. + +To check for an existing session, look at the `sessionId` field in the `/status` response. + +## Actions + +All action endpoints use `POST /session/SESSION_ID/actions` with W3C WebDriver pointer actions. + +### Tap + +```bash +curl -s -X POST http://localhost:8100/session/SESSION_ID/actions \ + -H 'Content-Type: application/json' \ + -d '{ + "actions": [{ + "type": "pointer", + "id": "finger1", + "parameters": {"pointerType": "touch"}, + "actions": [ + {"type": "pointerMove", "duration": 0, "x": X, "y": Y}, + {"type": "pointerDown"}, + {"type": "pointerUp"} + ] + }] + }' +``` + +#### Alternative: Element-Based Tapping + +WDA can find and tap elements directly without computing coordinates. This is useful when an element has a stable accessibility identifier: + +```bash +# Find the element by accessibility identifier +curl -s -X POST http://localhost:8100/session/SESSION_ID/elements \ + -H 'Content-Type: application/json' \ + -d '{"using": "accessibility id", "value": "settings-button"}' | jq . + +# Tap it (ELEMENT_ID comes from the response above, at value[0].ELEMENT) +curl -s -X POST http://localhost:8100/session/SESSION_ID/element/ELEMENT_ID/click +``` + +The coordinate approach above is preferred because it works directly with the tree data already being fetched. Use element-based tapping when coordinate parsing is awkward or when interacting with elements found by predicate. + +### Long Press + +Add a `pause` between `pointerDown` and `pointerUp`. Duration is in milliseconds. + +```bash +curl -s -X POST http://localhost:8100/session/SESSION_ID/actions \ + -H 'Content-Type: application/json' \ + -d '{ + "actions": [{ + "type": "pointer", + "id": "finger1", + "parameters": {"pointerType": "touch"}, + "actions": [ + {"type": "pointerMove", "duration": 0, "x": X, "y": Y}, + {"type": "pointerDown"}, + {"type": "pause", "duration": 1000}, + {"type": "pointerUp"} + ] + }] + }' +``` + +### Swipe + +Move from `(x1, y1)` to `(x2, y2)` with a duration (milliseconds) on the second `pointerMove`. + +```bash +curl -s -X POST http://localhost:8100/session/SESSION_ID/actions \ + -H 'Content-Type: application/json' \ + -d '{ + "actions": [{ + "type": "pointer", + "id": "finger1", + "parameters": {"pointerType": "touch"}, + "actions": [ + {"type": "pointerMove", "duration": 0, "x": X1, "y": Y1}, + {"type": "pointerDown"}, + {"type": "pointerMove", "duration": 500, "x": X2, "y": Y2}, + {"type": "pointerUp"} + ] + }] + }' +``` + +**Swipe direction guide** (given screen size `W x H`): +- **Up** (scroll down): from `(W/2, H/2 + H/6)` to `(W/2, H/2 - H/6)` +- **Down** (scroll up): from `(W/2, H/2 - H/6)` to `(W/2, H/2 + H/6)` +- **Left**: from `(W/2 + W/4, H/2)` to `(W/2 - W/4, H/2)` +- **Right**: from `(W/2 - W/4, H/2)` to `(W/2 + W/4, H/2)` +- **Back** (swipe from left edge): from `(5, H/2)` to `(W*2/3, H/2)` + +### Back Navigation + +To go back to the previous screen: + +- **Primary**: find a Button inside NavigationBar -- its label is typically the previous screen's title. Tap it. +- **Fallback**: edge swipe from `(5, H/2)` to `(W*2/3, H/2)` (see Swipe direction guide above) + +The button approach is more reliable because edge swipes can be finicky depending on gesture recognizers. + +### Type Text + +```bash +curl -s -X POST http://localhost:8100/session/SESSION_ID/wda/keys \ + -H 'Content-Type: application/json' \ + -d '{"value": ["h","e","l","l","o"]}' +``` + +The `value` array contains individual characters. An element must be focused first (tap a text field before typing). + +### Clear Text Field + +Select all text and delete it: + +```bash +# Select all (Ctrl+A) then delete +curl -s -X POST http://localhost:8100/session/SESSION_ID/wda/keys \ + -H 'Content-Type: application/json' \ + -d '{"value": ["\u0001"]}' +curl -s -X POST http://localhost:8100/session/SESSION_ID/wda/keys \ + -H 'Content-Type: application/json' \ + -d '{"value": ["\u007F"]}' +``` + +Alternatively, if you have an element ID: + +```bash +curl -s -X POST http://localhost:8100/session/SESSION_ID/element/ELEMENT_ID/clear +``` + +## Waiting for UI Stability + +After performing an action (tap, swipe, type), the UI may be animating or loading. Instead of using a fixed sleep, poll for the expected state: + +1. Fetch the accessibility tree +2. Check if the expected element or screen is present +3. If not found, sleep 0.5s and retry +4. After 10 failed attempts (5 seconds total), declare the element not found + +This approach is more reliable than fixed delays because it adapts to variable animation durations and network load times. + +## Scroll View Navigation + +To find an element in a long scrollable list: + +1. Fetch the tree and search for the target element +2. If found, tap it -- done +3. If not found, swipe up from the right edge to scroll down (use x = `screen_width - 30` to avoid tapping interactive elements) +4. Re-fetch the tree and search again +5. **Detect end of list**: if the tree content is identical after a scroll, you've reached the bottom +6. Stop and report element not found if the bottom is reached without finding the target + +Use the same pattern for horizontal scroll views, adjusting swipe direction accordingly. + +## Screenshots + +Use `simctl` for screenshots -- more reliable than WDA's base64 approach: + +```bash +xcrun simctl io screenshot /tmp/screenshot.png +``` + +To get the booted simulator's UDID: + +```bash +xcrun simctl list devices booted -j | jq -r '.devices | to_entries[].value[] | select(.state == "Booted")' +``` + +## Tips + +- **Tree coordinates, not screenshot pixels** -- screenshots may be at a different resolution than the tree's point-based coordinates. +- **Vertical swipes**: use the right edge x-coordinate (`screen_width - 30`) to avoid accidentally tapping interactive elements in the center. Use center only when needed. +- **Slow swipes on tappable items**: swipe gestures on tappable items may register as a tap. Use `duration: 1000` (1 second) for more reliable swipes. +- **WDA startup time**: ~60s the first time. Subsequent starts are faster with cached DerivedData. +- **Reconnecting**: if WDA disconnects, run `wda-start.rb` again -- it will reconnect. +- **Tab bar**: look for elements with type containing `TabBar` in the tree. Its children are the individual tabs. + +## Common Failures and Recovery + +### WDA Session Expiry + +WDA sessions can expire after inactivity. If action requests return HTTP 4xx errors, re-create the session: + +```bash +curl -s -X POST http://localhost:8100/session \ + -H 'Content-Type: application/json' \ + -d '{"capabilities":{"alwaysMatch":{}}}' | jq . +``` + +### Stale Element Coordinates + +After animations or screen transitions, previously fetched coordinates may be wrong. Always re-fetch the tree and recompute coordinates before tapping after any navigation action. + +### System Alert Interception + +System alerts (location permissions, notification permissions, tracking prompts) can block interactions with the app. Before retrying a failed tap: + +1. Fetch the tree and look for elements of type `Alert` or `Sheet` +2. If found, look for a dismiss button ("Allow", "Don't Allow", "OK", "Cancel") and tap it +3. Then retry the original action + +### App Crash Detection + +If actions consistently fail or the tree looks unexpected, the app may have crashed. Check and re-launch: + +```bash +# Check if the app process is running +xcrun simctl list devices booted + +# Re-launch the app +xcrun simctl launch +``` + +After re-launching, create a new WDA session before continuing. diff --git a/.claude/skills/ios-sim-navigation/scripts/wda-start.rb b/.claude/skills/ios-sim-navigation/scripts/wda-start.rb new file mode 100755 index 000000000000..68c5dd8e3ee0 --- /dev/null +++ b/.claude/skills/ios-sim-navigation/scripts/wda-start.rb @@ -0,0 +1,132 @@ +#!/usr/bin/env ruby +# Start WebDriverAgent server on a simulator. +# +# Runs `xcodebuild test-without-building` in the background and waits +# for WDA to respond on the specified port. +# +# Usage: ./wda-start.rb [--udid ] [--port ] +# +# Options: +# --udid Target a specific simulator (default: first booted) +# --port WDA port (default: 8100) +# +# Exit codes: +# 0 WDA started successfully +# 1 WDA failed to start +# 2 Configuration error + +require "optparse" +require "net/http" +require "json" + +DEFAULT_PORT = 8100 + +def get_booted_udid + output = `xcrun simctl list devices booted -j 2>/dev/null` + return nil unless $?.success? + + data = JSON.parse(output) + data.fetch("devices", {}).each_value do |devices| + devices.each do |d| + return d["udid"] if d["state"] == "Booted" + end + end + nil +end + +def resolve_udid(udid) + return udid if udid + + detected = get_booted_udid + unless detected + $stderr.puts "Error: No booted simulator found. Specify --udid or boot a simulator." + exit 2 + end + detected +end + +def wda_running?(port) + uri = URI("http://localhost:#{port}/status") + response = Net::HTTP.get_response(uri) + response.code.to_i == 200 +rescue Errno::ECONNREFUSED, Errno::ECONNRESET + false +end + +udid = nil +port = DEFAULT_PORT + +parser = OptionParser.new do |opts| + opts.banner = "Usage: wda-start.rb [options]" + opts.on("--udid UDID", "Target a specific simulator") { |v| udid = v } + opts.on("--port PORT", Integer, "WDA port (default: 8100)") { |v| port = v } +end +parser.parse! + +udid = resolve_udid(udid) + +# Check if WDA is already running +if wda_running?(port) + puts "WDA already running on port #{port}" + exit 0 +end + +# Find the WDA project +wda_project = File.join(Dir.pwd, ".build", "WebDriverAgent", "WebDriverAgent.xcodeproj") +unless File.exist?(wda_project) + $stderr.puts "Error: WebDriverAgent project not found at #{wda_project}" + $stderr.puts "Clone it: git clone https://github.com/appium/WebDriverAgent.git .build/WebDriverAgent" + exit 2 +end + +# Start xcodebuild test-without-building in the background +cmd = [ + "xcodebuild", "test-without-building", + "-project", wda_project, + "-scheme", "WebDriverAgentRunner", + "-destination", "id=#{udid}", + "USE_PORT=#{port}", + "CODE_SIGNING_ALLOWED=NO" +] + +log_path = "/tmp/wda-#{port}.log" +pid_path = "/tmp/wda-#{port}.pid" + +puts "Starting WDA on port #{port} for simulator #{udid}..." +puts "Log: #{log_path}" + +pid = spawn(*cmd, out: log_path, err: log_path) +File.write(pid_path, pid.to_s) +Process.detach(pid) + +# Wait for WDA to become ready +max_wait = 60 +interval = 2 +elapsed = 0 + +while elapsed < max_wait + sleep interval + elapsed += interval + + begin + uri = URI("http://localhost:#{port}/status") + response = Net::HTTP.get_response(uri) + if response.code.to_i == 200 + puts "WDA ready on port #{port} (took #{elapsed}s)" + puts "PID: #{pid} (saved to #{pid_path})" + exit 0 + end + rescue Errno::ECONNREFUSED, Errno::ECONNRESET + # Not ready yet + end +end + +$stderr.puts "Error: WDA did not start within #{max_wait}s" +$stderr.puts "Check log: #{log_path}" +# Try to kill the process +begin + Process.kill("TERM", pid) +rescue Errno::ESRCH + # Process already gone +end +exit 1 diff --git a/.claude/skills/ios-sim-navigation/scripts/wda-stop.rb b/.claude/skills/ios-sim-navigation/scripts/wda-stop.rb new file mode 100755 index 000000000000..dbb56e8e5228 --- /dev/null +++ b/.claude/skills/ios-sim-navigation/scripts/wda-stop.rb @@ -0,0 +1,59 @@ +#!/usr/bin/env ruby +# Stop a running WebDriverAgent server. +# +# Usage: ./wda-stop.rb [--port ] +# +# Options: +# --port WDA port (default: 8100) +# +# Exit codes: +# 0 WDA stopped (or was not running) +# 1 Failed to stop WDA + +require "optparse" + +port = 8100 + +parser = OptionParser.new do |opts| + opts.banner = "Usage: wda-stop.rb [options]" + opts.on("--port PORT", Integer, "WDA port (default: 8100)") { |v| port = v } +end +parser.parse! + +pid_path = "/tmp/wda-#{port}.pid" +stopped = false + +# Try the PID file first +if File.exist?(pid_path) + pid = File.read(pid_path).strip.to_i + if pid > 0 + begin + Process.kill("TERM", pid) + puts "Sent TERM to WDA process #{pid}" + stopped = true + rescue Errno::ESRCH + puts "WDA process #{pid} already gone" + stopped = true + end + end + File.delete(pid_path) +end + +# Also kill any xcodebuild processes running WebDriverAgent +pids = `pgrep -f "xcodebuild.*WebDriverAgent" 2>/dev/null`.strip.split("\n").map(&:to_i) +pids.each do |p| + next if p <= 0 + begin + Process.kill("TERM", p) + puts "Killed xcodebuild process #{p}" + stopped = true + rescue Errno::ESRCH + # Already gone + end +end + +if stopped + puts "WDA stopped" +else + puts "WDA was not running" +end diff --git a/.gitignore b/.gitignore index 6c8bbe359534..189e7e0fa842 100644 --- a/.gitignore +++ b/.gitignore @@ -134,3 +134,5 @@ Pods/ WordPress/Frameworks/*.xcframework WordPress/Frameworks/*.tar.gz WordPress/Frameworks/react-native-bundle-source-map + +Tests/AgentTests/results diff --git a/Tests/AgentTests/ui-tests/create-blank-page.md b/Tests/AgentTests/ui-tests/create-blank-page.md new file mode 100644 index 000000000000..64d64190a8a5 --- /dev/null +++ b/Tests/AgentTests/ui-tests/create-blank-page.md @@ -0,0 +1,26 @@ +# Create and Publish a Blank Page + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Tap the FAB (floating action button) or "+" button to create new content. +3. In the bottom sheet, select "Page". +4. If a page template picker appears, select "Blank page" to create a blank page. +5. Enter "Blank page title" as the page title. +6. Tap the "Publish" button in the top-right corner. +7. If a pre-publish confirmation appears, confirm by tapping "Publish" again. +8. Dismiss the post-publish confirmation screen by tapping "Done". + +## Verification (REST API) +- Use the WordPress REST API to search for a page titled "Blank page title" with status "publish". +- Verify the page exists. + +## Cleanup (REST API) +- Use the WordPress REST API to trash the page created during this test. + +## Expected Outcome +- The page "Blank page title" is published successfully and a confirmation screen is shown. +- The REST API confirms a published page with the title "Blank page title" exists. +- The page is trashed via the REST API. diff --git a/Tests/AgentTests/ui-tests/create-scheduled-post.md b/Tests/AgentTests/ui-tests/create-scheduled-post.md new file mode 100644 index 000000000000..0475c53519ac --- /dev/null +++ b/Tests/AgentTests/ui-tests/create-scheduled-post.md @@ -0,0 +1,28 @@ +# Create a Scheduled Post + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Tap the FAB (floating action button) or "+" button to create a new post. +3. If a bottom sheet appears, select "Post". +4. Enter "Scheduled post title" as the post title. +5. Tap the "Publish" button in the top-right corner. +6. In the pre-publish sheet, tap the publish date to change it. +7. Set the date to a future date (e.g., one day from now). +8. Confirm the date selection. +9. Tap "Schedule" to schedule the post. +10. Dismiss the confirmation screen by tapping "Done". + +## Verification (REST API) +- Use the WordPress REST API to search for a post titled "Scheduled post title" with status "future". +- Verify the post exists and has a future publish date. + +## Cleanup (REST API) +- Use the WordPress REST API to trash the post created during this test. + +## Expected Outcome +- The post is scheduled for a future date and a confirmation screen is shown. +- The REST API confirms a post with the title "Scheduled post title" exists with status "future". +- The post is trashed via the REST API. diff --git a/Tests/AgentTests/ui-tests/featured-image-add-remove.md b/Tests/AgentTests/ui-tests/featured-image-add-remove.md new file mode 100644 index 000000000000..ba4a4d80ec29 --- /dev/null +++ b/Tests/AgentTests/ui-tests/featured-image-add-remove.md @@ -0,0 +1,26 @@ +# Add and Remove Featured Image + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Tap the FAB (floating action button) or "+" button to create a new post. +3. If a bottom sheet appears, select "Post". +4. Enter "Featured image post" as the post title. +5. Tap below the title to add a paragraph block. +6. Type "Testing featured image functionality." as the paragraph content. +7. Open the post settings (tap the gear/settings icon or "Post Settings"). +8. Tap "Set Featured Image" or the featured image placeholder. +9. Choose a photo from the device's photo library. +10. Verify the featured image thumbnail is now displayed in the post settings. +11. Tap the featured image to open its options, then tap "Remove" or "Remove Featured Image". +12. Verify the featured image is removed and the placeholder is shown again. +13. Tap "Set Featured Image" again and choose a photo from the photo library. +14. Verify the featured image thumbnail is displayed again. +15. Close or save the post settings. + +## Expected Outcome +- After setting a featured image, a thumbnail is visible in the post settings. +- After removing the featured image, the thumbnail is no longer shown. +- The featured image can be set again after removal. diff --git a/Tests/AgentTests/ui-tests/gallery-block.md b/Tests/AgentTests/ui-tests/gallery-block.md new file mode 100644 index 000000000000..b3ffc7a23789 --- /dev/null +++ b/Tests/AgentTests/ui-tests/gallery-block.md @@ -0,0 +1,23 @@ +# Add a Gallery Block + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Tap the FAB (floating action button) or "+" button to create a new post. +3. If a bottom sheet appears, select "Post". +4. Enter "Gallery block post" as the post title. +5. Tap below the title to add a paragraph block. +6. Type "Testing gallery block." as the paragraph content. +7. Tap the "+" block inserter button to add a new block. +8. Search for or select the "Gallery" block. +9. In the gallery block, tap "Add Media" or the media picker option. +10. Select multiple photos from the device's photo library (at least 3 images). +11. Confirm the selection. +12. Wait for the images to finish uploading. +13. Verify the gallery block displays the selected images in the editor. + +## Expected Outcome +- The gallery block is added to the post and displays the selected images. +- The editor shows the paragraph block and gallery block together. diff --git a/Tests/AgentTests/ui-tests/image-optimization-enabled-by-default.md b/Tests/AgentTests/ui-tests/image-optimization-enabled-by-default.md new file mode 100644 index 000000000000..34d799441f08 --- /dev/null +++ b/Tests/AgentTests/ui-tests/image-optimization-enabled-by-default.md @@ -0,0 +1,13 @@ +# Verify Image Optimization is Enabled by Default + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Tap the "Me" tab. +3. Tap "App Settings". + +## Expected Outcome +- The App Settings screen is displayed. +- The "Optimize Images" switch is enabled (on) by default. diff --git a/Tests/AgentTests/ui-tests/media-blocks.md b/Tests/AgentTests/ui-tests/media-blocks.md new file mode 100644 index 000000000000..e8eaca38e70b --- /dev/null +++ b/Tests/AgentTests/ui-tests/media-blocks.md @@ -0,0 +1,30 @@ +# Add Media Blocks (Image, Video, Audio) + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Tap the FAB (floating action button) or "+" button to create a new post. +3. If a bottom sheet appears, select "Post". +4. Add an image block by tapping the "+" block inserter, selecting "Image", and choosing a photo from the device's photo library. +5. Wait for the image to finish uploading. +6. Verify the image block is displayed in the editor. +7. Tap the "+" block inserter to add a new block. +8. Search for or select the "Video" block. +9. In the video block, choose "Insert from URL". +10. Enter the URL: `http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4` +11. Confirm the URL entry. +12. Verify the video block is displayed in the editor. +13. Tap the "+" block inserter to add a new block. +14. Search for or select the "Audio" block. +15. In the audio block, choose "Insert from URL". +16. Enter the URL: `http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ElephantsDream.mp4` +17. Confirm the URL entry. +18. Verify the audio block is displayed in the editor. + +## Expected Outcome +- The editor contains three media blocks: image, video, and audio. +- The image block shows the uploaded photo. +- The video block shows the embedded video from the provided URL. +- The audio block shows the embedded audio from the provided URL. diff --git a/Tests/AgentTests/ui-tests/post-with-category-and-tag.md b/Tests/AgentTests/ui-tests/post-with-category-and-tag.md new file mode 100644 index 000000000000..925824742d62 --- /dev/null +++ b/Tests/AgentTests/ui-tests/post-with-category-and-tag.md @@ -0,0 +1,34 @@ +# Publish a Post with Category and Tag + +## Prerequisites +- Logged in to the app with the test account. +- The site has a category named "Wedding" (or another existing category). + +## Steps +1. Navigate to the "My Site" tab. +2. Tap the FAB (floating action button) or "+" button to create a new post. +3. If a bottom sheet appears, select "Post". +4. Enter "Category tag post" as the post title. +5. Tap below the title to add a paragraph block. +6. Type "This is a test post with category and tag." as the paragraph content. +7. Open the post settings (tap the gear/settings icon or "Post Settings"). +8. Under "Categories", select an existing category (e.g., "Wedding"). +9. Under "Tags", add a new tag with a more than 8 characters long random name. +10. Save the post settings. +11. Tap the "Publish" button in the top-right corner. +12. If a pre-publish confirmation appears, confirm by tapping "Publish" again. +13. Verify the post-publish confirmation screen shows the correct post title. +14. Dismiss the confirmation screen by tapping "Done". + +## Verification (REST API) +- Use the WordPress REST API to search for a post titled "Category tag post" with status "publish". +- Verify the post exists and has the expected category (e.g., "Wedding") and tag assigned. + +## Cleanup (REST API) +- Use the WordPress REST API to trash the post created during this test. + +## Expected Outcome +- The post is published with the selected category and tag. +- The post-publish confirmation screen displays the correct post title. +- The REST API confirms a published post with the title "Category tag post" exists with the correct category and tag. +- The post is trashed via the REST API. diff --git a/Tests/AgentTests/ui-tests/text-post-publish.md b/Tests/AgentTests/ui-tests/text-post-publish.md new file mode 100644 index 000000000000..e25f1a4849d3 --- /dev/null +++ b/Tests/AgentTests/ui-tests/text-post-publish.md @@ -0,0 +1,27 @@ +# Publish a Text Post + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Tap the FAB (floating action button) or "+" button to create a new post. +3. If a bottom sheet appears, select "Post". +4. Enter "Rich post title" as the post title. +5. Tap below the title to add a paragraph block. +6. Type "Lorem ipsum dolor sit amet, consectetur adipiscing elit." as the paragraph content. +7. Tap the "Publish" button in the top-right corner. +8. If a pre-publish confirmation appears, confirm by tapping "Publish" again. +9. Dismiss the post-publish confirmation screen by tapping "Done". + +## Verification (REST API) +- Use the WordPress REST API to search for a post titled "Rich post title" with status "publish". +- Verify the post exists. + +## Cleanup (REST API) +- Use the WordPress REST API to trash the post created during this test. + +## Expected Outcome +- The post "Rich post title" is published successfully and a confirmation screen is shown. +- The REST API confirms a published post with the title "Rich post title" exists. +- The post is trashed via the REST API. diff --git a/Tests/AgentTests/ui-tests/undo-redo-in-editor.md b/Tests/AgentTests/ui-tests/undo-redo-in-editor.md new file mode 100644 index 000000000000..4f43bba8daa0 --- /dev/null +++ b/Tests/AgentTests/ui-tests/undo-redo-in-editor.md @@ -0,0 +1,23 @@ +# Undo and Redo in the Block Editor + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Tap the FAB (floating action button) or "+" button to create a new post. +3. If a bottom sheet appears, select "Post". +4. Verify the Undo button is disabled. +5. Verify the Redo button is disabled. +6. Enter "Rich post title" as the post title. +7. Tap below the title to add a paragraph block. +8. Type "Lorem ipsum dolor sit amet" as the paragraph content. +9. Tap the Undo button twice to undo the paragraph and title. +10. Verify the editor content is empty (no blocks visible). +11. Tap the Redo button twice to restore the paragraph and title. +12. Verify the paragraph content "Lorem ipsum dolor sit amet" is visible again. + +## Expected Outcome +- After undoing, the editor content is empty. +- After redoing, the title and paragraph content are restored. +- Undo and Redo buttons correctly reflect available actions throughout the flow. diff --git a/Tests/AgentTests/ui-tests/users-screen-loads.md b/Tests/AgentTests/ui-tests/users-screen-loads.md new file mode 100644 index 000000000000..bc03e2e6f81a --- /dev/null +++ b/Tests/AgentTests/ui-tests/users-screen-loads.md @@ -0,0 +1,13 @@ +# Verify Users Screen Loads + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Scroll down and tap "More" or look for the "People" / "Users" menu item. +3. Tap "People" or "Users" to open the users screen. + +## Expected Outcome +- The Users / People screen is displayed and loads successfully. +- A list of site users is visible on the screen. diff --git a/Tests/AgentTests/ui-tests/view-site-from-my-site.md b/Tests/AgentTests/ui-tests/view-site-from-my-site.md new file mode 100644 index 000000000000..7641b2d7ad22 --- /dev/null +++ b/Tests/AgentTests/ui-tests/view-site-from-my-site.md @@ -0,0 +1,14 @@ +# View Site from My Site + +## Prerequisites +- Logged in to the app with the test account. + +## Steps +1. Navigate to the "My Site" tab. +2. Note the site title displayed on the screen. +3. Tap the site URL / site address shown below the site title. +4. A web view should open displaying the site. + +## Expected Outcome +- A web view opens after tapping the site address. +- The web view displays the site content, including the site title.