Smoke Test: Claude - 22256990263 #17488
Description
Automated smoke test run for the Claude engine. Run: §22256990263 · Timestamp: 2026-02-21
PR used for review tests: #17484 ("fix(parser): include per-field locations for safe-output schema failures")
Core Test Results
| # | Test | Status |
|---|---|---|
| 1 | GitHub MCP: Last 2 merged PRs | ✅ PRs #17479 & #17478 |
| 2 | SafeInputs GH CLI: Query 2 PRs | ✅ PRs #17484 & #17476 |
| 3 | Serena MCP: Activate + find symbols | ✅ 17 symbols found in schema_compiler.go |
| 4 | Make Build | ✅ Built gh-aw (version 612385d) |
| 5 | Playwright: github.com title check | ✅ Title contains "GitHub" |
| 6 | Tavily: "GitHub Agentic Workflows" search | ✅ 3 results returned |
| 7 | File Writing: smoke-test file creation | ✅ /tmp/gh-aw/agent/smoke-test-claude-22256990263.txt |
| 8 | Bash Tool: verify file contents | ✅ Content verified |
| 9 | Discussion: latest discussion + comment | ✅ Discussion #17485, comic-book comment added |
| 10 | Agentic Workflows MCP: smoke-claude status | ✅ Active, compiled, engine=claude |
PR Review Safe Output Test Results
| # | Test | Status |
|---|---|---|
| 11 | Update PR body | ✅ Appended test message to PR #17484 |
| 12 | PR Review Comments (2) | ✅ Inline comments on schema_compiler.go:341 and test file:391 |
| 13 | Submit PR Review | ✅ COMMENT review submitted on PR #17484 |
| 14 | Resolve Review Thread | |
| 15 | Add Reviewer | ✅ Added copilot as reviewer to PR #17484 |
| 16 | Push to PR Branch | main, not PR branch codex/issue-17480-safe-outputs-locations |
| 17 | Close PR (conditional) |
Overall Status: PARTIAL (10 passed, 3 skipped, 0 failed)
Test Details
GitHub MCP (#1): PRs #17479 "Constrain agentic-workflows agent to single-file output" (merged 2026-02-21T12:39Z) and #17478 "Update wasm golden fixtures for Copilot CLI 0.0.414" (merged 2026-02-21T12:41Z).
Serena (#3): Found symbols: validateWithSchemaAndLocation, formatSchemaFailureDetail, GetMainWorkflowSchema, compileSchema, getCompiledMainWorkflowSchema, getCompiledMcpConfigSchema, validateWithSchema, GetSafeOutputTypeKeys, and 9 variable symbols.
Agentic Workflows (#10): smoke-claude — compiled: Yes, status: active, triggers: pull_request (label: smoke), reaction (heart), schedule (every 12h), status-comment.
Review Comments (#12): Comment 1 on pkg/parser/schema_compiler.go line 341 re: formatSchemaFailureDetail doc comment suggestion. Comment 2 on pkg/parser/safe_outputs_error_location_test.go line 391 re: asserting aggregation message format.
References:
💥 [THE END] — Illustrated by Smoke Claude
- expires on Feb 21, 2026, 2:52 PM UTC