Conversation
…eturning last step content The extractPreAgentStepErrors function was returning the full content of the last step (by step number), which is typically "Complete job" teardown content. This caused the audit errors section to show unrelated cleanup logs instead of the actual failure reason. The fix scans all step log files for ##[error] annotations (the precise GitHub Actions failure markers) and returns those when found, falling back to the last step content only when no ##[error] annotations exist. Fixes #N/A (issue: audit failed run error extraction shows wrong log content) Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Fixes audit error extraction for failed runs where the agent never started by correctly pulling ##[error] annotations from the relevant step logs (instead of returning the highest-numbered “Complete job” teardown log content).
Changes:
- Reworked
extractPreAgentStepErrorsto scan all step log files for##[error]lines and prefer those over last-step fallback content. - Preserved the existing “last-step content” fallback when no
##[error]annotations exist anywhere. - Added targeted tests covering annotation priority, multi-step annotations, and fallback behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
pkg/cli/audit_report.go |
Updates step-log error extraction to prefer ##[error] annotations across all steps, with fallback to last step content. |
pkg/cli/audit_report_test.go |
Adds regression tests ensuring ##[error] annotations are selected over teardown logs and that fallback behavior remains intact. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Scan this step for ##[error] annotations | ||
| content, err := os.ReadFile(stepFilePath) | ||
| if err != nil { | ||
| auditReportLog.Printf("Failed to read step log %s: %v", stepFilePath, err) | ||
| continue | ||
| } | ||
|
|
||
| var errorLines []string | ||
| for line := range strings.SplitSeq(string(content), "\n") { | ||
| if strings.Contains(line, "##[error]") { | ||
| stripped := stripGHALogTimestamps(line) | ||
| if stripped != "" { | ||
| errorLines = append(errorLines, stripped) | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
os.ReadFile loads the entire step log into memory (and string(content) copies it again) even though we only need to detect/collect ##[error] lines. Step logs can be large, so this can significantly increase memory/time when scanning many steps. Consider streaming the file (e.g., bufio.Scanner / bufio.Reader) and collecting matching lines without reading the whole file; optionally stop once the accumulated message reaches maxMessageLen to cap work per step.
| // Scan this step for ##[error] annotations | |
| content, err := os.ReadFile(stepFilePath) | |
| if err != nil { | |
| auditReportLog.Printf("Failed to read step log %s: %v", stepFilePath, err) | |
| continue | |
| } | |
| var errorLines []string | |
| for line := range strings.SplitSeq(string(content), "\n") { | |
| if strings.Contains(line, "##[error]") { | |
| stripped := stripGHALogTimestamps(line) | |
| if stripped != "" { | |
| errorLines = append(errorLines, stripped) | |
| } | |
| } | |
| } | |
| // Scan this step for ##[error] annotations without loading the entire file into memory | |
| f, err := os.Open(stepFilePath) | |
| if err != nil { | |
| auditReportLog.Printf("Failed to open step log %s: %v", stepFilePath, err) | |
| continue | |
| } | |
| var errorLines []string | |
| scanner := bufio.NewScanner(f) | |
| accumulatedLen := 0 | |
| for scanner.Scan() { | |
| line := scanner.Text() | |
| if strings.Contains(line, "##[error]") { | |
| stripped := stripGHALogTimestamps(line) | |
| if stripped != "" { | |
| errorLines = append(errorLines, stripped) | |
| // Track accumulated length (including newline separators) | |
| accumulatedLen += len(stripped) + 1 | |
| if accumulatedLen >= maxMessageLen { | |
| break | |
| } | |
| } | |
| } | |
| } | |
| if err := scanner.Err(); err != nil { | |
| auditReportLog.Printf("Failed to scan step log %s: %v", stepFilePath, err) | |
| } | |
| _ = f.Close() |
| auditReportLog.Printf("Extracted ##[error] annotations from %s (step %d)", stepKey, num) | ||
| errorAnnotations = append(errorAnnotations, ErrorInfo{ | ||
| Type: "step_failure", | ||
| File: stepKey, | ||
| Message: message, |
There was a problem hiding this comment.
errorAnnotations are appended in the filesystem iteration order (os.ReadDir sorts lexicographically), which can produce non-numeric step ordering (e.g., 10_*.txt before 2_*.txt) and a confusing error list when multiple steps contribute annotations. Consider collecting (job, stepNum, stepKey, message) in an intermediate slice and sorting by job + stepNum before returning the final []ErrorInfo.
The
auditcommand's error extraction was returning the full log of the highest-numbered step (always "Complete job" teardown) instead of the actual##[error]annotations from the failing step, sending developers in the wrong direction.Changes
pkg/cli/audit_report.go— RewroteextractPreAgentStepErrorsto scan all step log files in a single pass, collecting##[error]-annotated lines from each step. Returns oneErrorInfoper step with annotations; falls back to last-step content only when no##[error]lines exist anywhere.pkg/cli/audit_report_test.go— Added three new test cases:##[error]annotations take priority over a higher-numbered "Complete job" step##[error]annotations existBefore / After
Before: step 3 has the real error, step 15 is "Complete job" → audit returns step 15 teardown content.
After: audit returns the
##[error]lines from step 3, stripped of GHA timestamps:Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
https://api.github.com/graphql/usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw main -lang=go1.25 git rev-�� --show-toplevel -dwarf=false /usr/bin/git go1.25.0 -c=4 -nolocalimports git(http block)https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1/usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha runs/20260224-010444-47910/test-1440987474/.github/workflows GO111MODULE 971710/b368/vet.cfg l GOMOD GOMODCACHE go env -json l /tmp/go-build350971710/b001/gh-aw.test GOINSECURE GOMOD GOMODCACHE /tmp/go-build350971710/b001/gh-aw.test(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v3/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha w.test GO111MODULE rtcfg.link GOINSECURE GOMOD GOMODCACHE g9VgsnZkcFLT7a977P/LjNWhbRiK6JQp5uNotQI/zl3NzS9vmet8lZmRgmyQ -c 0444-47910/test-4243298880/.github/workflows GOPROXY _.a GOSUMDB GOWORK 64/bin/go sh(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x-buildmode=exe GOINSECURE GOMOD GOMODCACHE go env runs/20260224-010444-47910/test-1549473206/.github/workflows GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha om/owner/repo.git go /usr/bin/git -json GO111MODULE 64/bin/go git conf�� user.email test@example.com /usr/bin/git -json GO111MODULE tions/node_modul--show-toplevel git(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v5/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go k/gh�� -json GO111MODULE x_amd64/compile GOINSECURE GOMOD erignore x_amd64/compile(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha ithub/workflows/brave.md GOPROXY /usr/bin/git GOSUMDB GOWORK 64/bin/go git rev-�� --show-toplevel go /usr/bin/git -json GO111MODULE 64/bin/go git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha GOMODCACHE go /usr/bin/git -json GO111MODULE ache/go/1.25.0/x--show-toplevel git rev-�� --show-toplevel go /usr/bin/git */*.ts' '**/*.jsgit GO111MODULE 64/bin/go /usr/bin/git(http block)https://api.github.com/repos/actions/github-script/git/ref/tags/v8/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env on' --ignore-patGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha go1.25.0 -c=4 -nolocalimports -importcfg /tmp/go-build350971710/b403/importcfg -pack /tmp/go-build350971710/b403/_testmain.go ode_�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/actions/setup-go/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha brave.md GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE cfg GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/actions/setup-node/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha vaScript2772946621/001/test-fron-test.timeout=10m0s GO111MODULE /opt/hostedtoolcache/go/1.25.0/x-test.short=true GOINSECURE GOMOD GOMODCACHE go env runs/20260224-010444-47910/test-1549473206/.github/workflows GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts/usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE 64/bin/go GOINSECURE 469f5569 GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts/usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go stlo�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts/usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go stlo�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts/usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts/usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts/usr/bin/gh gh run download 4 --dir test-logs/run-4 469f5569:go.mod 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts/usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/actions/workflows/usr/bin/gh gh workflow list --json name,state,path GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE go env th .prettierignoGOSUMDB GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env 709732020/.github/workflows GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999/usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env heck '**/*.cjs' '**/*.ts' '**/*.json' --ignore-pGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/nonexistent/repo/actions/runs/12345/usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go stlo�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/owner/repo/actions/workflows/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE go env json' --ignore-pGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo odules/npm/node_GOMODCACHE GOINSECURE GOMOD GOMODCACHE go env json' --ignore-pGOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/owner/repo/contents/file.md/tmp/go-build350971710/b381/cli.test /tmp/go-build350971710/b381/cli.test -test.testlogfile=/tmp/go-build350971710/b381/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/test-owner/test-repo/actions/secrets/usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE sh GOINSECURE GOMOD GOMODCACHE go(http block)If you need me to access, download, or install something from one of these locations, you can either:
Original prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.