[prompt-analysis] Copilot PR Prompt Analysis - 2026-02-06

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis

Analysis Period: Last 30 days
Total PRs: 1,000 | Merged: 687 (69.0%) | Closed: 308 (31.0%) | Open: 5 (0.5%)

---

Prompt Categories and Success Rates

Category	Total	Merged	Success Rate
Update	732	519	70.9%
Refactor	298	211	70.8%
Testing	764	536	70.2%
Feature Addition	860	601	69.9%
Bug Fix	976	675	69.2%
Documentation	858	586	68.3%

Key Insights

🎯 Critical Success Factor: Avoid WIP/Draft Tags

• WIP/Draft PRs are 11x more likely to be closed (32.8% of closed vs 2.9% of merged)
• This is the single strongest predictor of PR closure

📝 Action Verb Titles Matter

• Merged PRs use action verbs 76% more often than closed PRs
• 64.8% of merged PRs start with action verbs (fix, add, update, remove, refactor)
• Only 36.7% of closed PRs use action verbs
• Impact: Clear, actionable titles correlate with higher merge rates

📊 Category Success is Remarkably Consistent

• All categories show 68-71% success rates
• Update and refactor categories slightly outperform (70.8-70.9%)
• Insight: Prompt category matters less than execution quality

📏 Body Length is Not a Factor

• Merged PRs: 402 words average
• Closed PRs: 416 words average
• Conclusion: Verbosity doesn't help—clarity and specificity matter more

---

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average body length: 402 words
• Top keywords: copilot, agent, github, coding, workflow, test
• Action verbs: fix, add, update, remove, implement
• File references: 48.6% include specific file paths

Example successful prompts:

View Successful PR Examples

1. PR Fix dispatch_workflow to use PR branch ref and resolve default branch correctly #14062: Fix dispatch_workflow to use PR branch ref and resolve default branch correctly

   • Prompt: dispatch_workflow was using GITHUB_REF in all contexts, which points to refs/pull/{PR_NUMBER}/merge when triggered from PRs. The GitHub Actions API rejects merge refs for workflow dispatch...
   • Why successful: Specific problem statement, technical detail, clear fix scope

2. PR Add working directory logging and Docker workdir for agentic-workflows MCP server #14053: Add working directory logging and Docker workdir for agentic-workflows MCP server

   • Prompt: The agentic-workflows MCP server needed context logging and proper working directory setup for path resolution within containers...
   • Why successful: Clear feature addition, specific technical context

3. PR Remove GH_TOKEN from agentic-workflows MCP server config #14044: Remove GH_TOKEN from agentic-workflows MCP server config

   • Prompt: The "Daily Firewall Logs Collector and Reporter" workflow was failing with "undefined environment variable referenced: GH_TOKEN" because the agentic-workflows MCP server configuration referenced...
   • Why successful: References specific failure, clear root cause

4. PR Add plugin installation support via frontmatter with dual-format configuration and cascading token resolution #14041: Add plugin installation support via frontmatter with dual-format configuration

   • Prompt: Adds plugins frontmatter field supporting both array and object formats for flexible plugin configuration. Compiler generates installation steps using engine-specific CLI commands...
   • Why successful: Comprehensive feature description, multiple format support

5. PR Add telemetry.enterprise.githubcopilot.com to copilot engine default allowlist #14007: Add telemetry.enterprise.githubcopilot.com to copilot engine default allowlist

   • Prompt: Adds telemetry.enterprise.githubcopilot.com to CopilotDefaultDomains so copilot engine workflows can reach the enterprise telemetry endpoint...
   • Why successful: Specific domain, clear use case

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average body length: 416 words
• Top keywords: copilot, agent, github, workflow, coding
• WIP/Draft markers: 32.8% of closed PRs
• Missing action verbs: 63.3% lack clear action verbs

Example unsuccessful prompts:

View Closed PR Examples

1. PR [WIP] Debug workflow failure in agent performance analyzer #14057: [WIP] Debug workflow failure in agent performance analyzer

   • Issue: WIP tag, investigation-focused
   • Pattern: Exploratory work often gets closed in favor of focused fixes

2. PR [WIP] Debug workflow failure in Workflow Normalizer #14056: [WIP] Debug workflow failure in Workflow Normalizer

   • Issue: WIP tag, debugging rather than fixing
   • Pattern: Debug PRs typically superseded by proper fixes

3. PR [WIP] Update footer with community feedback message #14042: [WIP] Update footer with community feedback message

   • Issue: Generic WIP placeholder text
   • Pattern: Vague initial prompt, unclear scope

4. PR Remove nested groups in displayDirectories to eliminate redundant directory titles #13997: Remove nested groups in displayDirectories to eliminate redundant directory titles

   • Issue: No WIP tag but may have implementation issues
   • Pattern: Technical change without clear justification

5. PR Integrate gh-aw-mcpg v0.0.113 #13993: Integrate gh-aw-mcpg v0.0.113

   • Issue: Version update without context
   • Pattern: Mechanical updates may need more context

---

Recommendations

Based on the 1,000 PR analysis, here are data-driven recommendations:

🎯 DO: Use Action Verb Titles

Impact: 76% higher correlation with merges

• Start titles with: Fix, Add, Update, Remove, Refactor, Implement
• ✅ GOOD: "Fix dispatch_workflow to use PR branch ref"
• ❌ AVOID: "Dispatch workflow improvements"

🚫 DO NOT: Use WIP/Draft Tags Unless Necessary

Impact: 11x more likely to be closed

• WIP/Draft tags signal incomplete work
• Only use when you genuinely need feedback on incomplete work
• If work is complete, remove the tag before final review

📝 DO: Reference Specific Files/Components

Impact: Slight correlation with success (48.6% vs 45.8%)

• Include file paths in backticks: pkg/workflow/compiler.go
• Name specific functions, structs, or components
• ✅ GOOD: "Update parseConfig() in pkg/parser/config.go"
• ❌ AVOID: "Update configuration parsing"

🎯 DO: Choose Update/Refactor When Appropriate

Impact: Highest success rates (70.8-70.9%)

• Updates and refactors show marginally higher success
• Be honest about the change type—don't force-fit into these categories
• These categories may succeed because they're well-scoped

📏 DO NOT: Write Unnecessarily Long Prompts

Impact: No correlation found (402 vs 416 words)

• Longer prompts don't improve outcomes
• Focus on clarity and specificity, not length
• Include only essential context

---

Historical Trends

This is the first analysis run. Historical trends will appear after multiple daily runs.

Date	PRs	Success Rate	Top Category
2026-02-06	1,000	69.0%	Update (70.9%)

Trend: Baseline established. Future analyses will track changes over time.

---

Methodology

Data Source: 1,000 most recent Copilot-generated PRs from github/gh-aw repository

Categorization:

• Merged: mergedAt is not null
• Closed: state is "CLOSED" and mergedAt is null
• Categories: Based on keyword matching in title and body

Analysis Techniques:

• Keyword frequency analysis
• Pattern matching for file references, error messages, WIP tags
• Success rate calculation per category
• Statistical comparison of merged vs closed characteristics

Limitations:

• Repository-specific patterns may not generalize
• Correlation does not imply causation
• Some PRs may be closed for external reasons (project direction changes, etc.)

---

References: §21750403407

AI generated by Copilot PR Prompt Pattern Analysis

• expires on Feb 13, 2026, 12:27 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-02-13T12:27:03.238Z.

Closed by Workflow