[prompt-clustering] Copilot Agent Prompt Clustering Analysis - February 11, 2026

[github-actions[bot]]

🔬 Copilot Agent Prompt Clustering Analysis - February 11, 2026

Daily NLP-based clustering analysis of copilot agent task prompts using TF-IDF vectorization and K-means clustering.

Summary

Analysis Period: Last 30 days (1,218 tasks analyzed)
Clusters Identified: 7
Overall Success Rate: 66.0% (804/1218 PRs merged)
Data Source: Copilot-created PRs in github/gh-aw repository

Cluster Overview

Cluster	Size	Success Rate	Top Keywords
In The	443 (36.4%)	70.9%	agentic, update, project, safe, workflow
On The	296 (24.3%)	58.4%	workflow, issue, gh, aw, gh aw
Test + Pkg	169 (13.9%)	66.3%	test, pkg, code, error, lines
Mcp Server	120 (9.9%)	61.7%	mcp, server, mcp server, gateway, tool
Reference: Debug	82 (6.7%)	65.9%	reference, debug, fix, review, tests
On The	70 (5.7%)	67.1%	campaign, security, project, issue, fix
Job Id:	38 (3.1%)	78.9%	job, fix, workflow, url, logs

Visualizations

📊 Cluster Distribution

📈 Success Rates by Cluster

🗺️ Cluster Visualization (PCA)

📉 Elbow Analysis

View Full Analysis Report

General Insights

• Total Tasks Analyzed: 1218 copilot agent tasks from the last 30 days
• Merged PRs: 804 (66.0%)
• Closed (not merged): 399 (34.0%)
• Still Open: 15
• Clustering Method: TF-IDF vectorization (200 features) + K-means (k=7)
• Most Active Cluster: In The with 443 tasks
• Best Performing Cluster: Job Id: with 78.9% success rate

Detailed Cluster Analysis

Cluster 1: In The

Size: 443 tasks (36.4% of total)
Success Rate: 70.9% (314 merged)
Avg Code Changes: 18.7 files, +381/-283 lines

Top Keywords:

• agentic, update, project, safe, workflow, agent, create, workflows

Key Phrases:

• | ❌ |, ❌ | |, ## changes -, - [ ], ---- *this section

Characteristics:

• GitHub Actions workflow maintenance and fixes

Sample Tasks:

• ✅ #11053: Update parent issue template for agentic-workflow failures
• ✅ #11064: Add interactive engine selection and secret configuration to init command
• 🔴 #11065: [WIP] Fix network configuration for MCP server time

---

Cluster 2: On The

Size: 296 tasks (24.3% of total)
Success Rate: 58.4% (173 merged)
Avg Code Changes: 15.4 files, +267/-376 lines

Top Keywords:

• workflow, issue, gh, aw, gh aw, section, workflows, agent

Key Phrases:

• • [ ], ---- *this section, *this section details, section details on, details on the

Characteristics:

• GitHub Actions workflow maintenance and fixes

Sample Tasks:

• 🔴 #11054: Auto-assign @copilot to workflow sync issues when agent token available
• ✅ #11058: Fix ephemerals tests after blockquote prefix requirement in PR Fix expiration detection for quoted footers and legacy format #11036
• 🔴 #11059: Install Go toolchain in daily-cli-performance workflow

---

Cluster 3: Test + Pkg

Size: 169 tasks (13.9% of total)
Success Rate: 66.3% (112 merged)
Avg Code Changes: 8.2 files, +454/-267 lines

Top Keywords:

• test, pkg, code, error, lines, files, tests, validation

Key Phrases:

• • [ ], ---- *this section, *this section details, section details on, details on the

Characteristics:

• Code quality improvements and test fixes

Sample Tasks:

• ✅ #11066: Improve error messages for invalid target configuration in safe outputs
• ✅ #11069: Fix TypeScript type error in close_older_issues.cjs - add type guard for error.stack access
• ✅ #11082: Fix markdown code region balancer treating indented examples as nested fences

---

Cluster 4: Mcp Server

Size: 120 tasks (9.9% of total)
Success Rate: 61.7% (74 merged)
Avg Code Changes: 39.0 files, +503/-856 lines

Top Keywords:

• mcp, server, mcp server, gateway, tool, tools, mcp gateway, v0

Key Phrases:

• ---- *this section, *this section details, section details on, details on the, on the original

Characteristics:

• Focus on MCP server updates and tool additions

Sample Tasks:

• ✅ #11050: chore: Update Sentry MCP server to 0.27.0
• ✅ #11067: Add missing get_repository tool to repos toolset
• 🔴 #11085: Verify CLI version updates: Copilot 0.0.388, Sandbox Runtime 0.0.29, MCP Gateway v0.0.74

---

Cluster 5: Reference: Debug

Size: 82 tasks (6.7% of total)
Success Rate: 65.9% (54 merged)
Avg Code Changes: 23.7 files, +1016/-95 lines

Top Keywords:

• reference, debug, fix, review, tests, logs, run, failed

Key Phrases:

• reference: debug why, debug why the, reference: fix tests, fix tests reference:, reference: investigate why

Characteristics:

• Code quality improvements and test fixes

Sample Tasks:

• ✅ #11129: Fix safe-outputs server startup by copying tools.json to expected location
• 🔴 #11143: Add HTTP transport files to safe-outputs setup
• ✅ #11144: Fix setup.sh: Add missing safe-outputs MCP HTTP transport files

---

Cluster 6: On The

Size: 70 tasks (5.7% of total)
Success Rate: 67.1% (47 merged)
Avg Code Changes: 8.7 files, +383/-448 lines

Top Keywords:

• campaign, security, project, issue, fix, md, new, workflows

Key Phrases:

• security alert burndown, ---- *this section, *this section details, section details on, details on the

Characteristics:

• Campaign-related features and security fixes

Sample Tasks:

• ✅ #11070: chore: campaign discovery via label-based approach
• ✅ #11080: Clarify tracker-id is optional for campaign worker workflows
• ✅ #11087: Replace campaign fusion with first-class dispatch-only workers

---

Cluster 7: Job Id:

Size: 38 tasks (3.1% of total)
Success Rate: 78.9% (30 merged)
Avg Code Changes: 23.9 files, +2072/-139 lines

Top Keywords:

• job, fix, workflow, url, logs, cause, failure, actions

Key Phrases:

• analyze the workflow, the workflow logs,, workflow logs, identify, logs, identify the, identify the root

Characteristics:

• GitHub Actions workflow maintenance and fixes

Sample Tasks:

• 🔴 #11096: [WIP] Fix failing GitHub Actions workflow for JavaScript
• ✅ #11915: Fix staticcheck S1009 lint error: remove redundant nil check on map
• ✅ #12304: Fix lint-go workflow: Remove unused logger variable

---

Key Findings

1. Most Common Task Type: In The represents 36.4% of all tasks (443 tasks). These are primarily agentic-related tasks with a success rate of 70.9%.

2. Highest Success Rate: Job Id: cluster has the highest success rate at 78.9%. These tasks typically involve job, fix, workflow and tend to be more straightforward fixes.

3. Most Challenging Tasks: On The cluster has the lowest success rate at 58.4%. These tasks often involve workflow, issue, gh which may require more complex changes or multiple iterations.

4. Code Change Patterns: On average, tasks modify 18.4 files. Tasks with smaller, focused changes tend to have higher merge rates.

Recommendations

Based on clustering analysis:

1. Optimize for Success Patterns: Tasks in the 'Job Id:' cluster have 78.9% success rate. Consider breaking down complex tasks into smaller, focused requests similar to this pattern.

2. Improve Challenging Task Types: 'On The' tasks have lower success rates (58.4%). Consider providing more context, examples, or breaking these into multiple steps.

3. Provide Clear Requirements: Tasks with specific, actionable prompts tend to have higher success rates. Include file paths, expected outcomes, and acceptance criteria.

4. Leverage Successful Patterns: Review merged PRs in high-performing clusters to identify effective prompt patterns and replicate them.

5. Monitor Task Complexity: Tasks requiring changes to 5+ files have varied success rates. Consider splitting large tasks into multiple smaller PRs for better outcomes.

View Recent Tasks Table (Top 100)

PR #	Title	Cluster	Outcome	Files	+/-	Keywords
#14885	[WIP] Fix CI failure due to removed timeout_minute...	On The	🟡 Open	0	+0/-0	workflow, issue, gh
#14883	[WIP] Standardize path validation patterns across ...	Test + Pkg	🟡 Open	7	+325/-13	test, pkg, code
#14882	[WIP] Add security validation to jq filter process...	Test + Pkg	🟡 Open	2	+326/-3	test, pkg, code
#14881	[WIP] Fix MCP config generation for Copilot CLI	Mcp Server	🟡 Open	5	+107/-3	mcp, server, mcp server
#14878	Update CLI tools: Claude Code 2.1.39, Copilot 0.0....	Mcp Server	✅ Merged	149	+704/-698	mcp, server, mcp server
#14867	Set default max to 1 for assign-to-agent safe-outp...	In The	✅ Merged	4	+224/-5	agentic, update, project
#14866	Add 10-second delay between agent assignments to p...	In The	✅ Merged	5	+136/-4	agentic, update, project
#14860	Remove timeout_minutes from schema and add labels ...	On The	✅ Merged	6	+169/-28	workflow, issue, gh
#14854	Bump AWF to v0.13.14	On The	✅ Merged	148	+586/-582	workflow, issue, gh
#14853	Update npm dependencies: @actions/exec 3.0.0, @typ...	On The	✅ Merged	2	+17/-35	workflow, issue, gh

(Table truncated to first 10 entries for brevity - full data available in analysis files)

---

Analysis Date: 2026-02-11 05:07 UTC
Methodology: TF-IDF vectorization with K-means clustering (n_clusters=7)
Data Range: Last 30 days of copilot-created PRs
Repository: github/gh-aw

AI generated by Copilot Agent Prompt Clustering Analysis

• expires on Feb 18, 2026, 5:10 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-02-18T05:10:13.744Z.

Closed by Workflow