[prompt-clustering] Copilot Agent Prompt Clustering Analysis - February 2026

[github-actions[bot]]

Daily NLP-based clustering analysis of copilot agent task prompts using TF-IDF vectorization and K-means clustering.

Summary

• Total Tasks Analyzed: 999
• Clusters Identified: 7
• Overall Success Rate: 77.5%
• Analysis Date: 2026-02-01

Key Insights

1. Highest Success Rate: General Improvements cluster has 84.5% success rate with 265 tasks
2. Most Common Task Type: Bug Fixes & Error Handling represents 28.1% of all tasks
3. Attention Needed: Bug Fixes & Error Handling cluster has only 7.3% success rate - indicates potential challenges
4. Most Complex Tasks: General Improvements cluster averages 890 lines added per PR
5. Performance Distribution: 4 out of 7 clusters perform above the 77.5% overall average

Success Rate by Cluster

Rank	Cluster Theme	Tasks	Success Rate	Avg Changes	Top Keywords
1	Bug Fixes & Error Handling	281	77.2%	+531/-173	validation, error, field
2	General Improvements	265	84.5%	+868/-180	md, workflow, workflows
3	Testing & Quality Assurance	189	82.0%	+719/-300	pkg, pkg workflow, workflow
4	General Improvements	95	77.9%	+468/-493	mcp, lines, server
5	General Improvements	70	74.3%	+890/-232	firewall, network, workflow
6	Updates & Upgrades	58	84.5%	+362/-210	version, cli, updates
7	Bug Fixes & Error Handling	41	7.3%	+290/-124	pr, make, work

View Detailed Cluster Analysis

Cluster 1: Bug Fixes & Error Handling

• Size: 281 tasks (28.1% of total)
• Success Rate: 77.2% (217/281 merged)
• Average Changes: 531 additions, 173 deletions
• Average Files Changed: 12.8
• Average Comments: 1.6

Top Keywords: validation, error, field, job, workflow

Example PRs: #3779, #3917, #2813

View Sample Task

Summary The copilot engine now automatically creates all directories specified in --add-dir arguments before executing the copilot CLI. This prevents potential errors when the copilot CLI attempts to access directories that don t exist yet. Problem The copilot engine builds a list of --add-dir argum...

---

Cluster 2: General Improvements

• Size: 265 tasks (26.5% of total)
• Success Rate: 84.5% (224/265 merged)
• Average Changes: 868 additions, 180 deletions
• Average Files Changed: 7.7
• Average Comments: 1.3

Top Keywords: md, workflow, workflows, github, agentic

Example PRs: #3505, #4086, #4003

View Sample Task

Overview Updated the imports documentation docs src content docs reference imports.md to add a concise syntax reference for all supported import path formats. Changes Added Minimal Path Formats Section The documentation now includes a concise Path Formats section with syntax reference for 1. Local P...

---

Cluster 3: Testing & Quality Assurance

• Size: 189 tasks (18.9% of total)
• Success Rate: 82.0% (155/189 merged)
• Average Changes: 719 additions, 300 deletions
• Average Files Changed: 23.0
• Average Comments: 1.9

Top Keywords: pkg, pkg workflow, workflow, tests, test

Example PRs: #2253, #2400, #2173

View Sample Task

• x Rename logger variables to log in all 5 Go files - x pkg cli trialcommand.go - x pkg workflow claudeengine.go - x pkg workflow expressions.go - x pkg workflow js.go - x pkg workflow mcp-config.go - x Convert package-level loggers to function-scoped loggers - x Update .github workflows go-logger....

---

Cluster 4: General Improvements

• Size: 95 tasks (9.5% of total)
• Success Rate: 77.9% (74/95 merged)
• Average Changes: 468 additions, 493 deletions
• Average Files Changed: 18.8
• Average Comments: 1.7

Top Keywords: mcp, lines, server, mcp server, json

Example PRs: #2268, #4236, #3997

View Sample Task

Problem The smoke-opencode workflow was experiencing a 100 failure rate because OpenCode agents couldn t access configured MCP tools safe-outputs, github, etc. . The root cause was a configuration format mismatch - gh-aw generates MCP config in Copilot format at tmp gh-aw mcp-config mcp-servers.json...

---

Cluster 5: General Improvements

• Size: 70 tasks (7.0% of total)
• Success Rate: 74.3% (52/70 merged)
• Average Changes: 890 additions, 232 deletions
• Average Files Changed: 11.5
• Average Comments: 1.4

Top Keywords: firewall, network, workflow, logs, block

Example PRs: #2430, #2318, #3068

View Sample Task

Integrate gh-aw-firewall in copilot agentic engine - x Add firewall version constant to constants.go - x Update copilotengine.go to wrap CLI with firewall when network restrictions apply - x Implement logic to skip firewall when network allows all domains - x Use --env-all flag for environment varia...

---

Cluster 6: Updates & Upgrades

• Size: 58 tasks (5.8% of total)
• Success Rate: 84.5% (49/58 merged)
• Average Changes: 362 additions, 210 deletions
• Average Files Changed: 23.5
• Average Comments: 1.5

Top Keywords: version, cli, updates, workflow, updated

Example PRs: #2303, #3927, #2310

View Sample Task

Problem The Smoke OpenCode workflow run a href was failing because the createissue job couldn t find the expected agentoutput.json file. Investigation revealed that the agent job appeared to succeed but didn t actually execute the OpenCode agent, resulting in no safe-outputs file being generated. Ro...

---

Cluster 7: Bug Fixes & Error Handling

• Size: 41 tasks (4.1% of total)
• Success Rate: 7.3% (3/41 merged)
• Average Changes: 290 additions, 124 deletions
• Average Files Changed: 1.5
• Average Comments: 0.1

Top Keywords: pr, make, work, issue, agent tips

Example PRs: #2183, #2187, #3757

View Sample Task

Thanks for asking me to work on this. I will get started on it and keep this PR s description up to date as I form a plan and make progress. !-- START COPILOT CODING AGENT SUFFIX -- details summary Original prompt summary Migrate this JavaScript memory server to Wasm Component as one of the examples...

---

Recommendations

1. Improve Bug Fixes & Error Handling Tasks: With only 7.3% success rate, review the keywords (pr, make, work) and failed PRs to identify common failure patterns
2. Leverage Successful Patterns: General Improvements tasks show 84.5% success. Study these patterns (keywords: md, workflow, workflows) and apply similar approaches to other task types
3. Monitor Complex Tasks: General Improvements tasks average 890 additions. Consider breaking down large tasks or providing more specific requirements
4. Optimize High-Volume Tasks: Bug Fixes & Error Handling represents 281 tasks. Creating task templates or guidelines for this category could improve consistency and success rates

View Analysis Methodology

Methodology

1. Data Collection: Extracted task descriptions from 999 copilot-created PR bodies
2. Text Preprocessing: Cleaned markdown, removed code blocks, normalized whitespace
3. Feature Extraction: TF-IDF vectorization with 200 features, 1-3 word n-grams
4. Clustering: K-means with optimal k=7 determined by silhouette score
5. Theme Identification: Automated theme detection based on top keywords
6. Metrics Analysis: Success rates, code complexity, and engagement metrics per cluster

---

References:

• Workflow Run §21558161560

AI generated by Copilot Agent Prompt Clustering Analysis

• expires on Feb 8, 2026, 6:51 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-02-08T06:51:05.814Z.