[mcp-analysis] MCP Structural Analysis - 2026-02-17

[github-actions[bot]]

Analysis of GitHub MCP tool responses evaluating both quantitative metrics (response sizes in tokens) and qualitative assessment (structural usefulness for agentic workflows). This provides insights into which tools are most efficient and effective for autonomous agents.

Executive Summary

Metric	Value
Tools Analyzed	13
Successful Responses	11
Failed/Error Responses	2
Average Tokens (Valid)	1,710
Average Usefulness Rating	3.8/5
Best Rated Tools	get_file_contents, get_label, list_branches, get_commit (5/5)
Worst Rated Tools	get_me, list_projects (1/5 - errors)
Most Efficient Tools	get_label (50 tokens), list_branches (80 tokens)
Largest Responses	list_releases (7,108 tokens), list_pull_requests (5,343 tokens)

Full Structural Analysis Report

Usefulness Ratings for Agentic Work

Ratings are based on completeness, actionability, clarity, efficiency, and relationship data quality.

Tool	Toolset	Rating	Assessment
get_file_contents	repos	⭐⭐⭐⭐⭐	Excellent - Clean file content retrieval. Essential for reading repository files. Simple structure.
get_label	labels	⭐⭐⭐⭐⭐	Excellent - Minimal, focused response. Exactly what's needed for label info. Perfect efficiency.
list_branches	repos	⭐⭐⭐⭐⭐	Excellent - Minimal, efficient branch listing. Essential fields only. Perfect for agentic work.
get_commit	repos	⭐⭐⭐⭐⭐	Excellent - With include_diff=false, very efficient. Gets commit metadata without bloat.
list_issues	issues	⭐⭐⭐⭐	Good - Good issue listing with embedded user and label data. Includes pagination. Minor: user data could be richer.
list_discussions	discussions	⭐⭐⭐⭐	Good - Compact, well-structured response. Includes category, user, timestamps. Good for listing.
search_repositories	search	⭐⭐⭐⭐	Good - Good search results with metadata. Includes topics, stars, language. Compact for search.
list_commits	repos	⭐⭐⭐⭐	Good - Good commit data with author/committer info. Reasonable size for listing.
list_workflows	actions	⭐⭐⭐⭐	Good - Useful data but pagination not respected (returns 30 instead of 1). Large response.
list_pull_requests	pull_requests	⭐⭐⭐	Adequate - Very large response (21KB+). Includes full PR body, complete repo objects. Excessive nesting and redundant data. Could be optimized.
list_releases	releases	⭐⭐⭐	Adequate - Very large (28KB+) for perPage=1. Release body content is verbose. Could be more compact.
get_me	context	⭐	Poor - 403 error, not accessible by integration. Cannot get user context.
list_projects	projects	⭐	Poor - Failed with 'not found' error. May require different permissions or owner type detection.

Schema Analysis

Tool	Type	Depth	Key Fields	Notes
get_file_contents	object	1	content, sha, encoding	Clean structure
list_issues	object	3	issues, pageInfo, totalCount	Nested user data
list_pull_requests	array	4	id, number, title, body, labels, user, head, base	Deep nesting with repo objects
list_workflows	object	2	total_count, workflows	Array of workflow objects
list_discussions	object	2	discussions, pageInfo, totalCount	Well-structured
get_label	object	1	color, description, id, name	Minimal
search_repositories	object	2	total_count, incomplete_results, items	Search results
list_branches	array	1	name, sha, protected	Simple array
list_commits	array	3	sha, commit, author, committer	Nested commit objects
get_commit	object	3	sha, commit, author, committer	Similar to list
list_releases	array	2	tag_name, name, body, created_at	Release objects

Response Size Analysis

Toolset	Avg Tokens	Tools Tested	Size Rating
releases	7,108	1	🔴 Very Large
pull_requests	5,343	1	🔴 Very Large
actions	3,884	1	🟠 Large
repos	482	4	🟢 Efficient
issues	800	1	🟢 Efficient
search	450	1	🟢 Efficient
commits	325	2	🟢 Very Efficient
discussions	250	1	🟢 Very Efficient
labels	50	1	🟢 Extremely Efficient

Tool-by-Tool Analysis

Tool	Toolset	Tokens	Schema	Rating	Notes
get_me	context	0	error	1	403 error - not accessible by integration
get_file_contents	repos	1,200	object	5	Clean file content retrieval. Essential for reading repository files
list_issues	issues	800	object	4	Good issue listing with embedded user and label data
list_pull_requests	pull_requests	5,343	array	3	Very large response (21KB+), excessive nesting and redundant data
list_workflows	actions	3,884	object	4	Large response (15KB+) for perPage=1. Returns 30 workflows despite requesting 1
list_discussions	discussions	250	object	4	Compact, well-structured response
get_label	labels	50	object	5	Minimal, focused response. Excellent efficiency
search_repositories	search	450	object	4	Good search results with metadata
list_branches	repos	80	array	5	Minimal, efficient branch listing
list_commits	repos	350	array	4	Good commit data with author/committer info
get_commit	repos	300	object	5	With include_diff=false, very efficient
list_releases	releases	7,108	array	3	Very large (28KB+) for perPage=1. Verbose release body
list_projects	projects	0	error	1	Failed with 'not found' error

30-Day Trend Summary

Metric	Value
Data Points	13
Days of Data	1 (baseline)
Total Tokens Today	19,709
Average per Tool	1,516
Trend	N/A (first day)

Key Findings

🟢 Highly Efficient Tools (< 500 tokens, rating 4+)

• get_label (50 tokens, 5⭐) - Ideal for label operations
• list_branches (80 tokens, 5⭐) - Perfect for branch listing
• list_discussions (250 tokens, 4⭐) - Compact discussion listing
• get_commit (300 tokens, 5⭐) - Efficient commit metadata (without diff)
• list_commits (350 tokens, 4⭐) - Good for commit history
• search_repositories (450 tokens, 4⭐) - Efficient search

🟡 Moderate Size, Good Value (500-2000 tokens, rating 4+)

• list_issues (800 tokens, 4⭐) - Comprehensive issue data
• get_file_contents (1,200 tokens, 5⭐) - Essential file reading

🔴 Large Responses Needing Optimization (> 3000 tokens)

• list_workflows (3,884 tokens, 4⭐) - Pagination not respected
• list_pull_requests (5,343 tokens, 3⭐) - Excessive nesting, redundant repo data
• list_releases (7,108 tokens, 3⭐) - Verbose release body content

❌ Non-Functional Tools

• get_me - 403 permission error
• list_projects - Not found error (may need user vs org detection)

Recommendations

For Agent Developers

Prefer these high-value tools:

• Use get_file_contents for reading files (5⭐, efficient)
• Use list_branches for branch operations (5⭐, minimal)
• Use get_commit with include_diff=false for commit metadata (5⭐)
• Use get_label for label operations (5⭐, tiny response)
• Use list_issues for issue listing (4⭐, good balance)

Use with caution (large responses):

• list_pull_requests - Consider filtering or using search instead
• list_releases - Be aware of large body content
• list_workflows - Pagination parameter not respected

Avoid (non-functional):

• get_me - Use alternative methods for user context
• list_projects - May require different approach

For MCP Tool Maintainers

High Priority Optimizations:

1. list_workflows - Respect perPage parameter (currently returns 30 regardless)
2. list_pull_requests - Consider:
   • Option to exclude full repo objects from head/base
   • Truncate or paginate PR body content
   • Reduce nesting depth
3. list_releases - Add option to truncate release body
4. get_me - Fix permission issues or document requirements
5. list_projects - Improve error handling for user vs org detection

Context Efficiency Wins:

• get_commit with include_diff=false is a great pattern - apply to other tools
• get_label demonstrates ideal minimalism
• list_branches shows perfect balance of data vs size

Data Collection Methodology

Each tool was called with minimal parameters to establish baseline response characteristics:

• perPage=1 where applicable to minimize data
• include_diff=false for commit tools to avoid large diffs
• Focused on representative tools from each toolset

Token estimates calculated at ~4 characters per token. Schema analysis examined:

• Data structure types (object, array, primitive)
• Nesting depth (how deeply nested is the data)
• Key fields present in responses
• Pagination support and metadata

Usefulness ratings (1-5) evaluated:

• Completeness: All needed info present?
• Actionability: Can agent act directly on this data?
• Clarity: Intuitive, consistent structure?
• Efficiency: Optimized context usage?
• Relationships: Related entities included or linkable?

Analysis Artifacts

• Analysis Data: Stored in /tmp/gh-aw/cache-memory/mcp_analysis.jsonl for 30-day trending
• Visualization Script: /tmp/gh-aw/python/analyze_mcp.py (requires pandas, matplotlib, seaborn)
• Run ID: §22096346548

Next Steps

This is the first baseline analysis. Future runs will:

• Track trends over 30 days
• Identify response size changes
• Monitor usefulness rating evolution
• Generate visual charts when Python scientific libraries are available

Note: Python visualization requires pandas, matplotlib, and seaborn. Current environment lacks these dependencies.

AI generated by GitHub MCP Structural Analysis

• expires on Feb 24, 2026, 11:22 AM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-02-24T11:22:06.157Z.

Closed by Workflow