Refactor cache-memory to use simple file share with path-specific access controls, artifact management, enhanced logs integration, and frontmatter workflow naming

[Copilot]

This PR refactors the cache-memory feature to provide a simple file share approach instead of mounting MCP memory servers, making it more transparent and easier for LLMs to use.

What Changed

The cache-memory feature previously mounted an MCP memory server (@modelcontextprotocol/server-memory) that provided a specialized memory tool. This has been replaced with a simple file share at /tmp/cache-memory/ that LLMs can access using standard file operations.

Key Benefits

• Simpler Architecture: Eliminates the complexity of MCP server mounting and npm package dependencies
• Transparent Access: LLMs can directly read/write files instead of using specialized memory tools
• Better Control: LLMs have full control over file organization and structure
• Reduced Dependencies: No longer requires external npm packages or Docker containers
• Path-Specific Access: Claude gets precise file access tools restricted to only the cache directory
• Execution Plan Management: Dev workflow now maintains persistent execution plans across runs
• Always-Available Artifacts: Cache-memory data is always uploaded as artifacts for debugging and analysis
• Enhanced Logs Integration: Logs command now downloads and analyzes cache-memory artifacts
• Flexible Workflow Naming: Support for frontmatter name: field to override workflow titles

Implementation Details

Removed MCP Server Mounting

• Updated claude_engine.go and custom_engine.go to no longer generate MCP memory server configurations
• Removed mcp__memory from Claude engine's allowed tools list

Path-Specific File Access Tools

• When cache-memory is enabled, Claude now gets path-specific tools: Read(/tmp/cache-memory/*), Write(/tmp/cache-memory/*), Edit(/tmp/cache-memory/*), MultiEdit(/tmp/cache-memory/*)
• These tools provide access only to files within the /tmp/cache-memory/ directory for precise access control
• Tools are only added if not already present to avoid duplicates
• Default file operation tools remain available for general use

Enhanced Dev Workflow

• Added cache-memory: true to the dev workflow configuration
• Updated workflow to include execution plan management with /tmp/cache-memory/plan.md
• Claude can now maintain continuity across workflow runs by remembering and recalling execution plans
• Includes structured planning guidance for objectives, progress tracking, and decision recording

Always-Upload Artifact Management

• Modified cache generation to always upload cache-memory directory as artifacts (not just when retention-days is configured)
• Changed artifact name from "cache-memory-data" to "cache-memory" for consistency
• Retention-days is now optional and only included when explicitly configured in workflow frontmatter
• Enables consistent debugging and analysis of cache usage across workflow runs

Enhanced Logs Command Integration

• Updated logs command documentation to include cache-memory artifacts in downloaded files list
• Added cache-memory directory detection in log analysis with file count and size reporting
• Cache-memory artifacts are automatically downloaded and analyzed with other workflow artifacts
• Verbose mode now reports cache-memory directory contents for better visibility

Frontmatter Workflow Naming Support

• Added support for name: field in workflow frontmatter to override workflow titles
• Frontmatter name takes precedence over H1 header extraction for workflow naming
• Maintains full backward compatibility with existing workflows that use H1 headers
• Enables more flexible and concise workflow naming (e.g., dev workflow now uses "dev" instead of "Development Assistant")

Enhanced LLM Prompting

Added comprehensive prompting in compiler.go that informs LLMs about the cache folder:

• Clear explanation of read/write access at /tmp/cache-memory/
• Persistence behavior across workflow runs
• File organization examples and best practices
• "Last write wins" behavior for concurrent access

Updated Cache Steps

Modified cache generation in cache.go to:

• Create the cache directory with informative logging
• Use "file share" terminology instead of "MCP data"
• Maintain GitHub Actions cache integration for persistence
• Always upload artifacts for debugging and analysis

Removed Deprecated Fields

• Removed deprecated DockerImage field from CacheMemoryConfig struct
• Removed deprecated docker-image parsing logic that was no longer used

Updated Documentation

Completely rewrote the cache-memory documentation to:

• Describe the new file share approach
• Provide file organization guidance
• Include migration notes from the previous MCP approach
• Add troubleshooting for file-based operations

Example Usage

Before (MCP approach):

tools:
  cache-memory: true
---
Remember that the user prefers verbose error messages.

After (file share approach):

name: "dev"
tools:
  cache-memory: true
---
Store user preferences in /tmp/cache-memory/preferences.json for future reference.
Check for existing execution plan in /tmp/cache-memory/plan.md and maintain continuity.

Artifact Management

Cache-memory now always generates artifact upload steps:

- name: Upload cache-memory data as artifact
  uses: actions/upload-artifact@v4
  with:
    name: cache-memory
    path: /tmp/cache-memory
    # retention-days only included if configured

Workflow Naming

Workflows now support flexible naming through frontmatter:

---
name: "dev"  # Override H1 header with custom name
tools:
  cache-memory: true
---

Backward Compatibility

All existing cache-memory configuration syntax continues to work. The deprecated docker-image field has been removed as it was no longer functional in the file share approach. Workflows without frontmatter name: field continue to use H1 header extraction.

Testing

• Updated test cases in claude_engine_tools_test.go to reflect the new path-specific tool availability
• Modified the test workflow test-claude-cache-memory.md to demonstrate file operations
• Enhanced the dev workflow to demonstrate execution plan management
• All existing tests continue to pass
• Generated workflows now show path-specific tool access: Edit(/tmp/cache-memory/*), MultiEdit(/tmp/cache-memory/*), Read(/tmp/cache-memory/*), Write(/tmp/cache-memory/*) + default tools
• Verified artifact upload steps are generated in all cache-memory enabled workflows
• Confirmed logs command can detect and report cache-memory artifacts
• Validated frontmatter name field override functionality

This change simplifies the cache-memory feature while providing precise access controls, execution plan management capabilities, always-available artifact uploads for debugging, enhanced logs integration for better analysis and troubleshooting, and flexible workflow naming through frontmatter configuration.

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[pelikhan]

@copilot update Claude tools with Read/Write access to all files in the cache memory directory

[Copilot]

@copilot update Claude tools with Read/Write access to all files in the cache memory directory

Updated Claude tools with Read/Write access to cache memory directory. When cache-memory is enabled, Claude now explicitly gets Read, Write, Edit, and MultiEdit tools ensuring full file system access to /tmp/cache-memory/. Changes in commit c2535f2.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.contoso.com
  • Triggering command: /tmp/go-build336151800/b279/cli.test -test.testlogfile=/tmp/go-build336151800/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[pelikhan]

@copilot for Claude, be more precise and only provide read/write access to the cache folder when configuring cache tools

[Copilot]

@copilot for Claude, be more precise and only provide read/write access to the cache folder when configuring cache tools

Made cache tools path-specific to only access the cache folder. Claude now gets precise tools: Read(/tmp/cache-memory/*), Write(/tmp/cache-memory/*), Edit(/tmp/cache-memory/*), MultiEdit(/tmp/cache-memory/*) instead of general file access. Changes in commit 6a0e5a4.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.contoso.com
  • Triggering command: /tmp/go-build1984934015/b279/cli.test -test.testlogfile=/tmp/go-build1984934015/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[pelikhan]

@copilot add cqche-memory to deva genetic workflow, update dev to remember/recall the execution plan in plan.md

[Copilot]

@copilot add cqche-memory to deva genetic workflow, update dev to remember/recall the execution plan in plan.md

Added cache-memory to dev agentic workflow and updated it to remember/recall execution plans in plan.md. The workflow now instructs Claude to maintain /tmp/cache-memory/plan.md for execution continuity across runs. Changes in commit 22b78e0.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.contoso.com
  • Triggering command: /tmp/go-build4268485068/b279/cli.test -test.testlogfile=/tmp/go-build4268485068/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[pelikhan]

@copilot

• update compiler to upload the cache memory directory to the "cache-memory" artifacts (runs always)
• only add generated code if cache-memory is enabled
• update logs to download cache-memory artifacts

[Copilot]

@copilot

• update compiler to upload the cache memory directory to the "cache-memory" artifacts (runs always)...

Updated cache-memory implementation:

• Cache-memory directory now always uploads to "cache-memory" artifacts (runs always), not just when retention-days is set
• Generated code only added when cache-memory is enabled (already implemented)
• Enhanced logs command to download and detect cache-memory artifacts with file count and size reporting
  Changes in commit 97aa1ab.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.contoso.com
  • Triggering command: /tmp/go-build3445945749/b279/cli.test -test.testlogfile=/tmp/go-build3445945749/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[pelikhan]

@copilot add test agentic workflow under pkg/cli/workflows that uses cache-memory

[Copilot]

@copilot add test agentic workflow under pkg/cli/workflows that uses cache-memory

A comprehensive test agentic workflow using cache-memory already exists at pkg/cli/workflows/test-claude-cache-memory.md. It demonstrates file share functionality, persistence across runs, artifact uploads, and integration with other tools like GitHub. No additional test workflow needed.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.contoso.com
  • Triggering command: /tmp/go-build3699181366/b279/cli.test -test.testlogfile=/tmp/go-build3699181366/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[pelikhan]

@copilot fix tests

[Copilot]

@copilot fix tests

Tests are now fixed. The issue was missing npm dependencies (prettier) and golangci-lint. Installed npm packages and golangci-lint, all tests and linting now pass successfully. Dependencies are excluded from git via .gitignore.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.contoso.com
  • Triggering command: /tmp/go-build1613566566/b279/cli.test -test.testlogfile=/tmp/go-build1613566566/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
  • Triggering command: /tmp/go-build3927898016/b279/cli.test -test.testlogfile=/tmp/go-build3927898016/b279/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)