Skip to content

use claude cli directly#824

Merged
pelikhan merged 15 commits intomainfrom
copilot/fix-f0ab22fd-0c95-419f-b0d3-a61baae89f29
Sep 15, 2025
Merged

use claude cli directly#824
pelikhan merged 15 commits intomainfrom
copilot/fix-f0ab22fd-0c95-419f-b0d3-a61baae89f29

Conversation

Copy link
Contributor

Copilot AI commented Sep 14, 2025

Implemented support for generating the allowlist of playwright tools based on the copilot agent supported tools, as requested in comment #3290301233.

Changes Made

  • Added shared GetCopilotAgentPlaywrightTools() function - Located in pkg/workflow/agentic_engine.go to provide consistent access to all 21 copilot agent playwright tools across engines
  • Enhanced Claude engine playwright support - Updated expandNeutralToolsToClaudeTools() and computeAllowedClaudeToolsString() to recognize playwright as a standard MCP tool and expand it with copilot agent tools
  • Enhanced Codex engine playwright support - Added expandNeutralToolsToCodexTools() method and updated RenderMCPConfig() to support the same playwright tool expansion
  • Added comprehensive test coverage - Created codex_playwright_test.go and updated existing tests to verify all 21 playwright tools are properly included when using playwright: null
  • Updated neutral tools tests - Modified tests to expect the full playwright MCP configuration with all copilot agent tools
  • Verified cross-engine compatibility - All engines (Claude, Codex, Custom) now consistently provide the same playwright tools list

Playwright Tools Included (21 total)

The following tools are now automatically included when a workflow specifies playwright: null:

  • mcp__playwright__browser_click
  • mcp__playwright__browser_close
  • mcp__playwright__browser_console_messages
  • mcp__playwright__browser_drag
  • mcp__playwright__browser_evaluate
  • mcp__playwright__browser_file_upload
  • mcp__playwright__browser_fill_form
  • mcp__playwright__browser_handle_dialog
  • mcp__playwright__browser_hover
  • mcp__playwright__browser_install
  • mcp__playwright__browser_navigate
  • mcp__playwright__browser_navigate_back
  • mcp__playwright__browser_network_requests
  • mcp__playwright__browser_press_key
  • mcp__playwright__browser_resize
  • mcp__playwright__browser_select_option
  • mcp__playwright__browser_snapshot
  • mcp__playwright__browser_tabs
  • mcp__playwright__browser_take_screenshot
  • mcp__playwright__browser_type
  • mcp__playwright__browser_wait_for

All tests pass and workflows compile correctly with the enhanced playwright tool configuration.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Migrate the claude agentic engine to use the v1.x releases of the claude code custom github action. - find current v1.x release and pin to it - Update the generated code according to the instructions in https://github.com/anthropics/claude-code-actio... Migrate Claude agentic engine to use claude-code-action v1.x Sep 14, 2025
Copilot AI requested a review from pelikhan September 14, 2025 19:09
Copy link
Contributor

@pelikhan pelikhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot

  • add --mcp-debug in claude args
  • render allow tool list as comment in yaml

- Add --mcp-debug flag to claude_args for improved MCP debugging capabilities
- Render allowed tools list as YAML comments above the with: section for better visibility
- Fix claude_args formatting to properly handle standalone flags vs paired arguments
- Optimize comment line appending using gosimple suggestion

Both changes improve the debugging and visibility of Claude Code Action v1.x workflows while maintaining backward compatibility and all test coverage.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Sep 14, 2025

@copilot

  • add --mcp-debug in claude args...

Both changes have been implemented successfully:

  1. Added --mcp-debug flag: The flag is now included in the claude_args as a standalone parameter for enhanced MCP debugging capabilities.

  2. Added allowed tools comment: The complete sorted list of allowed tools is now rendered as YAML comments above the with: section for better visibility and debugging.

The implementation properly handles standalone flags vs paired arguments in the claude_args formatting, and all tests pass. The changes improve debugging capabilities while maintaining full backward compatibility.

Commit: faa0f9c

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.contoso.com
    • Triggering command: /tmp/go-build3933355634/b278/cli.test -test.testlogfile=/tmp/go-build3933355634/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build4179524052/b278/cli.test -test.testlogfile=/tmp/go-build4179524052/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build1705290801/b278/cli.test -test.testlogfile=/tmp/go-build1705290801/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title Migrate Claude agentic engine to use claude-code-action v1.x Migrate Claude agentic engine to use claude-code-action v1.x with enhanced debugging Sep 14, 2025
Copilot AI requested a review from pelikhan September 14, 2025 19:31
@pelikhan
Copy link
Contributor

@copilot add playwright tool in dev agentic workflow

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Sep 14, 2025

@copilot add playwright tool in dev agentic workflow

Added playwright tool to the dev agentic workflow with GitHub domain access. The workflow now includes containerized Playwright support for browser automation, web testing, and accessibility analysis.

Commit: d16ba4a

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.contoso.com
    • Triggering command: /tmp/go-build3475448428/b278/cli.test -test.testlogfile=/tmp/go-build3475448428/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build2262188096/b278/cli.test -test.testlogfile=/tmp/go-build2262188096/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title Migrate Claude agentic engine to use claude-code-action v1.x with enhanced debugging Migrate Claude agentic engine to use claude-code-action v1.x with enhanced debugging and add Playwright tool to dev workflow Sep 14, 2025
@pelikhan
Copy link
Contributor

@copilot update generated code to use claude code cli directly using npx. Do not use action anymore.

https://www.npmjs.com/package/@anthropic-ai/claude-code

…ub Action

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Sep 14, 2025

@copilot update generated code to use claude code cli directly using npx. Do not use action anymore.

https://www.npmjs.com/package/@anthropic-ai/claude-code

Migration completed successfully!

The Claude engine now uses npx @anthropic-ai/claude-code@latest directly instead of the GitHub Action. All functionality is preserved while simplifying the dependency chain.

Key changes:

  • Replaced anthropics/claude-code-action@v1 with direct npx CLI execution
  • Converted all action inputs to CLI arguments (--print, --allowed-tools, --mcp-config, etc.)
  • Maintained all existing features: MCP config, network permissions, timeout, environment variables
  • Updated all tests to reflect the new CLI structure

All 331 tests pass and workflow compilation works correctly. You can see the new CLI structure in the recompiled .github/workflows/dev.lock.yml.

Commit: 776dd9e

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.contoso.com
    • Triggering command: /tmp/go-build217971491/b278/cli.test -test.testlogfile=/tmp/go-build217971491/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build1509632307/b278/cli.test -test.testlogfile=/tmp/go-build1509632307/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build1576335102/b278/cli.test -test.testlogfile=/tmp/go-build1576335102/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title Migrate Claude agentic engine to use claude-code-action v1.x with enhanced debugging and add Playwright tool to dev workflow Migrate Claude agentic engine to use claude-code CLI directly via npx and add Playwright tool to dev workflow Sep 14, 2025
@pelikhan
Copy link
Contributor

@copilot

  • use tee to pipe the Claude cli output, make sure the exit code is passed to the github action runner

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Sep 15, 2025

@copilot

Update Claude code arguments:...

Added both CLI arguments to Claude engine configuration:

  1. --verbose flag: Controlled by new boolean verbose field in engine config - only added when verbose: true is specified
  2. --max-runs argument: Controlled by new integer max-runs field in engine config

Example configuration:

engine:
  id: claude
  verbose: true
  max-runs: 3

Updated schemas, configuration parsing, and CLI generation. All tests pass and backward compatibility is maintained.

Commit: 9ac5eb4

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.contoso.com
    • Triggering command: /tmp/go-build1682057997/b278/cli.test -test.testlogfile=/tmp/go-build1682057997/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build3744813866/b278/cli.test -test.testlogfile=/tmp/go-build3744813866/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build3804111759/b278/cli.test -test.testlogfile=/tmp/go-build3804111759/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title use claude cli directly use claude cli directly with enhanced configuration support Sep 15, 2025
Copilot AI requested a review from pelikhan September 15, 2025 00:35
@pelikhan
Copy link
Contributor

@copilot revert previous commit.

  • always add --verbose to the Claude arguments
  • make sure --max-turns is supported

…guments

- Reverted conditional --verbose and --max-runs support from previous commit
- Claude now always includes --verbose flag for enhanced debugging output
- --max-turns support remains available as requested
- All tests pass and workflows compile correctly

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Sep 15, 2025

@copilot revert previous commit.

  • always add --verbose to the Claude arguments...

Reverted the previous commit and implemented the requested changes:

  1. Reverted commit 9ac5eb4 - Removed conditional --verbose and --max-runs support
  2. Always add --verbose - Claude CLI now always includes --verbose flag for enhanced debugging output
  3. --max-turns support confirmed - The --max-turns parameter remains available and properly supported

All tests pass and workflows compile correctly with the new configuration.

Commit: 8bac88d

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.contoso.com
    • Triggering command: /tmp/go-build187988065/b278/cli.test -test.testlogfile=/tmp/go-build187988065/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build909309107/b278/cli.test -test.testlogfile=/tmp/go-build909309107/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@pelikhan
Copy link
Contributor

@dsyme looks like it's working. might be some log parsing issue with the verbose flag but this will make debugging mcp servers might easier to track.

@pelikhan
Copy link
Contributor

@copilot update claude code args to disable telemetry.

set env var DISABLE_TELEMETRY=1, DISABLE_ERROR_REPORTING=1, DISABLE_BUG_COMMAND=1

Docs:

Data usage

Learn about Anthropic's data usage policies for Claude

Data policies

Data training policy

Consumer users (Free, Pro, and Max plans):
Starting August 28, 2025, we're giving you the choice to allow your data to be used to improve future Claude models.

We will train new models using data from Free, Pro, and Max accounts when this setting is on (including when you use Claude Code from these accounts).

  • If you're a current user, you can select your preference now and your selection will immediately go into effect.
    This setting will only apply to new or resumed chats and coding sessions on Claude. Previous chats with no additional activity will not be used for model training.
  • You have until September 28, 2025 to make your selection.
    If you're a new user, you can pick your setting for model training during the signup process.
    You can change your selection at any time in your Privacy Settings.

Commercial users: (Team and Enterprise plans, API, 3rd-party platforms, and Claude Gov) maintain existing policies: Anthropic does not train generative models using code or prompts sent to Claude Code under commercial terms, unless the customer has chosen to provide their data to us for model improvement (e.g. Developer Partner Program).

Development Partner Program

If you explicitly opt in to methods to provide us with materials to train on, such as via the Development Partner Program, we may use those materials provided to train our models. An organization admin can expressly opt-in to the Development Partner Program for their organization. Note that this program is available only for Anthropic first-party API, and not for Bedrock or Vertex users.

Feedback using the /bug command

If you choose to send us feedback about Claude Code using the /bug command, we may use your feedback to improve our products and services. Transcripts shared via /bug are retained for 30 days.

Data retention

Anthropic retains Claude Code data based on your account type and preferences.

Consumer users (Free, Pro, and Max plans):

  • Users who allow data use for model improvement: 5-year retention period to support model development and safety improvements
  • Users who don't allow data use for model improvement: 30-day retention period
  • Privacy settings can be changed at any time at claude.ai/settings/data-privacy-controls.

Commercial users (Team, Enterprise, and API):

  • Standard: 30-day retention period
  • Zero data retention: Available with appropriately configured API keys - Claude Code will not retain chat transcripts on servers
  • Local caching: Claude Code clients may store sessions locally for up to 30 days to enable session resumption (configurable)

Learn more about data retention practices in our Privacy Center.

For full details, please review our Commercial Terms of Service (for Team, Enterprise, and API users) or Consumer Terms (for Free, Pro, and Max users) and Privacy Policy.

Data flow and dependencies

Claude Code data flow diagram

Claude Code is installed from NPM. Claude Code runs locally. In order to interact with the LLM, Claude Code sends data over the network. This data includes all user prompts and model outputs. The data is encrypted in transit via TLS and is not encrypted at rest. Claude Code is compatible with most popular VPNs and LLM proxies.

Claude Code is built on Anthropic's APIs. For details regarding our API's security controls, including our API logging procedures, please refer to compliance artifacts offered in the Anthropic Trust Center.

Telemetry services

Claude Code connects from users' machines to the Statsig service to log operational metrics such as latency, reliability, and usage patterns. This logging does not include any code or file paths. Data is encrypted in transit using TLS and at rest using 256-bit AES encryption. Read more in the Statsig security documentation. To opt out of Statsig telemetry, set the DISABLE_TELEMETRY environment variable.

Claude Code connects from users' machines to Sentry for operational error logging. The data is encrypted in transit using TLS and at rest using 256-bit AES encryption. Read more in the Sentry security documentation. To opt out of error logging, set the DISABLE_ERROR_REPORTING environment variable.

When users run the /bug command, a copy of their full conversation history including code is sent to Anthropic. The data is encrypted in transit and at rest. Optionally, a Github issue is created in our public repository. To opt out of bug reporting, set the DISABLE_BUG_COMMAND environment variable.

Default behaviors by API provider

By default, we disable all non-essential traffic (including error reporting, telemetry, and bug reporting functionality) when using Bedrock or Vertex. You can also opt out of all of these at once by setting the CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC environment variable. Here are the full default behaviors:

Service Anthropic API Vertex API Bedrock API
Statsig (Metrics) Default on.
DISABLE_TELEMETRY=1 to disable.
Default off.
CLAUDE_CODE_USE_VERTEX must be 1.
Default off.
CLAUDE_CODE_USE_BEDROCK must be 1.
Sentry (Errors) Default on.
DISABLE_ERROR_REPORTING=1 to disable.
Default off.
CLAUDE_CODE_USE_VERTEX must be 1.
Default off.
CLAUDE_CODE_USE_BEDROCK must be 1.
Anthropic API (/bug reports) Default on.
DISABLE_BUG_COMMAND=1 to disable.
Default off.
CLAUDE_CODE_USE_VERTEX must be 1.
Default off.
CLAUDE_CODE_USE_BEDROCK must be 1.

All environment variables can be checked into settings.json (read more).

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Sep 15, 2025

@copilot update claude code args to disable telemetry.

set env var DISABLE_TELEMETRY=1, DISABLE_ERROR_REPORTING=1, DISABLE_BUG_COMMAND=1
...

Added telemetry disabling environment variables to Claude CLI execution. The following variables are now set for all Claude workflows:

  • DISABLE_TELEMETRY=1
  • DISABLE_ERROR_REPORTING=1
  • DISABLE_BUG_COMMAND=1

This ensures no data is sent to external telemetry services for privacy and security.

Commit: 909eb9d

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.contoso.com
    • Triggering command: /tmp/go-build1570844558/b278/cli.test -test.testlogfile=/tmp/go-build1570844558/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build576840391/b278/cli.test -test.testlogfile=/tmp/go-build576840391/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)
    • Triggering command: /tmp/go-build1655489246/b278/cli.test -test.testlogfile=/tmp/go-build1655489246/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title use claude cli directly with enhanced configuration support use claude cli directly with enhanced configuration support and telemetry disabling Sep 15, 2025
@pelikhan pelikhan changed the title use claude cli directly with enhanced configuration support and telemetry disabling use claude cli directly Sep 15, 2025
@pelikhan
Copy link
Contributor

@copilot integrate support for generating the allow list of tools for the "playwright" based on the copilot agent supported tools.

See #823

…P tools

- Add GetCopilotAgentPlaywrightTools() function to shared agentic engine code
- Update Claude engine to expand playwright tools to include all 21 copilot agent tools
- Update Codex engine with same playwright tool expansion functionality
- Add comprehensive test coverage for playwright tool expansion
- Update neutral tools tests to verify expected playwright MCP configuration
- Ensure cross-engine compatibility for consistent playwright tools across all engines

Implements functionality from PR #823 to generate playwright allowlist based on copilot agent supported tools, enabling full browser automation capabilities in workflows using `playwright: null` configuration.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Sep 15, 2025

@copilot integrate support for generating the allow list of tools for the "playwright" based on the copilot agent supported tools.

See #823

Added support for generating playwright tool allowlist based on copilot agent supported tools. The implementation includes:

  • Shared function: GetCopilotAgentPlaywrightTools() returns all 21 copilot agent playwright tools
  • Claude engine: Updated to expand playwright: null into full MCP configuration with copilot agent tools
  • Codex engine: Added same functionality for consistent cross-engine support
  • Test coverage: Comprehensive tests verify all 21 tools are included when using playwright

Now when workflows specify playwright: null, they automatically get access to all browser automation capabilities available in the copilot agent environment.

Commit: 3f47e10

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.contoso.com
    • Triggering command: /tmp/go-build903066112/b278/cli.test -test.testlogfile=/tmp/go-build903066112/b278/testlog.txt -test.paniconexit0 -test.timeout=10m0s -test.v=true (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

run: |
set -o pipefail
# Execute Claude Code CLI with prompt from file
npx @anthropic-ai/claude-code@latest --print --mcp-config /tmp/mcp-config/mcp-servers.json --allowed-tools "ExitPlanMode,Glob,Grep,LS,NotebookRead,Read,Task,TodoWrite,mcp__github__download_workflow_run_artifact,mcp__github__get_code_scanning_alert,mcp__github__get_commit,mcp__github__get_dependabot_alert,mcp__github__get_discussion,mcp__github__get_discussion_comments,mcp__github__get_file_contents,mcp__github__get_issue,mcp__github__get_issue_comments,mcp__github__get_job_logs,mcp__github__get_me,mcp__github__get_notification_details,mcp__github__get_pull_request,mcp__github__get_pull_request_comments,mcp__github__get_pull_request_diff,mcp__github__get_pull_request_files,mcp__github__get_pull_request_reviews,mcp__github__get_pull_request_status,mcp__github__get_secret_scanning_alert,mcp__github__get_tag,mcp__github__get_workflow_run,mcp__github__get_workflow_run_logs,mcp__github__get_workflow_run_usage,mcp__github__list_branches,mcp__github__list_code_scanning_alerts,mcp__github__list_commits,mcp__github__list_dependabot_alerts,mcp__github__list_discussion_categories,mcp__github__list_discussions,mcp__github__list_issues,mcp__github__list_notifications,mcp__github__list_pull_requests,mcp__github__list_secret_scanning_alerts,mcp__github__list_tags,mcp__github__list_workflow_jobs,mcp__github__list_workflow_run_artifacts,mcp__github__list_workflow_runs,mcp__github__list_workflows,mcp__github__search_code,mcp__github__search_issues,mcp__github__search_orgs,mcp__github__search_pull_requests,mcp__github__search_repositories,mcp__github__search_users" --debug --verbose --permission-mode bypassPermissions --output-format json --settings /tmp/.claude/settings.json "$(cat /tmp/aw-prompts/prompt.txt)" 2>&1 | tee /tmp/test-ai-inference-github-models.log
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-code@latest seems brave as exposes us to Claude Code breaking changes. Maybe beta or latest official release ?

@pelikhan pelikhan merged commit c129112 into main Sep 15, 2025
15 of 17 checks passed
@pelikhan pelikhan deleted the copilot/fix-f0ab22fd-0c95-419f-b0d3-a61baae89f29 branch September 15, 2025 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants