gh-aw not working in cloud enterprise environments

[JoshGreenslade]

Relates to Issue #16312

Recently upgraded from v0.45.0 to v.0.50.4.

Been having trouble using this on cloud enterprise (i.e. MYCOMPANY.ghe.com) for a while, but had workarounds.
On v0.45.0, Had to manually add the following lines to the .lock,yml file under the Execute Github Copilot Cli step, in order to get copilot to use our cloud environment rather than github:

      - name: Execute GitHub Copilot CLI
        id: agentic_execution
        # Copilot CLI tool arguments (sorted):
        timeout-minutes: 20
        run: |
          set -o pipefail
          # shellcheck disable=SC1003
          sudo -E awf --env-all --container-workdir "${GITHUB_WORKSPACE}" --allow-domains "*.blob.core.windows.net,*.vsblob.vsassets.io,api.business.githubcopilot.com,api.enterprise.githubcopilot.com,api.github.com,api.githubcopilot.com,api.individual.githubcopilot.com,api.nuget.org,api.snapcraft.io,archive.ubuntu.com,azure.archive.ubuntu.com,azuresearch-usnc.nuget.org,azuresearch-ussc.nuget.org,builds.dotnet.microsoft.com,ci.dot.net,crl.geotrust.com,crl.globalsign.com,crl.identrust.com,crl.sectigo.com,crl.thawte.com,crl.usertrust.com,crl.verisign.com,crl3.digicert.com,crl4.digicert.com,crls.ssl.com,dc.services.visualstudio.com,dist.nuget.org,dot.net,dotnet.microsoft.com,dotnetcli.blob.core.windows.net,github.com,host.docker.internal,json-schema.org,json.schemastore.org,keyserver.ubuntu.com,nuget.org,nuget.pkg.github.com,nugetregistryv2prod.blob.core.windows.net,ocsp.digicert.com,ocsp.geotrust.com,ocsp.globalsign.com,ocsp.identrust.com,ocsp.sectigo.com,ocsp.ssl.com,ocsp.thawte.com,ocsp.usertrust.com,ocsp.verisign.com,oneocsp.microsoft.com,packagecloud.io,packages.cloud.google.com,packages.microsoft.com,pkgs.dev.azure.com,ppa.launchpad.net,raw.githubusercontent.com,registry.npmjs.org,s.symcb.com,s.symcd.com,security.ubuntu.com,telemetry.enterprise.githubcopilot.com,ts-crl.ws.symantec.com,ts-ocsp.ws.symantec.com,www.microsoft.com",OURCOMPANY.ghe.com,api.OURCOMPANY.ghe.com --log-level info --proxy-logs-dir /tmp/gh-aw/sandbox/firewall/logs --enable-host-access --image-tag 0.23.0 --skip-pull --enable-api-proxy \
            -- /bin/bash -c '/usr/local/bin/copilot --add-dir /tmp/gh-aw/ --log-level all --log-dir /tmp/gh-aw/sandbox/agent/logs/ --add-dir "${GITHUB_WORKSPACE}" --disable-builtin-mcps --allow-all-tools --allow-all-paths --prompt "$(cat /tmp/gh-aw/aw-prompts/prompt.txt)"${GH_AW_MODEL_AGENT_COPILOT:+ --model "$GH_AW_MODEL_AGENT_COPILOT"}' 2>&1 | tee -a /tmp/gh-aw/agent-stdio.log
        env:
          COPILOT_AGENT_RUNNER_TYPE: STANDALONE
          COPILOT_GITHUB_TOKEN: ${{ secrets.COPILOT_GITHUB_TOKEN }}
          GH_AW_MCP_CONFIG: /home/runner/.copilot/mcp-config.json
          GH_AW_MODEL_AGENT_COPILOT: ${{ vars.GH_AW_MODEL_AGENT_COPILOT || '' }}
          GH_AW_PROMPT: /tmp/gh-aw/aw-prompts/prompt.txt
          GH_AW_SAFE_OUTPUTS: ${{ env.GH_AW_SAFE_OUTPUTS }}
          GITHUB_HEAD_REF: ${{ github.head_ref }}
          GITHUB_MCP_SERVER_TOKEN: ${{ secrets.GH_AW_GITHUB_MCP_SERVER_TOKEN || secrets.GH_AW_GITHUB_TOKEN || secrets.GITHUB_TOKEN }}
          GITHUB_REF_NAME: ${{ github.ref_name }}
          GITHUB_STEP_SUMMARY: ${{ env.GITHUB_STEP_SUMMARY }}
          GITHUB_WORKSPACE: ${{ github.workspace }}
          GITHUB_SERVER_URL: https://OURCOMPANY.ghe.com
          GITHUB_API_URL: https://OURCOMPANY.ghe.com/api/v3
          GITHUB_ENTERPRISE_HOST: OURCOMPANY.ghe.com
          GITHUB_HOST: OURCOMPANY.ghe.com
          GH_HOST: OURCOMPANY.ghe.com
          XDG_CONFIG_HOME: /home/runner
      - name: Configure Git credentials
        env:

However, on the latest version this no longer works, and I get the following error on that step.

Error: Authentication failed

Your GitHub token may be invalid, expired, or lacking the required permissions.

To resolve this, try the following:
  • Start 'copilot' and run the '/login' command to re-authenticate
  • If using a Fine-Grained PAT, ensure it has the 'Copilot Requests' permission enabled
  • If using COPILOT_GITHUB_TOKEN, GH_TOKEN or GITHUB_TOKEN environment variable, verify the token is valid and not expired
  • Run 'gh auth status' to check your current authentication status

I have confirmed the PAT token is valid, and swapping back to 0.45.0 and adding my modifications works.

I believe the issue is probably how gh-aw is getting the server URL, and it failing in some instances when the url is not github.com/, but instead COMPANY.ghe.com/ - i.e. for enterprise cloud customers

[dsyme]

@JoshGreenslade Thanks for the report!

[dsyme]

/scout check what's going on here, see if you can spot the problem

[github-actions]

🔍 Scout Research Report

Triggered by @dsyme

Executive Summary

After analyzing the codebase, I've identified two likely root causes for the enterprise authentication failure in v0.50.4. The primary issue is that the Copilot engine's execution step does not generate enterprise GitHub environment variables (GITHUB_SERVER_URL, GITHUB_API_URL, etc.) in the compiled lockfile. A secondary issue is that the API proxy sidecar (--enable-api-proxy) may be routing Copilot authentication calls to the wrong endpoint for enterprise environments.

The fact that the same manual workaround that worked in v0.45.0 no longer works in v0.50.4 points to a behavioral regression — most likely in how the Copilot CLI (v0.0.417) handles enterprise authentication, possibly interacting with the API proxy.

Detailed Findings

Finding 1: Missing Enterprise Env Vars in Generated Lockfile

File: pkg/workflow/copilot_engine_execution.go (lines 250–258)

The Copilot engine GetExecutionSteps() generates a fixed env block that does not include enterprise-required environment variables:

env := map[string]string{
    "XDG_CONFIG_HOME":           "/home/runner",
    "COPILOT_AGENT_RUNNER_TYPE": "STANDALONE",
    "COPILOT_GITHUB_TOKEN":      copilotGitHubToken,
    "GITHUB_STEP_SUMMARY":       "${{ env.GITHUB_STEP_SUMMARY }}",
    "GITHUB_HEAD_REF":           "${{ github.head_ref }}",
    "GITHUB_REF_NAME":           "${{ github.ref_name }}",
    "GITHUB_WORKSPACE":          "${{ github.workspace }}",
}

Missing for enterprise compatibility:

• GITHUB_SERVER_URL: ${{ github.server_url }}
• GITHUB_API_URL: ${{ github.api_url }}

For public GitHub: github.server_url = https://github.com, github.api_url = https://api.github.com
For enterprise: github.server_url = (company.ghe.com/redacted), `github.api_url = (company.ghe.com/redacted)

Including these as GitHub Actions expressions (not hardcoded values) would make the lockfile work for both environments automatically, without any manual modifications.

Contrast with other engines: codex_engine.go:473 explicitly includes GITHUB_SERVER_URL in its env vars. The Copilot engine lacks this.

Finding 2: FilterEnvForSecrets May Block User-Added Secret References

File: pkg/workflow/engine_helpers.go (lines 270–300)

A security filter is applied at compilation time:

allowedSecrets := e.GetRequiredSecretNames(workflowData)
filteredEnv := FilterEnvForSecrets(env, allowedSecrets)

GetRequiredSecretNames for Copilot only allows: COPILOT_GITHUB_TOKEN, MCP_GATEWAY_API_KEY, GITHUB_MCP_SERVER_TOKEN, HTTP MCP header secrets, and safe-inputs secrets.

Any user-added env var containing ${{ secrets.SOMETHING }} (where SOMETHING is not in the allowed list) would be silently removed during lockfile compilation.

However, plain string values like GITHUB_SERVER_URL: (company.ghe.com/redacted) or ${{ github.server_url }} are NOT filtered. So this only matters if users add new secret references.

Finding 3: API Proxy May Not Handle Enterprise Copilot Endpoints

Files: pkg/workflow/copilot_engine.go (line 48), pkg/workflow/awf_helpers.go (lines 194–197)

supportsLLMGateway: true  // Always enables API proxy
// ...
if config.UsesAPIProxy {
    awfArgs = append(awfArgs, "--enable-api-proxy")
}

The Copilot engine always enables the API proxy (--enable-api-proxy) because CopilotLLMGatewayPort = 10002 > 0. This starts a sidecar inside AWF that intercepts Copilot LLM API calls.

For enterprise Copilot users (COMPANY.ghe.com), the Copilot API endpoint is api.COMPANY.ghe.com or api.enterprise.githubcopilot.com. If the API proxy only routes to the public Copilot API (api.github.com or api.githubcopilot.com), authentication would fail for enterprise tokens.

The user's v0.45.0 config also included --enable-api-proxy, so this alone shouldn't be the regression. However, the proxy behavior or configuration may have changed between versions.

Finding 4: CopilotDefaultDomains Missing Enterprise Domains

File: pkg/workflow/domains.go (lines 24–35)

var CopilotDefaultDomains = []string{
    "api.business.githubcopilot.com",
    "api.enterprise.githubcopilot.com",
    "api.github.com",
    "api.githubcopilot.com",
    "api.individual.githubcopilot.com",
    "github.com",
    // ... no *.ghe.com domains
}

Enterprise GitHub domains (COMPANY.ghe.com, api.COMPANY.ghe.com) are not in the default allowed domains. This was already known from issue #16312 (the related issue). The user has to add these manually via network configuration or directly in the lockfile.

Finding 5: GitHub MCP Server Missing Enterprise Config

Visible in: smoke-copilot.golden (lines 485–494)

The generated GitHub MCP server configuration:

"github": {
    "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_MCP_SERVER_TOKEN}",
        "GITHUB_READ_ONLY": "1",
        "GITHUB_TOOLSETS": "..."
    }
}

Missing: GITHUB_HOST, GH_HOST, or GITHUB_API_URL for the GitHub MCP server container. This would cause the MCP server to point to api.github.com instead of the enterprise endpoint.

Recommendations

1. Add GITHUB_SERVER_URL and GITHUB_API_URL to the Copilot execution step — use ${{ github.server_url }} and ${{ github.api_url }} expressions so they resolve correctly at both compile time and runtime:

   • GITHUB_SERVER_URL: ${{ github.server_url }}
   • GITHUB_API_URL: ${{ github.api_url }}

2. Pass enterprise config to GitHub MCP server — inject GITHUB_HOST or GH_HOST into the GitHub MCP server container configuration when an enterprise host is detected.

3. Investigate API proxy behavior for enterprise — determine whether --enable-api-proxy (the LLM gateway sidecar at port 10002) correctly routes Copilot authentication calls when GITHUB_SERVER_URL is set to an enterprise host. This may be the regression between v0.45.0 and v0.50.4.

4. Consider adding a github-enterprise-host option — allow users to declare their enterprise host in workflow frontmatter (e.g., github-enterprise-host: COMPANY.ghe.com) to auto-configure all the necessary domains and env vars.

Suggested Next Steps

1. Check if the Copilot CLI version bumped between v0.45.0 and v0.50.4 changed authentication handling for enterprise (GH_HOST / GITHUB_HOST resolution). The current default version is 0.0.417 (pkg/constants/constants.go:329).

2. Test whether adding GITHUB_SERVER_URL: ${{ github.server_url }} and GITHUB_API_URL: ${{ github.api_url }} to the Copilot execution step env block fixes the issue in v0.50.4.

3. Verify whether --enable-api-proxy routes auth validation calls correctly when GITHUB_SERVER_URL points to an enterprise host.

4. Add enterprise GitHub support as a documented network configuration option rather than requiring manual lockfile edits.

Key Source Files

• pkg/workflow/copilot_engine_execution.go — env block generation (lines 250–258)
• pkg/workflow/copilot_engine.go — engine feature flags (line 48, supportsLLMGateway: true)
• pkg/workflow/domains.go — CopilotDefaultDomains (lines 24–35)
• pkg/workflow/engine_helpers.go — FilterEnvForSecrets (lines 270–300)
• pkg/workflow/awf_helpers.go — BuildAWFArgs (--enable-api-proxy logic)
• pkg/constants/constants.go — DefaultCopilotVersion = "0.0.417" (line 329)

References:

• §22442592260

🔭 Intelligence gathered by Scout

[kbreit-insight]

I was about to post about this same problem. I never got it working, regardless of the changes I made. I'm happy to provide troubleshooting output if needed.

[pelikhan]

#18504

[lpcox]

@JoshGreenslade thanks for this. we are working on a fix.