Add Ingress connectivity troubleshooting and environment-specific configuration documentation

[Copilot]

User reported "Connection refused" on port 443 when accessing ingress endpoints. Added debugging documentation for ingress connectivity issues and environment-specific configuration guides.

Changes

• New troubleshooting section in docs/troubleshooting.md:

  • Connection refused (port 443) - NGINX Ingress Controller not accessible
  • LoadBalancer service pending/no external IP
  • 404/503 errors on ingress paths
  • authz-adapter external auth failures
  • Cloud-specific firewall checks (AWS/GCP/Azure)
  • Debugging cheat sheet with common kubectl commands

• Environment-Specific Ingress Configuration section:

  • AWS EKS: NLB/ALB setup, IAM permissions, subnet tagging requirements, security group configuration, troubleshooting pending LoadBalancer
  • On-Premises / Bare Metal: MetalLB setup (recommended), NodePort option, HostNetwork option, firewall configuration
  • Quick reference comparison table for choosing between environments

Quick Debugging

# Check ingress controller status
kubectl get pods -n ingress-nginx
kubectl get svc -n ingress-nginx  # Look for EXTERNAL-IP

# Verify ingress resources
kubectl get ingress -A
kubectl describe ingress ingress-authz-workflows -n argo-stack

# Check controller logs
kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx --tail=100

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[bwalsh]

@copilot
It doesn't look like nginx is running

"""
$ kubectl get pods -n ingress-nginx
No resources found in ingress-nginx namespace.
"""

[Copilot]

@copilot
It doesn't look like nginx is running

...

That confirms the issue - NGINX Ingress Controller is not installed. Install it with:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
  -n ingress-nginx --create-namespace \
  --set controller.service.type=LoadBalancer

Then wait for the LoadBalancer to get an external IP:

kubectl get svc -n ingress-nginx -w

Once EXTERNAL-IP is assigned, the ingress endpoints should be reachable.

[bwalsh]

@copilot
How long do I need to wait for EXTERNAL-IP?
Stuck in pending for several minutes

"""
$ kubectl get svc -n ingress-nginx -w
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.96.17.138 80:32097/TCP,443:31953/TCP 6m44s
ingress-nginx-controller-admission ClusterIP 10.96.142.77 443/TCP 6m44s
"""

[Copilot]

@copilot
How long do I need to wait for EXTERNAL-IP?
Stuck in pending for several minutes
...

A pending EXTERNAL-IP usually means the cloud provider can't provision a LoadBalancer. Debug with:

kubectl describe svc ingress-nginx-controller -n ingress-nginx

Look for events showing errors.

Common causes:

1. Local cluster (minikube/kind/k3d): Use NodePort or install MetalLB
2. AWS: Check IAM permissions for LoadBalancer creation
3. Quota exceeded: Check cloud provider console for LB quotas

Quick fix for local clusters:

# Use NodePort instead
helm upgrade ingress-nginx ingress-nginx/ingress-nginx \
  -n ingress-nginx \
  --set controller.service.type=NodePort

Then access via kubectl get svc -n ingress-nginx and use the NodePort (30000-32767 range).

For AWS EKS, ensure the AWS Load Balancer Controller is installed or your nodes have the right IAM role.

[bwalsh]

@copilot
I'm running on AWS
Add a section to the documentation that describes specific configuration and trouble shooting for AWS vs in house on-premises nodes

[Copilot]

@copilot
I'm running on AWS
Add a section to the documentation that describes specific configuration and trouble shooting for AWS vs in house on-premises nodes
...

Added comprehensive documentation for AWS EKS and on-premises ingress configuration in commit 74ef786.

New sections include:

AWS EKS:

• Installing NGINX Ingress with NLB/ALB
• Required IAM permissions
• Subnet tagging requirements
• Security group configuration
• Troubleshooting pending LoadBalancer

On-Premises:

• MetalLB setup (recommended)
• NodePort option (simple)
• HostNetwork option (direct access)
• Firewall configuration

See the new "Environment-Specific Ingress Configuration" section in docs/troubleshooting.md.

[bwalsh]

@copilot
argo-workflows and argocd do not appear to be connected to the external ingress
"""
$ kubectl get ingress -A
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
argo-events github-repo-registrations-eventsource calypr-demo.ddns.net 100.22.124.96 80 39m
argo-stack ingress-authz-applications nginx calypr-demo.ddns.net 80, 443 38m
argo-stack ingress-authz-registrations nginx calypr-demo.ddns.net 80, 443 38m
argo-stack ingress-authz-workflows nginx calypr-demo.ddns.net 80, 443 38m
argo-workflows argo-workflows nginx argo.localtest.me 80 39m
argocd argocd-server nginx argocd.localtest.me 80 39m
calypr-api ingress-authz-api nginx calypr-demo.ddns.net 80, 443 38m
calypr-tenants ingress-authz-tenants nginx calypr-demo.ddns.net 80, 443 38m
"""

[bwalsh]

https://gh.io/copilot-coding-agent-docs
argo-workflows and argocd do not appear to be connected to the external ingress
"""
$ kubectl get ingress -A
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
argo-events github-repo-registrations-eventsource calypr-demo.ddns.net 100.22.124.96 80 39m
argo-stack ingress-authz-applications nginx calypr-demo.ddns.net 80, 443 38m
argo-stack ingress-authz-registrations nginx calypr-demo.ddns.net 80, 443 38m
argo-stack ingress-authz-workflows nginx calypr-demo.ddns.net 80, 443 38m
argo-workflows argo-workflows nginx argo.localtest.me 80 39m
argocd argocd-server nginx argocd.localtest.me 80 39m
calypr-api ingress-authz-api nginx calypr-demo.ddns.net 80, 443 38m
calypr-tenants ingress-authz-tenants nginx calypr-demo.ddns.net 80, 443 38m
"""

[Copilot]

@copilot
argo-workflows and argocd do not appear to be connected to the external ingress
"""...

The argo-workflows and argocd ingresses need to be enabled in your Helm values. Check your values file for:

ingress:
  argoWorkflows:
    enabled: true
    host: "your-domain.com"  # Set to your actual host
    tls:
      enabled: true
      secretName: "your-tls-secret"
  argocd:
    enabled: true
    host: "argocd.your-domain.com"
    tls:
      enabled: true
      secretName: "your-tls-secret"

Then upgrade your Helm release:

helm upgrade argo-stack ./helm/argo-stack \
  -n argo-stack \
  --values your-values.yaml

Verify ingresses were created:

kubectl get ingress -n argo-workflows
kubectl get ingress -n argocd

Alternatively, use the ingress-authz-overlay for unified path-based routing (all services on one host):

helm upgrade --install ingress-authz-overlay \
  ./helm/argo-stack/overlays/ingress-authz-overlay \
  --set ingressAuthzOverlay.enabled=true \
  --set ingressAuthzOverlay.host=your-domain.com

This creates ingresses for /workflows, /applications, and other paths on a single host.

[Copilot]

https://gh.io/copilot-coding-agent-docs
argo-workflows and argocd do not appear to be connected to the external ingress
"""...

Your ingress setup is actually correct. You have two sets of ingresses:

1. ingress-authz-overlay (recommended) - These are working correctly:

• ingress-authz-workflows → Access via https://calypr-demo.ddns.net/workflows
• ingress-authz-applications → Access via https://calypr-demo.ddns.net/applications

2. Direct ingresses (default dev hosts) - These use localtest.me:

• argo-workflows → argo.localtest.me
• argocd-server → argocd.localtest.me

The localtest.me entries are the default development hosts. Since you're using the ingress-authz-overlay, you should access services via the unified calypr-demo.ddns.net host with path routing:

# Test your endpoints
curl -v https://calypr-demo.ddns.net/workflows
curl -v https://calypr-demo.ddns.net/applications

If you want to remove the duplicate localtest.me ingresses, update your values.yaml:

ingress:
  argoWorkflows:
    enabled: false  # Disable if using overlay
  argocd:
    enabled: false  # Disable if using overlay