Feature/ingress

[bwalsh]

Feature Request: Authz-Aware Ingress Overlay for Multi-Tenant UIs and APIs

Summary

Add a small Helm overlay to helm/argo-stack that:

• Provides a single host, path-based ingress for:
  • /workflows → argo-stack-argo-workflows-server (2746)
  • /applications → argo-stack-argocd-server (8080)
  • /registrations → github-repo-registrations-eventsource-svc (12000)
  • /api → calypr-api (3000, namespace calypr-api)
  • /tenants → calypr-tenants (3001, namespace calypr-tenants)
• Ensures all endpoints are protected via the shared authz-adapter (NGINX external auth)
• Integrates with existing Let’s Encrypt / cert-manager and host config
• Is packaged as a drop-in overlay under helm/argo-stack/overlays/ingress-authz-overlay

[bwalsh]

@copilot

Read #91
Implement it here on this branch

[Copilot]

@bwalsh I've opened a new pull request, #93, to work on those changes. Once the pull request is ready, I'll request review from you.

[bwalsh]

@copilot

How is letsencrypt-prod-account-key and letsencrypt-staging-account-key set?
Document how we retrieve it from letsencrypt and how we set it.

[Copilot]

@bwalsh I've opened a new pull request, #95, to work on those changes. Once the pull request is ready, I'll request review from you.

[bwalsh]

@copilot

Getting error

"""
1125 02:40:39.668424 1 setup.go:276] "ACME server URL host and ACME private key registration host differ. Re-checking ACME account registration" logger="cert-manager.controller" resource_name="letsencrypt-staging" resource_namespace="" resource_kind="ClusterIssuer" resource_version="v1" related_resource_name="letsencrypt-staging-account-key" related_resource_namespace="cert-manager" related_resource_kind="Secret"
I1125 02:40:40.035363 1 setup.go:410] "verified existing registration with ACME server" logger="cert-manager.controller" resource_name="letsencrypt-staging" resource_namespace="" resource_kind="ClusterIssuer" resource_version="v1" related_resource_name="letsencrypt-staging-account-key" related_resource_namespace="cert-manager" related_resource_kind="Secret"
I1125 02:40:42.905362 1 sync.go:446] "certificate resource is not owned by this object. refusing to update non-owned certificate resource for object" logger="cert-manager.controller.ingress-shim" resource_name="ingress-authz-applications" resource_namespace="argo-stack" resource_kind="" resource_version="" related_resource_name="calypr-demo-tls" related_resource_namespace="argo-stack" related_resource_kind="Certificate" related_resource_version="v1"
I1125 02:40:42.909646 1 sync.go:446] "certificate resource is not owned by this object. refusing to update non-owned certificate resource for object" logger="cert-manager.controller.ingress-shim" resource_name="ingress-authz-workflows" resource_namespace="argo-stack" resource_kind="" resource_version="" related_resource_name="calypr-demo-tls" related_resource_namespace="argo-stack" related_resource_kind="Certificate" related_resource_version="v1"
"""

[Copilot]

@bwalsh I've opened a new pull request, #96, to work on those changes. Once the pull request is ready, I'll request review from you.

[bwalsh]

@copilot

Examine ingress configuration:

• namespace: security is running authz-adapter, image: argo-helm:latest this looks good
• namespace: argo-workflows is running authz-adapter, image: argo-helm:latest is erroring

Why are there two running?
Can we configure one, the one in namespace: security ?

[Copilot]

@bwalsh I've opened a new pull request, #97, to work on those changes. Once the pull request is ready, I'll request review from you.

[bwalsh]

@copilot
We are retrieving certificates, however, I can't connect to any of the paths
There are no obvious log entries.
How can I debug ?

"""
$ curl -vv https://calypr-demo.ddns.net/repositories
14:08:08.727173 [0-0] * Host calypr-demo.ddns.net:443 was resolved.
14:08:08.727321 [0-0] * IPv6: (none)
14:08:08.727420 [0-0] * IPv4: 100.22.124.96
14:08:08.727524 [0-0] * [HTTPS-CONNECT] added
14:08:08.727634 [0-0] * [HTTPS-CONNECT] connect, init
14:08:08.727868 [0-0] * [HTTPS-CONNECT] connect, check h21
14:08:08.727994 [0-0] * Trying 100.22.124.96:443...
14:08:08.728157 [0-0] * [HTTPS-CONNECT] connect -> 0, done=0
14:08:08.728257 [0-0] * [HTTPS-CONNECT] adjust_pollset -> 1 socks
14:08:08.728362 [0-0] * [HTTPS-CONNECT] connect, check h21
14:08:08.728463 [0-0] * [HTTPS-CONNECT] connect -> 0, done=0
14:08:08.728565 [0-0] * [HTTPS-CONNECT] adjust_pollset -> 1 socks
14:08:08.728666 [0-0] * [HTTPS-CONNECT] connect, check h21
14:08:08.728786 [0-0] * connect to 100.22.124.96 port 443 from 172.31.23.226 port 53896 failed: Connection refused
14:08:08.728853 [0-0] * Failed to connect to calypr-demo.ddns.net port 443 after 2 ms: Could not connect to server
14:08:08.728940 [0-0] * [HTTPS-CONNECT] connect, all failed
14:08:08.729037 [0-0] * [HTTPS-CONNECT] connect -> 7, done=0
14:08:08.729152 [0-0] * closing connection #0
14:08:08.729253 [0-0] * [HTTPS-CONNECT] close
14:08:08.729351 [0-0] * [SETUP] close
14:08:08.729447 [0-0] * [SETUP] destroy
14:08:08.729544 [0-0] * [HTTPS-CONNECT] destroy
curl: (7) Failed to connect to calypr-demo.ddns.net port 443 after 2 ms: Could not connect to server
"""

[Copilot]

@bwalsh I've opened a new pull request, #98, to work on those changes. Once the pull request is ready, I'll request review from you.

[bwalsh]

https://gh.io/copilot-coding-agent-docs
argo-workflows and argocd do not appear to be connected to the external ingress
"""
$ kubectl get ingress -A
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
argo-events github-repo-registrations-eventsource calypr-demo.ddns.net 100.22.124.96 80 39m
argo-stack ingress-authz-applications nginx calypr-demo.ddns.net 80, 443 38m
argo-stack ingress-authz-registrations nginx calypr-demo.ddns.net 80, 443 38m
argo-stack ingress-authz-workflows nginx calypr-demo.ddns.net 80, 443 38m
argo-workflows argo-workflows nginx argo.localtest.me 80 39m
argocd argocd-server nginx argocd.localtest.me 80 39m
calypr-api ingress-authz-api nginx calypr-demo.ddns.net 80, 443 38m
calypr-tenants ingress-authz-tenants nginx calypr-demo.ddns.net 80, 443 38m
"""

[bwalsh]

@copilot

github-repo-registrations-eventsource has no ingress CLASS assigned
All other ingress has no ADDRESS assigned

"""
$ kubectl get ingress -A
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
argo-events github-repo-registrations-eventsource calypr-demo.ddns.net 100.22.124.96 80 24m
argo-stack ingress-authz-applications nginx calypr-demo.ddns.net 80, 443 22m
argo-stack ingress-authz-registrations nginx calypr-demo.ddns.net 80, 443 22m
argo-stack ingress-authz-workflows nginx calypr-demo.ddns.net 80, 443 22m
argo-workflows argo-workflows nginx calypr-demo.ddns.net 80 24m
argocd argocd-server nginx calypr-demo.ddns.net 80 24m
calypr-api ingress-authz-api nginx calypr-demo.ddns.net 80, 443 22m
calypr-tenants ingress-authz-tenants nginx calypr-demo.ddns.net 80, 443 22m
"""

[Copilot]

@bwalsh I've opened a new pull request, #99, to work on those changes. Once the pull request is ready, I'll request review from you.

[bwalsh]

@copilot
How to configure external IP address for ingress type NodePort?

[Copilot]

@bwalsh I've opened a new pull request, #102, to work on those changes. Once the pull request is ready, I'll request review from you.

[bwalsh]

@copilot

"""
$ kubectl logs -n ingress-nginx -l app.kubernetes.io/component=controller --tail=50 | grep inClusterConfig.
W1126 01:14:35.311028 13 client_config.go:667] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
"""

[Copilot]

@bwalsh I've opened a new pull request, #103, to work on those changes. Once the pull request is ready, I'll request review from you.

[Copilot]

Pull request overview

This PR introduces a comprehensive ingress overlay system for the Argo Stack, providing unified, path-based routing with centralized authorization for multi-tenant UIs and APIs. The implementation includes extensive documentation, test specifications, and GitHub Copilot instruction files to guide future development.

Key Changes

• Ingress AuthZ Overlay: New Helm overlay chart providing single-host, path-based ingress for all services (/workflows, /applications, /registrations, /api, /tenants) with centralized auth via authz-adapter
• Cross-Namespace Routing: Support for routing ingress traffic across namespaces using ExternalName services
• Comprehensive Documentation: Added 1,800+ lines of troubleshooting guide covering ingress, TLS certificates, DNS, and environment-specific configurations (AWS EKS, bare metal)

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
helm/argo-stack/values.yaml	Added ingressAuthzOverlay configuration block with route definitions for all services
helm/argo-stack/overlays/ingress-authz-overlay/values.yaml	Complete overlay values with auth adapter config, TLS settings, and route specifications
helm/argo-stack/overlays/ingress-authz-overlay/templates/*.yaml	Helm templates for ingress resources, ExternalName services, and auth adapter deployment
helm/argo-stack/overlays/ingress-authz-overlay/docs/authz-ingress-user-guide.md	586-line user guide covering installation, cert-manager setup, and troubleshooting
docs/troubleshooting.md	Added 1,700+ lines of ingress troubleshooting content including connection issues, certificate management, and environment-specific configurations
Makefile	Updated ports target to deploy ingress overlay and configure NodePort for kind clusters
kind-config.yaml	Added port mappings for HTTP (30080→80) and HTTPS (30443→443)
.github/instructions/*.md	Added GitHub Copilot instruction files for Python, Helm/Kubernetes, Docker, and Bash
QUICKSTART.md, CONTRIBUTING.md	Restructured documentation removing redundant content
ORPHAN_ANALYSIS.md, IMPLEMENTATION_SUMMARY.md, FINAL_SUMMARY.md, CLEANUP_SUMMARY.md	Removed old analysis documents (cleanup from previous work)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The ingressAuthzOverlay.enabled is set to true by default in the main values.yaml. This could cause issues for users who haven't configured the overlay or don't want to use it. Consider setting this to false by default to make it opt-in, which is safer and follows the principle of least surprise.

Suggested change

	enabled: true
	enabled: false

[Copilot]

The host value calypr-demo.ddns.net appears to be an environment-specific value. This should either be left empty with a comment indicating it must be set, or use a placeholder like example.com. Having a specific environment value in the default configuration could lead to confusion or misconfigurations.

Suggested change

	host: calypr-demo.ddns.net
	# Set this to the external DNS host for your deployment environment.
	host: example.com

[Copilot]

There's commented-out code for cert-manager cluster-issuer annotation (lines 44-47). If this functionality is intentionally disabled, consider removing it entirely or documenting why it's commented out. If it's meant to be used, it should be uncommented and properly configured.

Suggested change

	# {{- if and $config.tls.enabled $route.primary }}
	# # Let's Encrypt / cert-manager integration (only on primary route to avoid ownership conflicts)
	# cert-manager.io/cluster-issuer: {{ $config.tls.clusterIssuer | quote }}
	# {{- end }}
	{{- if and $config.tls.enabled $route.primary }}
	# Let's Encrypt / cert-manager integration (only on primary route to avoid ownership conflicts)
	cert-manager.io/cluster-issuer: {{ $config.tls.clusterIssuer | quote }}
	{{- end }}

[Copilot]

The Makefile ports target contains hardcoded environment-specific values:

• Domain: calypr-demo.ddns.net (lines 163-164, 185)
• IP address: 100.22.124.96 (line 188)

These should be replaced with variables (e.g., ${ARGO_HOSTNAME} for the domain and a configurable ${EXTERNAL_IP} variable for the IP address). The hardcoded values will not work for users deploying to their own environments.

[bwalsh]

@copilot open a new pull request to apply changes based on this feedback

[Copilot]

The IP address 100.22.124.96 is hardcoded and appears to be environment-specific. While there's a comment saying "Adjust to your available IP range", this file is in the repository and will be deployed as-is by users who may not read the comment. Consider:

1. Using a more obviously placeholder range like 192.168.1.240-192.168.1.240
2. Adding a more prominent warning at the top of the file
3. Moving this to an example file rather than a directly deployable overlay

[Copilot]

@bwalsh I've opened a new pull request, #104, to work on those changes. Once the pull request is ready, I'll request review from you.

[bwalsh]

@copilot

Examine ingress all of these routes return a 404:

• /registrations
• /applications
• /workflows
• /api
• /tenants

[Copilot]

@bwalsh I've opened a new pull request, #105, to work on those changes. Once the pull request is ready, I'll request review from you.