feat: integrate OpenZiti networking

.gitignore

@@ -3,6 +3,7 @@ node_modules
 .turbo
 node-compile-cache/
 .env
+.ziti/
 packages/platform-server/build-ts
 # Bundled artifacts generated during build
 packages/platform-server/dist

.npmrc

@@ -0,0 +1 @@
+allow-scripts=@openziti/ziti-sdk-nodejs

README.md

@@ -22,7 +22,8 @@ Intended use cases:
 - Operating a local development environment with supporting infra.
 
 ## Repository Structure
-- docker-compose.yml — Development infra: Postgres, agents-db, Vault (+ auto-init), NCPS, LiteLLM, cAdvisor, Prometheus, Grafana.
+- docker-compose.yml — Third-party development infra: Postgres, agents-db, Vault (+ auto-init), NCPS, LiteLLM, OpenZiti, Prometheus, Grafana, etc.
+- docker-compose.dev.yml — Overlay used for container image builds; dev mode runs platform-server and docker-runner on the host via pnpm.
 - .github/workflows/
   - ci.yml — Linting, tests (server/UI), Storybook build + smoke, type-check build steps.
   - docker-ghcr.yml — Build and publish platform-server and platform-ui images to GHCR.
@@ -117,7 +118,10 @@ pnpm install
 ```bash
 docker compose up -d
 # Starts postgres (5442), agents-db (5443), vault (8200), ncps (8501),
-# litellm (127.0.0.1:4000), docker-runner (7071)
+# litellm (127.0.0.1:4000), registry-mirror, and the OpenZiti controller stack
+# Optional monitoring (prometheus/grafana) lives in docker-compose.monitoring.yml.
+# Enable with: docker compose -f docker-compose.yml -f docker-compose.monitoring.yml up -d
+
 # Optional monitoring (prometheus/grafana) lives in docker-compose.monitoring.yml.
 # Enable with: docker compose -f docker-compose.yml -f docker-compose.monitoring.yml up -d
 ```
@@ -141,6 +145,8 @@ pnpm --filter @agyn/platform-ui dev
 # docker-runner (Fastify dev server)
 pnpm --filter @agyn/docker-runner dev
 ```
+> Supported dev mode runs platform-server and docker-runner via `pnpm dev` on the host.
+> Docker Compose is reserved for shared dependencies (Postgres, LiteLLM, Vault, OpenZiti, etc.).
 Server listens on PORT (default 3010; see packages/platform-server/src/index.ts and Dockerfile), UI dev server on default Vite port.
 
 The docker-runner dev script automatically loads the first `.env` it finds (prefers repo root, falls back to packages/docker-runner) when `NODE_ENV` is not `production`. Production `pnpm start` keeps relying solely on the surrounding environment, so missing `.env` files do not crash the process.
@@ -165,6 +171,47 @@ docker run --rm -p 8080:80 \
   ghcr.io/agynio/platform-ui:latest
 ```
 
+### Secure docker-runner connectivity (OpenZiti)
+
+The dev stack now ships an OpenZiti controller, initializer, and edge router. All docker-runner traffic flows through
+the overlay; there is no plain-HTTP fallback:
+
+1. Prepare the shared volumes once per checkout (`pnpm ziti:prepare`). This keeps `.ziti/controller`,
+   `.ziti/identities`, and `.ziti/tmp` writable even on SELinux hosts.
+2. Approve the OpenZiti SDK build step (`pnpm approve-builds` → select `@openziti/ziti-sdk-nodejs`).
+3. Copy `.env.example` to `.env` for both `packages/platform-server` and `packages/docker-runner`, keeping the
+   `ZITI_*` defaults that point to `./.ziti/identities/...` unless you have custom paths.
+4. Start the controller stack: `docker compose up -d ziti-controller ziti-edge-router`.
+   - Watch `docker compose logs -f ziti-edge-router` until you see the router enroll and connect to `ziti-controller`.
+   - For a clean bootstrap, stop the stack and wipe any stale state first:
+
+```bash
+docker compose down -v ziti-controller ziti-edge-router
+rm -rf ./.ziti/controller ./.ziti/identities ./.ziti/tmp
+```
+
+5. Bootstrap the controller state (service, policies, identities) via the bundled init job. Passing your UID/GID keeps
+   the emitted identity files readable without a manual `chmod`:
+
+```bash
+docker compose run --rm --user "$(id -u):$(id -g)" ziti-controller-init
+```
+
+The init container wraps the OpenZiti CLI, mirrors identity JSON into `./.ziti/identities`, and can be re-run
+whenever you need to regenerate enrollment material (no host `ziti` binary required). If the job reports that the
+router has not enrolled yet, keep the `ziti-edge-router` container running, wait for it to connect, then re-run the init job.
+
+6. Launch docker-runner and platform-server via host `pnpm dev` (the only supported dev path).
+   - Terminal A: `pnpm --filter @agyn/docker-runner dev` (reads `packages/docker-runner/.env`).
+   - Terminal B: `DOCKER_RUNNER_BASE_URL=http://127.0.0.1:17071 pnpm --filter @agyn/platform-server dev`.
+   ConfigService binds the local Ziti proxy to 127.0.0.1:17071 by default so the same URL works out of the box.
+
+The platform-server now retries the connectivity probe (defaults: 30 attempts, 2s interval). Override via
+`DOCKER_RUNNER_PROBE_MAX_ATTEMPTS` / `DOCKER_RUNNER_PROBE_INTERVAL_MS` if you need a longer window before the runner
+comes online.
+
+See [docs/containers/ziti.md](docs/containers/ziti.md) for the step-by-step host-mode workflow and smoke test commands.
+
 ## Configuration
 
 Key environment variables (server) from packages/platform-server/.env.example and src/core/services/config.service.ts:
@@ -185,9 +232,14 @@ Key environment variables (server) from packages/platform-server/.env.example an
 - Workspace/Docker:
   - WORKSPACE_NETWORK_NAME (default agents_net)
   - DOCKER_MIRROR_URL (default http://registry-mirror:5000)
-  - DOCKER_RUNNER_BASE_URL (required; default http://docker-runner:7071)
   - DOCKER_RUNNER_SHARED_SECRET (required HMAC credential)
   - DOCKER_RUNNER_TIMEOUT_MS (optional request timeout; default 30000)
+  - DOCKER_RUNNER_BASE_URL (default http://127.0.0.1:17071) — platform-server's local Ziti proxy endpoint
+- OpenZiti transport (required for runner connectivity):
+  - ZITI_MANAGEMENT_URL / ZITI_USERNAME / ZITI_PASSWORD — controller credentials
+  - ZITI_SERVICE_NAME / ZITI_ROUTER_NAME — service plus edge router handles
+  - ZITI_PLATFORM_IDENTITY_FILE / ZITI_RUNNER_IDENTITY_FILE — identity output paths under `./.ziti/`
+  - ZITI_RUNNER_PROXY_PORT (default 17071) — local HTTP proxy for docker-runner calls
 - Nix/NCPS:
   - NCPS_ENABLED (default false)
   - NCPS_URL_SERVER, NCPS_URL_CONTAINER (default http://ncps:8501)
@@ -219,10 +271,10 @@ UI variables (packages/platform-ui/.env.example):
   - vault — HashiCorp Vault (8200), auto-init helper vault-auto-init
   - ncps — Nix cache proxy (8501)
   - litellm + litellm-db — LLM proxy with UI (4000 loopback)
-  - docker-runner — authenticated Docker API proxy (7071, mounts /var/run/docker.sock)
   - Optional monitoring overlay (docker-compose.monitoring.yml) adds prometheus (9090) and grafana (3000) without mounting the Docker socket; provide your own scrape targets via configuration.
+- Platform services (platform-server, docker-runner) run via `pnpm dev` on the host. `docker-compose.dev.yml` remains for image builds but is not a supported dev path.
 
-To start services:
+To start shared dependencies (Postgres, LiteLLM, Vault, NCPS, OpenZiti, monitoring):
 ```bash
 docker compose up -d
 ```

docker-compose.dev.yml

@@ -0,0 +1,78 @@
+services:
+  platform-server:
+    build:
+      context: .
+      dockerfile: packages/platform-server/Dockerfile
+    restart: unless-stopped
+    depends_on:
+      postgres:
+        condition: service_healthy
+      agents-db:
+        condition: service_healthy
+      litellm:
+        condition: service_started
+      docker-runner:
+        condition: service_started
+    environment:
+      NODE_ENV: production
+      PORT: ${PLATFORM_SERVER_PORT:-3010}
+      AGENTS_DATABASE_URL: ${AGENTS_DATABASE_URL:-postgresql://agents:agents@agents-db:5432/agents}
+      LLM_PROVIDER: ${LLM_PROVIDER:-litellm}
+      LITELLM_BASE_URL: ${LITELLM_BASE_URL:-http://litellm:4000}
+      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY:-sk-dev-master-1234}
+      DOCKER_RUNNER_SHARED_SECRET: ${DOCKER_RUNNER_SHARED_SECRET:-dev-shared-secret}
+      WORKSPACE_NETWORK_NAME: ${WORKSPACE_NETWORK_NAME:-agents_net}
+      VAULT_ADDR: ${VAULT_ADDR:-http://vault:8200}
+      VAULT_TOKEN: ${VAULT_TOKEN:-dev-root}
+      ZITI_MANAGEMENT_URL: ${ZITI_MANAGEMENT_URL:-https://ziti-controller:1280/edge/management/v1}
+      ZITI_USERNAME: ${ZITI_USERNAME:-admin}
+      ZITI_PASSWORD: ${ZITI_PASSWORD:-admin}
+      ZITI_INSECURE_TLS: ${ZITI_INSECURE_TLS:-true}
+      ZITI_SERVICE_NAME: ${ZITI_SERVICE_NAME:-dev.agyn-platform.platform-api}
+      ZITI_ROUTER_NAME: ${ZITI_ROUTER_NAME:-dev-edge-router}
+      ZITI_RUNNER_PROXY_HOST: ${ZITI_RUNNER_PROXY_HOST:-0.0.0.0}
+      ZITI_RUNNER_PROXY_PORT: ${ZITI_RUNNER_PROXY_PORT:-17071}
+      ZITI_PLATFORM_IDENTITY_FILE: ${ZITI_PLATFORM_IDENTITY_FILE:-/opt/app/.ziti/identities/dev.agyn-platform.platform-server.json}
+      ZITI_RUNNER_IDENTITY_FILE: ${ZITI_RUNNER_IDENTITY_FILE:-/opt/app/.ziti/identities/dev.agyn-platform.docker-runner.json}
+      ZITI_IDENTITIES_DIR: ${ZITI_IDENTITIES_DIR:-/opt/app/.ziti/identities}
+      ZITI_TMP_DIR: ${ZITI_TMP_DIR:-/opt/app/.ziti/tmp}
+      GRAPH_REPO_PATH: ${GRAPH_REPO_PATH:-/opt/app/data/graph}
+    ports:
+      - "${PLATFORM_SERVER_PORT:-3010}:3010"
+      - "${ZITI_RUNNER_PROXY_PORT:-17071}:17071"
+    volumes:
+      - type: bind
+        source: ./.ziti
+        target: /opt/app/.ziti
+      - type: bind
+        source: ./data/graph
+        target: /opt/app/data/graph
+    networks:
+      - agents_net
+
+  docker-runner:
+    build:
+      context: .
+      dockerfile: packages/docker-runner/Dockerfile
+    user: root
+    restart: unless-stopped
+    depends_on:
+      ziti-edge-router:
+        condition: service_started
+    environment:
+      DOCKER_RUNNER_SHARED_SECRET: ${DOCKER_RUNNER_SHARED_SECRET:-dev-shared-secret}
+      DOCKER_RUNNER_PORT: ${DOCKER_RUNNER_PORT:-7071}
+      ZITI_IDENTITY_FILE: ${ZITI_RUNNER_IDENTITY_FILE:-/opt/app/.ziti/identities/dev.agyn-platform.docker-runner.json}
+      ZITI_SERVICE_NAME: ${ZITI_SERVICE_NAME:-dev.agyn-platform.platform-api}
+    volumes:
+      - type: bind
+        source: /var/run/docker.sock
+        target: /var/run/docker.sock
+      - type: bind
+        source: ./.ziti
+        target: /opt/app/.ziti
+        read_only: true
+    ports:
+      - "${DOCKER_RUNNER_PORT:-7071}:7071"
+    networks:
+      - agents_net

docker-compose.yml

@@ -36,7 +36,7 @@ services:
 
   # Dedicated Postgres for agents persistence
   agents-db:
-    image: postgres:16-alpine
+    image: ${POSTGRES_IMAGE:-public.ecr.aws/docker/library/postgres:16-alpine}
     oom_score_adj: -900
     container_name: agents-db
     restart: unless-stopped
@@ -73,6 +73,8 @@ services:
       interval: 10s
       timeout: 5s
       retries: 5
+    networks:
+      - agents_net
 
   vault:
     image: hashicorp/vault:1.17
@@ -247,23 +249,129 @@ services:
     networks:
       - agents_net
 
-  docker-runner:
-    build:
-      context: .
-      dockerfile: packages/docker-runner/Dockerfile
+  ziti-controller:
+    image: ${ZITI_IMAGE:-mirror.gcr.io/openziti/quickstart}:${ZITI_VERSION:-latest}
+    container_name: ziti-controller
+    hostname: ziti-controller
     restart: unless-stopped
+    oom_score_adj: -900
+    entrypoint:
+      - "/var/openziti/scripts/run-controller.sh"
     environment:
-      DOCKER_RUNNER_SHARED_SECRET: ${DOCKER_RUNNER_SHARED_SECRET:-dev-shared-secret}
-      DOCKER_RUNNER_PORT: ${DOCKER_RUNNER_PORT:-7071}
+      ZITI_NETWORK: ${ZITI_NETWORK:-dev.agyn-platform}
+      ZITI_CTRL_NAME: dev-controller
+      ZITI_CTRL_EDGE_ADVERTISED_ADDRESS: ziti-controller
+      ZITI_CTRL_EDGE_ADVERTISED_PORT: "1280"
+      ZITI_CTRL_EDGE_IP_OVERRIDE: 127.0.0.1
+      ZITI_CTRL_ADVERTISED_ADDRESS: ziti-controller
+      ZITI_CTRL_ADVERTISED_PORT: "6262"
+      ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION: 168h
+      ZITI_ROUTER_ENROLLMENT_DURATION: 168h
+      ZITI_USER: admin
+      ZITI_PWD: admin
+    ports:
+      - "127.0.0.1:1280:1280"
+      - "127.0.0.1:6262:6262"
+    healthcheck:
+      test:
+        [
+          "CMD-SHELL",
+          "curl -m 1 -s -k -f https://${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-controller}:${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}/edge/client/v1/version",
+        ]
+      interval: 5s
+      timeout: 5s
+      retries: 20
     volumes:
       - type: bind
-        source: /var/run/docker.sock
-        target: /var/run/docker.sock
+        source: ./.ziti/controller
+        target: /persistent
+        bind:
+          selinux: z
+    networks:
+      - agents_net
+
+  ziti-controller-init:
+    image: ${ZITI_IMAGE:-mirror.gcr.io/openziti/quickstart}:${ZITI_VERSION:-latest}
+    container_name: ziti-controller-init
+    restart: "no"
+    depends_on:
+      ziti-controller:
+        condition: service_healthy
+    entrypoint:
+      - "/bin/bash"
+      - "/scripts/ziti/controller-init.sh"
+    environment:
+      ZITI_NETWORK: ${ZITI_NETWORK:-dev.agyn-platform}
+      ZITI_CTRL_EDGE_ADVERTISED_ADDRESS: ziti-controller
+      ZITI_CTRL_EDGE_ADVERTISED_PORT: "1280"
+      ZITI_USER: admin
+      ZITI_PWD: admin
+      ZITI_SERVICE_NAME: dev.agyn-platform.platform-api
+      ZITI_PLATFORM_IDENTITY_NAME: dev.agyn-platform.platform-server
+      ZITI_RUNNER_IDENTITY_NAME: dev.agyn-platform.docker-runner
+      ZITI_ROUTER_NAME: dev-edge-router
+      ZITI_ENROLLMENT_DURATION_MINUTES: "1440"
+      ZITI_IDENTITIES_DIR: /identities
+      ZITI_IDENTITIES_TMP: /ziti-tmp
+      ZITI_PLATFORM_IDENTITY_FILE: /identities/dev.agyn-platform.platform-server.json
+      ZITI_RUNNER_IDENTITY_FILE: /identities/dev.agyn-platform.docker-runner.json
+    volumes:
+      - type: bind
+        source: ./.ziti/controller
+        target: /persistent
+        bind:
+          selinux: z
+      - type: bind
+        source: ./.ziti/identities
+        target: /identities
+        bind:
+          selinux: z
+      - type: bind
+        source: ./.ziti/tmp
+        target: /ziti-tmp
+        bind:
+          selinux: z
+      - type: bind
+        source: ./scripts/ziti
+        target: /scripts/ziti
+    networks:
+      - agents_net
+
+  ziti-edge-router:
+    image: ${ZITI_IMAGE:-mirror.gcr.io/openziti/quickstart}:${ZITI_VERSION:-latest}
+    container_name: ziti-edge-router
+    hostname: ziti-edge-router
+    restart: unless-stopped
+    depends_on:
+      ziti-controller:
+        condition: service_healthy
+    entrypoint: /bin/bash
+    command: "/var/openziti/scripts/run-router.sh edge"
+    environment:
+      ZITI_NETWORK: ${ZITI_NETWORK:-dev.agyn-platform}
+      ZITI_CTRL_ADVERTISED_ADDRESS: ziti-controller
+      ZITI_CTRL_ADVERTISED_PORT: "6262"
+      ZITI_CTRL_EDGE_ADVERTISED_ADDRESS: ziti-controller
+      ZITI_CTRL_EDGE_ADVERTISED_PORT: "1280"
+      ZITI_ROUTER_NAME: dev-edge-router
+      ZITI_ROUTER_ADVERTISED_ADDRESS: ziti-edge-router
+      ZITI_ROUTER_PORT: "3022"
+      ZITI_ROUTER_LISTENER_BIND_PORT: "10080"
+      ZITI_ROUTER_ROLES: router.platform
+      ZITI_USER: admin
+      ZITI_PWD: admin
     ports:
-      - "${DOCKER_RUNNER_PORT:-7071}:7071"
+      - "127.0.0.1:3022:3022"
+    volumes:
+      - type: bind
+        source: ./.ziti/controller
+        target: /persistent
+        bind:
+          selinux: z
     networks:
       - agents_net
 
+
 volumes:
   vault-file:
     driver: local