Add proposal for dynamic webhook middleware

[JAORMX]

Summary

Introduces a dynamic webhook middleware system for ToolHive that enables external services to participate in the MCP request pipeline, inspired by Kubernetes admission webhooks.

Problem

Organizations need to integrate ToolHive with existing enterprise systems (policy engines, SIEM, ticketing, HR, CMDB) but currently require forking ToolHive and maintaining custom builds to add new middleware functionality.

Solution

Two new webhook types:

• Validating webhooks: Accept/deny requests based on external policy decisions
• Mutating webhooks: Transform requests (enrich with context, redact PII, etc.)

Key Features

• ✅ Multiple webhooks with sequential execution (pipeline pattern)
• ✅ File-based CLI configuration (--webhook-config webhook.yaml)
• ✅ Kubernetes MCPWebhookConfig CRD for operator
• ✅ TLS/mTLS support with file-based certificates
• ✅ Configurable failure policies (fail-open/fail-closed)
• ✅ Full observability (Prometheus metrics, audit logs, OpenTelemetry tracing)
• ✅ Versioned protocol (version: v0.1.0) for evolution

Use Cases

1. External Policy Engine Integration - Use OPA or custom policy engines
2. Request Enrichment - Add context from HR/CMDB systems
3. Approval Workflows - Require manager approval for high-risk operations
4. SIEM Integration - Send custom-formatted logs to enterprise SIEM
5. Rate Limiting - Enforce limits using external rate limiters
6. PII Redaction - Remove sensitive data before processing

Architecture

Webhooks integrate into existing middleware chain:

1. Authentication → 2. Token Exchange → 3. MCP Parser → 4. Tool Filter → 5. Mutating Webhooks → 6. Validating Webhooks → 7. Telemetry → 8. Authorization → 9. Audit

Multiple webhooks execute sequentially in configuration order.

Configuration Examples

CLI:

thv run postgres-mcp \
  --webhook-config policy-webhook.yaml \
  --webhook-config enrichment-webhook.yaml

Kubernetes:

apiVersion: toolhive.stacklok.dev/v1alpha1
kind: MCPWebhookConfig
metadata:
  name: company-policy-webhook
spec:
  type: validating
  url: https://policy.company.com/validate
  failure_policy: fail

Implementation Plan

• Phase 1: Core HTTP client, CLI integration
• Phase 2: Kubernetes operator and CRD
• Phase 3: Advanced features (circuit breaker, caching)
• Phase 4: Ecosystem (examples, SDKs, integration guides)

Documentation

Full proposal: docs/proposals/THV-0000-dynamic-webhook-middleware.md

Will update filename with issue number once PR is created.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 53.61%. Comparing base (5948453) to head (e8a46d4).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2300   +/-   ##
=======================================
  Coverage   53.61%   53.61%           
=======================================
  Files         239      239           
  Lines       30704    30704           
=======================================
  Hits        16463    16463           
  Misses      13071    13071           
  Partials     1170     1170           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.