feat: add passthrough target type for config routing by roh26it · Pull Request #1553 · Portkey-AI/gateway

roh26it · 2026-03-05T21:22:46Z

Summary

Adds passthrough: true option for targets that use the original request's provider/model instead of defining them in the config
Useful for conditional routing where unmatched requests should pass through unchanged
Works with fallback, loadbalance, and conditional strategies

Example Usage

{
  "strategy": { "mode": "conditional" },
  "targets": [
    {
      "name": "openai-target",
      "provider": "openai",
      "api_key": "...",
      "override_params": { "model": "gpt-4o" }
    },
    {
      "name": "default",
      "passthrough": true
    }
  ]
}

Changes

src/types/requestBody.ts: Added passthrough?: boolean to Targets interface
src/middlewares/requestValidator/schema/config.ts: Added schema validation for passthrough
src/handlers/handlerUtils.ts: Handle passthrough in tryTargetsRecursively

Test plan

Test passthrough target in conditional routing
Test passthrough target in fallback strategy
Verify original request provider/model are used when passthrough is true

Made with Cursor

Adds `passthrough: true` option for targets that use the original request's provider and model instead of defining them in the config. Useful for: - Conditional routing where unmatched requests pass through unchanged - Fallback strategies where the original request should be tried first Made-with: Cursor

Copilot

Pull request overview

Adds support for a passthrough: true target option so routing strategies (conditional/fallback/loadbalance) can forward an unmatched request using the original request’s provider/model instead of requiring provider configuration on that target.

Changes:

Adds passthrough?: boolean to the Targets type.
Extends Zod config validation to accept the passthrough flag.
Updates tryTargetsRecursively to merge “passthrough” provider options into the selected target before calling tryPost.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
`src/types/requestBody.ts`	Adds the `passthrough` field to `Targets` with explanatory JSDoc.
`src/middlewares/requestValidator/schema/config.ts`	Allows `passthrough` in schema + validation refine logic/message.
`src/handlers/handlerUtils.ts`	Implements passthrough behavior in `tryTargetsRecursively` and threads a cached config through recursion.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-05T21:36:11Z

src/handlers/handlerUtils.ts

+  const passthroughConfig =
+    inheritedConfig.passthroughConfig ??
+    constructConfigFromRequestHeaders(requestHeaders);


passthroughConfig is computed via constructConfigFromRequestHeaders(requestHeaders), but that helper prioritizes x-${POWERED_BY}-config and returns the parsed config object when present. In the common routing case (config header contains strategy/targets), this means passthroughConfig will not contain a provider/apiKey, so a leaf target with passthrough: true will call tryPost with provider undefined (leading to Providers[provider] access failing).

Consider building passthrough options from the base request headers (e.g., x-${POWERED_BY}-provider + authorization + provider-specific headers) while explicitly ignoring x-${POWERED_BY}-config, or store the original provider/apiKey separately before parsing the config header and reuse that here. Also add a clear runtime error if passthrough is requested but no base provider details are present.

Suggested change

const passthroughConfig =

inheritedConfig.passthroughConfig ??

constructConfigFromRequestHeaders(requestHeaders);

const headerConfig = constructConfigFromRequestHeaders(requestHeaders);

// Build passthrough config from base headers (provider/auth), ensuring that

// provider/apiKey are available even when x-${POWERED_BY}-config is present.

const baseProviderHeaderKey = HEADER_KEYS.PROVIDER;

const baseProvider =

requestHeaders[baseProviderHeaderKey] ??

requestHeaders[baseProviderHeaderKey.toLowerCase()];

const baseApiKeyHeaderKey =

(HEADER_KEYS as any).API_KEY ?? (HEADER_KEYS as any).APIKEY;

const baseApiKey =

(baseApiKeyHeaderKey &&

(requestHeaders[baseApiKeyHeaderKey] ??

requestHeaders[baseApiKeyHeaderKey.toLowerCase?.() as string])) ||

requestHeaders['authorization'] ||

requestHeaders['Authorization'];

const enrichedPassthroughConfig = {

...headerConfig,

// Do not override any explicit provider/apiKey that may already be present

provider: (headerConfig as any)?.provider ?? baseProvider,

apiKey: (headerConfig as any)?.apiKey ?? baseApiKey,

};

const passthroughConfig =

inheritedConfig.passthroughConfig ?? enrichedPassthroughConfig;

The passthrough feature is primarily intended for configs loaded via virtual keys or config IDs - not for configs passed directly in x-portkey-config header. In the typical flow:

Config comes from a virtual key (no x-portkey-config header present)

User provides x-portkey-provider and authorization headers separately

constructConfigFromRequestHeaders correctly returns provider/apiKey from these base headers

Having both a routing config in header AND separate provider headers is an unusual edge case that we're okay not supporting for now.

Copilot · 2026-03-05T21:36:11Z

src/handlers/handlerUtils.ts

+          targetToUse = {
+            ...currentTarget,
+            ...passthroughConfig,


When passthrough is enabled, the merge { ...currentTarget, ...passthroughConfig } will overwrite fields beyond provider/model. In particular, constructConfigFromRequestHeaders always includes defaultInputGuardrails/defaultOutputGuardrails (defaulting to [] when the headers are absent), which will wipe any inherited/configured default guardrails/hooks on the passthrough path.

To preserve routing-config guardrails by default, merge only the specific provider identity fields needed for passthrough (e.g., provider, apiKey/virtualKey, and provider-specific connection fields), or conditionally apply default*Guardrails only when the corresponding headers are actually present.

Suggested change

targetToUse = {

...currentTarget,

...passthroughConfig,

const {

provider,

model,

apiKey,

virtualKey,

baseURL,

} = (passthroughConfig || {}) as any;

targetToUse = {

...currentTarget,

...(provider !== undefined ? { provider } : {}),

...(model !== undefined ? { model } : {}),

...(apiKey !== undefined ? { apiKey } : {}),

...(virtualKey !== undefined ? { virtualKey } : {}),

...(baseURL !== undefined ? { baseURL } : {}),

Guardrails flow through currentInheritedConfig, not currentTarget. Our passthrough merge only builds targetToUse for the tryPost call - it doesn't touch currentInheritedConfig where guardrails are tracked. The inherited guardrails continue to flow correctly through the separate inheritance chain.

Copilot · 2026-03-05T21:36:12Z

src/types/requestBody.ts

+  /** When true, uses the provider and model from the original request headers/body instead of this target's config.
+   * Useful for conditional routing where unmatched requests should pass through unchanged,
+   * or for fallback strategies where the original request should be tried first. */


The new JSDoc says passthrough "uses the provider and model from the original request headers/body". In the current implementation, passthrough merges config derived from headers and still applies inherited overrideParams (including potentially model) unless the request itself supplied override params in x-*-config. That can result in the model being overridden even when passthrough: true.

Either adjust the documentation to describe the exact behavior (e.g., provider/auth/provider-specific connection fields only), or update the passthrough logic to ensure the request’s model is preserved (for example by not applying inherited overrideParams.model when passthrough is true).

Suggested change

/** When true, uses the provider and model from the original request headers/body instead of this target's config.

* Useful for conditional routing where unmatched requests should pass through unchanged,

* or for fallback strategies where the original request should be tried first. */

/** When true, uses provider/auth and other connection details derived from the original

* request headers/body instead of this target's provider configuration.

*

* The original request body (including its model and parameters) is forwarded, but any

* inherited `overrideParams` for this target (including `model`) may still be applied

* unless the request itself provides explicit override params via `x-*-config`.

*

* Useful for conditional routing where unmatched requests should mostly pass through

* unchanged, or for fallback strategies where the original request should be tried first. */

We'll leave the documentation as-is for now. The behavior of overrideParams inheritance is consistent with other target types, and passthrough primarily refers to using the original provider/auth rather than bypassing all config inheritance.

roh26it · 2026-03-06T10:50:40Z

Thanks for the review! After analysis, we believe these concerns don't apply to the typical use case:

Comment 1 (provider missing when config has targets):
The passthrough feature is primarily intended for configs loaded via virtual keys or config IDs - not for configs passed directly in x-portkey-config header. In the typical flow:

Config comes from a virtual key (no x-portkey-config header present)
User provides x-portkey-provider and authorization headers separately
constructConfigFromRequestHeaders correctly returns provider/apiKey from these base headers

Having both a routing config in header AND separate provider headers is an unusual edge case.

Comment 2 (guardrails being wiped):
Guardrails flow through currentInheritedConfig, not currentTarget. Our passthrough merge only builds targetToUse for the tryPost call - it doesn't touch currentInheritedConfig where guardrails are tracked. The inherited guardrails continue to flow correctly.

Comment 3 (documentation):
We'll leave the documentation as-is for now. The behavior of overrideParams inheritance is consistent with other target types.

roh26it changed the base branch from main to 2.0.0 March 5, 2026 21:24

roh26it requested review from VisargD, Copilot and narengogi and removed request for narengogi March 5, 2026 21:30

Copilot started reviewing on behalf of roh26it March 5, 2026 21:31 View session

Copilot AI reviewed Mar 5, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add passthrough target type for config routing#1553

feat: add passthrough target type for config routing#1553
roh26it wants to merge 1 commit into2.0.0from
feature/passthrough-target

roh26it commented Mar 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 5, 2026

Uh oh!

roh26it Mar 6, 2026

Uh oh!

Copilot AI Mar 5, 2026

Uh oh!

roh26it Mar 6, 2026

Uh oh!

Copilot AI Mar 5, 2026

Uh oh!

roh26it Mar 6, 2026

Uh oh!

roh26it commented Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

-  const passthroughConfig =
-    inheritedConfig.passthroughConfig ??
-    constructConfigFromRequestHeaders(requestHeaders);
+  const headerConfig = constructConfigFromRequestHeaders(requestHeaders);
+  // Build passthrough config from base headers (provider/auth), ensuring that
+  // provider/apiKey are available even when x-${POWERED_BY}-config is present.
+  const baseProviderHeaderKey = HEADER_KEYS.PROVIDER;
+  const baseProvider =
+    requestHeaders[baseProviderHeaderKey] ??
+    requestHeaders[baseProviderHeaderKey.toLowerCase()];
+  const baseApiKeyHeaderKey =
+    (HEADER_KEYS as any).API_KEY ?? (HEADER_KEYS as any).APIKEY;
+  const baseApiKey =
+    (baseApiKeyHeaderKey &&
+      (requestHeaders[baseApiKeyHeaderKey] ??
+        requestHeaders[baseApiKeyHeaderKey.toLowerCase?.() as string])) ||
+    requestHeaders['authorization'] ||
+    requestHeaders['Authorization'];
+  const enrichedPassthroughConfig = {
+    ...headerConfig,
+    // Do not override any explicit provider/apiKey that may already be present
+    provider: (headerConfig as any)?.provider ?? baseProvider,
+    apiKey: (headerConfig as any)?.apiKey ?? baseApiKey,
+  };
+  const passthroughConfig =
+    inheritedConfig.passthroughConfig ?? enrichedPassthroughConfig;

-          targetToUse = {
-            ...currentTarget,
-            ...passthroughConfig,
+          const {
+            provider,
+            model,
+            apiKey,
+            virtualKey,
+            baseURL,
+          } = (passthroughConfig || {}) as any;
+          targetToUse = {
+            ...currentTarget,
+            ...(provider !== undefined ? { provider } : {}),
+            ...(model !== undefined ? { model } : {}),
+            ...(apiKey !== undefined ? { apiKey } : {}),
+            ...(virtualKey !== undefined ? { virtualKey } : {}),
+            ...(baseURL !== undefined ? { baseURL } : {}),

-  /** When true, uses the provider and model from the original request headers/body instead of this target's config.
-   * Useful for conditional routing where unmatched requests should pass through unchanged,
-   * or for fallback strategies where the original request should be tried first. */
+  /** When true, uses provider/auth and other connection details derived from the original
+   * request headers/body instead of this target's provider configuration.
+   *
+   * The original request body (including its model and parameters) is forwarded, but any
+   * inherited `overrideParams` for this target (including `model`) may still be applied
+   * unless the request itself provides explicit override params via `x-*-config`.
+   *
+   * Useful for conditional routing where unmatched requests should mostly pass through
+   * unchanged, or for fallback strategies where the original request should be tried first. */

Conversation

roh26it commented Mar 5, 2026

Summary

Example Usage

Changes

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

roh26it Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

roh26it Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

roh26it Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

roh26it commented Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants