Skip to content

Conversation

@anurag27397
Copy link

…> system)

@anurag27397 anurag27397 changed the title sanitize chat payload roles for strict providers (map unknown roles -… Fix: sanitize outgoing chat roles for strict OpenAI-compatible providers Nov 18, 2025
@anurag27397
Copy link
Author

anurag27397 commented Nov 18, 2025

Summary

Some OpenAI-compatible providers (for example DeepSeek, Moonshot Kimi/K2 and similar strict wrappers) strictly validate the role field on chat messages and return a 400 error when they encounter non-standard role strings (e.g. "developer", "dev", "agent"). OpenAI itself is forgiving about non-standard roles, so tools that rely on OpenAI's lenient behavior may work there but fail with strict providers.

This patch normalizes outgoing chat message roles before the payload is serialized and sent to upstream providers. Unknown or custom roles are mapped to the system role (while preserving content). This prevents deserialization errors like:
400 Bad Request: {"error":{"message":"Failed to deserialize the JSON body into the target type: messages[1].role: unknown variant developer, expected one of system, user, assistant, tool"}}

What this change does

  • Adds a small sanitizer just before payload serialization that ensures every message role is one of:
    • system, user, assistant, tool
  • Non-standard roles (e.g. developer, dev, agent, or non-string values) are replaced with "system".
  • Keeps message content intact (so no loss of user intent).

Rationale

  • OpenAI’s API accepts/fault-tolerates non-standard roles, but many self-hosted or third-party OpenAI-compatible services use strict enum validation (often with Rust serde), which rejects unknown role variants.
  • Mapping unknown roles to system is conservative and consistent with the intended usage of such custom roles (developer/agent instructions are generally control/instructional text).
  • Ensures the CLI works across strict providers (DeepSeek, Kimi K2, Moonshot, Groq wrappers, etc.) without breaking behavior for OpenAI.

Tests & QA

  • Locally built and tested: the outgoing payload no longer includes "developer" or other unknown roles when the sanitizer runs.
  • Existing tests referencing chat payloads were run to confirm no regressions (see unit/integration tests).
  • Recommended: maintainers may want to add a unit test asserting that developersystem mapping is applied.

Backwards compatibility

  • OpenAI behavior is unchanged (OpenAI remains tolerant).
  • Consumers who relied on the presence of non-standard role strings inside the role field (rare) will see them normalized; however the original intent remains in content so the instruction semantics are preserved.

Implementation details

The sanitizer is inserted just before serde_json::to_string(&payload) / outgoing POST where the final JSON is produced, in order to cover all provider POSTs from the CLI.

Files changed:

  • code-rs/core/src/chat_completions.rs (primary)
  • (optional) code-rs/core/src/client.rs — same sanitizer pattern applied where payload_json is serialized.

If maintainers prefer a different fallback (e.g., mapping to user instead of system, or embedding the original role into the content for traceability), I can update the patch accordingly. Happy to open a follow-up PR with unit tests if desired.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant