Skip to content

Add cache control support for system prompts, messages, and tools#285

Open
CamilleScholtz wants to merge 1 commit intolaravel:0.xfrom
CamilleScholtz:feat/anthropic-system-prompt-cache-control
Open

Add cache control support for system prompts, messages, and tools#285
CamilleScholtz wants to merge 1 commit intolaravel:0.xfrom
CamilleScholtz:feat/anthropic-system-prompt-cache-control

Conversation

@CamilleScholtz
Copy link

@CamilleScholtz CamilleScholtz commented Mar 18, 2026

Summary

There's currently no way to configure cache control on system prompts, messages, or tools. Providers like Anthropic support prompt caching via cache breakpoints, and Prism already supports this via withProviderOptions() on messages and tools — but the Laravel AI SDK doesn't expose it.

This PR adds cache control support at three levels:

1. System prompt caching (via agent provider options)

class MyAgent implements Agent, HasProviderOptions
{
    public function providerOptions(Lab|string $provider): array
    {
        return match ($provider) {
            Lab::Anthropic => [
                'system_prompt_caching' => 'ephemeral',
                // or with TTL:
                // 'system_prompt_caching' => ['type' => 'ephemeral', 'ttl' => '5m'],
            ],
            default => [],
        };
    }
}

Anthropic supports 'ephemeral' as the cache type, with TTL values of '5m' (default) or '1h'.

2. Message-level cache control

Adds withProviderOptions() to the Message base class. Options flow through to Prism messages during conversion:

use Laravel\Ai\Messages\UserMessage;

$message = (new UserMessage('...'))->withProviderOptions([
    'cacheType' => 'ephemeral',
]);

Works on UserMessage, AssistantMessage, and ToolResultMessage.

3. Tool cache control

When a tool implements HasProviderOptions, its provider options are passed through to the Prism tool:

use Laravel\Ai\Contracts\HasProviderOptions;
use Laravel\Ai\Contracts\Tool;
use Laravel\Ai\Enums\Lab;

class SearchTool implements Tool, HasProviderOptions
{
    public function providerOptions(Lab|string $provider): array
    {
        return match ($provider) {
            Lab::Anthropic => ['cacheType' => 'ephemeral'],
            default => [],
        };
    }
}

Changes

  • PrismGateway: Extracted applySystemPrompt() that reads system_prompt_caching from agent provider options and wraps instructions in a SystemMessage with cache control. Updated addTools() to pass the provider through.
  • CreatesPrismTextRequests: Strips system_prompt_caching from agent provider options before passing to Prism (gateway-level concern, not a request-level Prism option).
  • AddsToolsToPrismRequests: createPrismTool() now checks if the tool implements HasProviderOptions and passes options to the Prism tool via withProviderOptions().
  • Message: Added $providerOptions property and withProviderOptions() method.
  • PrismMessages: Updated fromLaravelMessages() to transfer provider options from Laravel AI messages to Prism messages for all message types.

@CamilleScholtz CamilleScholtz force-pushed the feat/anthropic-system-prompt-cache-control branch from e321d05 to 85b6ecc Compare March 18, 2026 20:16
@CamilleScholtz CamilleScholtz changed the title Add Anthropic system prompt cache control Add system prompt cache control support via provider options Mar 18, 2026
Allow agents to opt-in to system prompt caching by specifying a
`system_prompt_caching` key in their provider options.

Add `withProviderOptions()` to Message for per-message cache control
that flows through to Prism messages during conversion.

Pass tool provider options through to Prism tools when tools implement
HasProviderOptions, enabling cache control on tool definitions.
@CamilleScholtz CamilleScholtz force-pushed the feat/anthropic-system-prompt-cache-control branch from 85b6ecc to d034ba8 Compare March 18, 2026 20:27
@CamilleScholtz CamilleScholtz changed the title Add system prompt cache control support via provider options Add cache control support for system prompts, messages, and tools Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant