Add cache control support for system prompts, messages, and tools#285
Open
CamilleScholtz wants to merge 1 commit intolaravel:0.xfrom
Open
Add cache control support for system prompts, messages, and tools#285CamilleScholtz wants to merge 1 commit intolaravel:0.xfrom
CamilleScholtz wants to merge 1 commit intolaravel:0.xfrom
Conversation
e321d05 to
85b6ecc
Compare
Allow agents to opt-in to system prompt caching by specifying a `system_prompt_caching` key in their provider options. Add `withProviderOptions()` to Message for per-message cache control that flows through to Prism messages during conversion. Pass tool provider options through to Prism tools when tools implement HasProviderOptions, enabling cache control on tool definitions.
85b6ecc to
d034ba8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
There's currently no way to configure cache control on system prompts, messages, or tools. Providers like Anthropic support prompt caching via cache breakpoints, and Prism already supports this via
withProviderOptions()on messages and tools — but the Laravel AI SDK doesn't expose it.This PR adds cache control support at three levels:
1. System prompt caching (via agent provider options)
Anthropic supports
'ephemeral'as the cache type, with TTL values of'5m'(default) or'1h'.2. Message-level cache control
Adds
withProviderOptions()to theMessagebase class. Options flow through to Prism messages during conversion:Works on
UserMessage,AssistantMessage, andToolResultMessage.3. Tool cache control
When a tool implements
HasProviderOptions, its provider options are passed through to the Prism tool:Changes
PrismGateway: ExtractedapplySystemPrompt()that readssystem_prompt_cachingfrom agent provider options and wraps instructions in aSystemMessagewith cache control. UpdatedaddTools()to pass the provider through.CreatesPrismTextRequests: Stripssystem_prompt_cachingfrom agent provider options before passing to Prism (gateway-level concern, not a request-level Prism option).AddsToolsToPrismRequests:createPrismTool()now checks if the tool implementsHasProviderOptionsand passes options to the Prism tool viawithProviderOptions().Message: Added$providerOptionsproperty andwithProviderOptions()method.PrismMessages: UpdatedfromLaravelMessages()to transfer provider options from Laravel AI messages to Prism messages for all message types.