Anthropic models configured through an `openrouter` provider (and, by
the same code path, `vercel` and generic `openai-compat` providers)
never got prompt caching. Two independent gaps: chatd only emitted
`cache_control` breakpoints when the fantasy client was `anthropic`, and
even when a client sent them, the AI Gateway chat-completions
interceptor rebuilt the upstream body from typed
`openai.ChatCompletionNewParams` and silently dropped the unknown key.
Dogfood UAT on main @ 77479408b4 measured 0 cached tokens on 27/27 turns
for `anthropic/claude-haiku-4.5` via OpenRouter while the direct
Anthropic provider cached 7k to 11k tokens per turn.
## Fix
**AI Gateway** (`aibridge/intercept/chatcompletions`):
`ChatCompletionNewParamsWrapper` records every message-level and
content-part-level `cache_control` from the raw request and re-applies
it at the same JSON path when the upstream body is rebuilt, before the
Google-specific patching. The blocking path now sends the raw body
whenever preserved fields exist; the streaming path already sent the
rebuilt body. Nothing is added unless the client sent it, and at most
four markers are recorded per request (Anthropic's breakpoint limit) so
the per-marker body rewrites stay bounded.
**chatd** (`coderd/x/chatd`): the caching gate becomes a small strategy
resolver keyed on the fantasy client and the model ID. `anthropic` keeps
the existing message-level breakpoints; `openrouter` and `vercel` reuse
them for Anthropic-family IDs since both clients translate them;
`openai-compat` gets content-part-level `ContentExtraFields` markers
(last text part of the last system message plus the final two
text-bearing user or assistant messages, tool messages skipped, at most
3 markers). `chatprovider.IsAnthropicFamilyModelID` recognizes
`anthropic/...`, Bedrock-style `anthropic.`/`.anthropic.` IDs, and bare
`claude` IDs. The `claude` substring match is deliberately broad; a
non-Anthropic model whose ID contains it would receive markers that
OpenAI-compatible upstreams ignore.
Docs: a short "Prompt caching" note in `docs/ai-coder/agents/models.md`.
`coderd/x/chatd/ARCHITECTURE.md` has no caching text to update.
## Dogfood UAT
Remote Coder Agents UAT on dogfood (chat:
https://dogfood.cdr.dev/agents/9f81cc40-add0-4b6f-b76c-f348b04762e4),
OpenRouter provider with real keys:
- Baseline (main @ `77479408b4`): 0 cached tokens on every OpenRouter
Haiku turn; direct Anthropic control cached normally; a hand-added
`cache_control` on the same key cached fine.
- Fix (`1d685f7e3e`, the first three commits):
`anthropic/claude-haiku-4.5` via OpenRouter now sends 2 markers on turn
1 and 3 afterwards (system last part plus the last two text-bearing
messages, never on tool messages). OpenRouter's `/generation` ground
truth reported `native_tokens_cached` 7137 to 7255 on turns 2+, equal to
Coder's `cache_read_tokens` and the AI Gateway's
`cache_read_input_tokens` on all 43 measured rows. Turn-2 prompt cost
dropped about 8x versus baseline. Agentic S3 cache reads grew from 7156
to 13993 as a 2500-line file entered the prompt. `openai/gpt-5-mini`,
`google/gemini-2.5-flash`, `deepseek/deepseek-chat-v3.1`, and
`moonshotai/kimi-k2.5` received 0 markers with unchanged request shapes;
the direct Anthropic control was unchanged. 76 upstream attempts, 0
non-200. The remote agent reported FAIL only because one acceptance
criterion I wrote asked the chat usage indicator to display cached
tokens, which the UI has never done for any provider; the runner
endorsed PASS for the fix itsel... [truncated]
- Smoke re-tests on `20ad259999b` (marker-code simplification) and on
`e31f1806748` (docs reflow plus the four-marker cap; the later head
`92b8f8ad0b8` only renames `google_openai_compat.go` to
`request_body.go` with no content change): S1 cache reads 0 / 7139 /
7161 / 7184 / 7207 equal to OpenRouter ground truth on every row,
markers 2 then 3 and never on tool messages, tool-heavy turn all 200, a
12 KB file read grew cache reads to 13548 with every request body
inspected for marker placement, `openai/gpt-5-mini` still 0 markers.
Known follow-ups surfaced by the UAT and out of scope here:
`provider_options.openrouter.*` never reach the wire on the gateway
path; OpenRouter cache writes are folded into `input_tokens` in chat
message usage (`cache_creation_tokens` stays 0) although the gateway
records them; the AI Gateway sessions UI has no cache columns and labels
every OpenAI-compatible provider "OpenAI"; attachment-only messages (no
text) fail upstream with HTTP 400 on OpenRouter (pre-existing,
reproduced without markers).
> Xum acted on behalf of @ibetitsmike for this pull request.