Commit Graph
98 Commits
Author SHA1 Message Date
Ethan 5cbf086517 feat: send tool result media to providers like user files (#31383)
Part of [CODAGT-1430](https://linear.app/codercom/issue/CODAGT-1430).
This is the first of four stacked PRs.

Today a PDF in an MCP tool result reaches no provider. On Anthropic we
strip it in chatd. On OpenAI Chat Completions fantasy swaps it for a
"cannot be displayed" note, and on Responses it sends a placeholder that
points at a user message it never sends.

coder/fantasy#73 and coder/fantasy#74 make fantasy convert tool result
media with the converter it already uses for user files: an Anthropic
document inside `tool_result`, an `input_file` inside
`function_call_output` on Responses, and a user file part straight after
the tool message on Chat Completions. Both have merged, and this PR pins
`charm.land/fantasy` to the coder_2_33 tip that has them. Chat
Completions file parts now refuse text files, as Responses already did,
so `AcceptsFilePartMediaType` follows.

The converters take a filename, so an MCP embedded resource blob is now
named after the last element of its URI path, and we store the name with
the tool result so later turns send the same one. OpenAI picks the file
type from the extension, so we append the right one when it's missing or
wrong. We drop names with control characters, as Postgres `jsonb`
rejects NUL, and we redact the name like the rest of the tool response,
since a URI like `https://host/sse/<token>` would otherwise put the
token in the chat history.

There's also a test proving aibridge's Chat Completions re-encode keeps
a user file part, as that's where these PDFs end up.

Anthropic still strips tool result PDFs until the next PR in the stack.
2026-11-11 00:06:40 +11:00
Michael SuchaczandEthan Dickson 082668fbc1 feat: run computer use on OpenAI-compatible AI providers (#31895)
Computer use only ran on Anthropic and OpenAI AI providers. Chatd
derives the tool flavor from the AI provider type, and `openai-compat`
had no flavor, so every self-hosted model was marked unavailable for
`type=computer_use`, including the model a single-provider deployment
already uses for chat.

Hosted computer tools are not an option on those servers: an
OpenAI-compatible endpoint accepts a `computer_20250124` tool over
`/v1/messages` and a `computer-use-preview` tool over `/v1/responses`
and then drops both, so the model never sees a way to act. Computer use
is now declared as an ordinary function tool for those providers. The
model sends one action per call and every call answers with a fresh
screenshot, which is the shape these models already handle through the
shared action runner. The tool omits `zoom`, which the agent desktop
cannot perform. The desktop clamps a point outside the screen to its
edge, so the tool rejects a coordinate outside the declared screenshot
with an error that states the valid range, and a model that aims off the
screen learns that it missed. Media tool results reach the model because
the pinned fantasy fork promotes them to user-role messages, so no
dependency changed.

Coder can't verify an arbitrary OpenAI-compatible model, so those models
keep the unverified warning icon in the admin UI. The virtual desktop
docs, the Coder Agents index, the chatd architecture doc, and the
organization model settings copy now list OpenAI-compatible providers as
supported, and the docs say what to check before using one.

Three shared behaviors needed fixing to make this usable. A coordinate
of `[0,0]` is the top-left pixel and decoded as an omitted field, so the
desktop rejected those clicks with a missing-coordinate error;
coordinates now decode by key presence, which also keeps an absent
coordinate absent. A `scroll_amount` of zero or less, or a key `repeat`
of zero, was also forwarded as omitted, so the desktop scrolled its
default of three clicks or pressed the key once; the runner now rejects
both, and the function tool declares a minimum of 1 for each. And
because a chat completions model sends one action per call, it plans
every call of a step against one screen, so a failed action now halts
the later calls of that step, as it already did for Anthropic. The
hosted OpenAI flavor still differs here on purpose: it delivers an array
of actions in one call. Halting would strand a mouse press whose release
was planned for a later call, so the function tool offers
`left_click_drag` instead of separate press and release actions, and
rejects any action it does not declare.

Provider types that reuse the chat completions client without being
OpenAI-compatible (Azure, Google, OpenRouter, Vercel, Copilot) stay
unsupported until each is verified.

> [!NOTE]
> `coderd/apidoc/docs.go`, `coderd/apidoc/swagger.json`,
`docs/reference/api/schemas.md` and `site/src/api/typesGenerated.ts` are
hand-edited to match the new enum value, because the environment that
prepared this change had no pnpm to re-run the generators. The same
environment had no node, so `pnpm run typecheck`, biome and prettier did
not run locally; CI is the validator for the `site/` change and the
generated files.

> [!NOTE]
> Prepared by Xum, Mike's coding agent, from the measurement that showed
an OpenAI-compatible server dropping both hosted computer tools.

---------

Co-authored-by: Ethan Dickson <ethanndickson@gmail.com>
2026-11-09 17:26:10 +11:00
Michael SuchaczandEthan Dickson b6dc7158b4 feat: use the Anthropic computer use toolset for Claude 4.8 and later (#31521)
> Xum acted for Mike: this PR was written and opened by an AI agent on
Mike's behalf.

Computer use subagents on Claude 5.5 models (`claude-opus-5-5`,
`claude-sonnet-5-5`) failed on their first request: these models reject
`computer_20251124` and accept only the GA toolset
`computer_toolset_20260801`.

chatd now picks the Anthropic computer use tool version from the chat's
model with `ComputerUseToolVersionForModel` from coder/fantasy: the
toolset for Claude 4.8 and later, `computer_20251124` for Opus 4.5 to
4.7, Sonnet 4.6 and model IDs it cannot parse, and `computer_20250124`
for Haiku and Sonnet 4.5. Claude Platform on AWS and Amazon Bedrock
offer computer use only through the earlier versions, so an Anthropic
provider configured for either declares `computer_20251124` where the
model would take the toolset. For the toolset, chatd answers like
Anthropic's SDK tool runner: actions answer with text and only
`screenshot` returns an image, `cursor_position` returns the position as
text, `key` honors `repeat` (1 to 100), and zoom is declared disabled
because the agent desktop has no zoom action. The earlier versions still
answer every action with a screenshot.

A step's computer calls run one at a time in call order. For every
Anthropic version, the calls after one that failed, or that was rejected
before it ran (malformed input or a pre_tool_use hook denial), return
`Not executed: an earlier computer action in this turn failed.` without
running, which is the SDK runner's text. The chat loop enforces this
halt, so the computer tool keeps no state between calls. `wait` and
`hold_key` durations are read as seconds (capped at 300).

The frontend marks Claude Opus 5.5 and Sonnet 5.5 as verified for
computer use, and the computer tool row now labels each action ("Left
click", "Type", "Press key") instead of "Screenshot", shows text output,
and shows the error of failed or halted calls inline. The error block is
shared with the write_file and edit_files rows as
`ToolCall.ErrorDetail`.

Uses coder/fantasy#80 and coder/fantasy#82, both merged. `go.mod` pins
the fork to `7882f906a89a`.

<details>
<summary>UAT</summary>

TL;DR: it works. A remote Coder Agents UAT on dogfood passed on
`00143d5b12`, with all model traffic logged through a proxy:

- `claude-opus-5-5`, `claude-sonnet-5-5` and `claude-opus-4-8` declared
`computer_toolset_20260801` with zoom disabled and no beta header.
`claude-opus-4-6` and an unknown model ID declared `computer_20251124`
with the `computer-use-2025-11-24` beta, `claude-haiku-4-5` declared
`computer_20250124`, and a Claude Platform on AWS provider declared
`computer_20251124`. Opus 5.5, Sonnet 5.5, Opus 4.6 and Haiku 4.5 each
completed a desktop task.
- Toolset actions answered with text and only `screenshot` returned an
image. `repeat` 100 ran, 150 failed, and `wait` 400 stopped at 300
seconds.
- A failed call (`repeat` 150 on the toolset, a desktop error on 4.6)
and a pre_tool_use hook denial each halted the rest of the step without
touching the desktop. The next step ran normally.
- The computer tool rows showed action labels, text output and the halt
error. Opus 5.5 and Sonnet 5.5 showed no warning in the computer use
model selector. Switching a chat between 4.6 and 5.5 worked in both
directions.

Not tested: the tool type on Bedrock, and a halt after malformed model
input. Known, unchanged behavior: the desktop reports success for
unknown key names and off-screen coordinates, the model can still emit
`zoom`, which fails, and `scroll` up fails in `agent/x/agentdesktop`
because the negative amount is read as a flag. Both failures halt the
rest of their batch.

</details>

---------

Co-authored-by: Ethan Dickson <ethanndickson@gmail.com>
2026-11-02 21:28:22 +11:00
Mathias Fredriksson c0f7663e95 feat: support editing queued chat messages (#29215)
The owner of a chat can edit a queued message through
`PATCH /api/v2/chats/{chat}/queue/{queuedMessage}`. chatd never sends
a row under edit (`editing_since`): rows ahead of it still send in
order, and a turn that ends at it puts the chat in the new `paused`
status. Ending the head's edit, deleting it, or promoting a queued row
continues the chat. Interrupting sends still run ahead of the queue.

The chat state machine classifies a queue whose head is under edit as
its own sub-state (R1P, I1P, A1P, E1P, XE1P, P), so every path to
`paused` is in the transition matrix. A chat whose family has a paused
member cannot be archived, and auto-archive skips that family. Queue
edit and delete are owner-only, because both can send the next message
with the owner's credentials.

`ChatQueuedMessage` gains `editing_since` and `reasoning_effort`, and
`ChatStatus` gains `paused`.

Closes CODAGT-1028
Refs CODAGT-971
2026-10-30 15:52:22 +02:00
Michael Suchacz 78fa9a8613 refactor(coderd/x/chatd)!: retire the close_agent tool alias (#31005)
This PR breaks compatibility with chat histories that contain
`close_agent` tool calls, which chats from v2.34 and earlier have.
#26673 (v2.35) renamed `close_agent` to `interrupt_agent` and kept a
hidden alias, so a model that copied the old name from such a history
still interrupted the child. This PR deletes the alias:
`subagentToolNameAliases`, `ExecuteLocalToolsOptions.ToolNameAliases`
with `(*toolBatch).resolve` and its five call sites in `chatloop`, the
alias step in builtin input validation, and the `close_agent` entries in
the unbilled tool set, the plan policy, and the chatprompt lifecycle
list. Coder Agents is experimental (`coderd/x`), and our rule for
experimental persisted data is to drop the legacy shape outright and
delete the code that existed only to support it, instead of carrying it
forward.

What happens to an old chat:

- Replay does not change, because the replay path does not look up tool
names: the model still sees the historical `close_agent` call and its
stored result, and the UI renders them as before. The write path reads a
tool name only to parse `execute` commands and to keep structured error
results from subagent lifecycle tools. Dropping `close_agent` from that
lifecycle list changes nothing, because its calls are now rejected with
a plain-text error.
- If the model calls `close_agent` again, nothing runs and the child
keeps running. The tool result is the error `Tool not active in this
turn: close_agent`, and the model can retry with `interrupt_agent`.

**Why it is safe**

- Deleted alias-only tests:
`TestExecuteSingleTool_ResolvesToolNameAlias`, the
`AliasNamesClassifyAsCalled` billing row, the alias subtest of
`TestPartitionAmbiguousToolCallsGatesOnBuiltins`, the alias loop in
`TestUnbilledSubagentToolNamesMatchCatalog`, and the aliased calls that
#30939 added to `TestExecuteToolsSerialToolCallOrder` and
`TestExecuteLocalTools_ProviderToolRunner`; both tests are back to their
#30938 form.
- `TestExecuteSingleTool_UnknownAliasFallsThrough` is now
`TestExecuteSingleTool_RejectsUnknownTool` and asserts the exact error
above. The observer tests call the tool by its real name.
- Mutants of the active tool check, step observer notification, serial
dispatch, the unbilled set, the plan policy and builtin input validation
each fail a remaining test. Only the deleted alias test caught an empty
name passed to the tool handler; no chatd tool reads that name (MCP
tools use their original name).
- `git grep` finds no use of the removed identifiers and no backend code
that resolves `close_agent`. The rejection test still uses that name as
its input. The frontend keeps its `close_agent` display mapping, which
old histories still use.

**Review focus**

- A rejected `close_agent` call is now billed like any rejected tool
call; the rejection is immediate.
- `ARCHITECTURE.md`, "Generation goroutine": this PR deletes the
sub-bullet that called `close_agent` a deprecated alias of
`interrupt_agent`.
- The `resolved_tool_name` debug log field is gone. It differed from
`tool_name` only for aliased calls, and the alias is gone.

**Validation**

- `go test -count=1 ./coderd/x/chatd/ ./coderd/x/chatd/chatloop/
./coderd/x/chatd/chatprompt/`: ok
- `go vet` and `golangci-lint run --fix=false ./coderd/x/chatd/...`:
clean
- `paralleltestctx -custom-funcs="testutil.Context,chatdTestContext"
./coderd/x/chatd/...`: clean
- pre-commit hook: passed

**Stack**

This stack retires chatd compatibility paths for chat data that older
Coder versions stored.

This PR replaces #31003 and carries its change, adapted to the
`toolBatch` code that #30938 and #30939 merged. Mike chose to keep the
fallbacks that #31001 and #31002 removed, so those two layers and #31003
left the stack.

1. #30996 stamp chat message fixtures with the current content version
2. #30997 drop tests of the legacy content codec
3. #30998 retire the V0 chat message content decoder
4. #31000 inline the model config provider identity
5. #31005 retire the `close_agent` tool alias (this PR)

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
2026-10-30 02:00:59 +00:00
Callum Styan 2255794d65 feat: add response_size_bytes metric to chatd (#29816) 2026-10-28 18:35:20 -07:00
Mathias Fredriksson 2cb10b5cec feat: show web searches and the pages they found in chats (#29861)
Provider-executed web searches were invisible in chats. The UI skipped
provider-executed parts, and the only trace was one source row per
message that mixed OpenAI citations with Anthropic search results.

The fantasy pin moves to coder/fantasy#62, which tags each found page
with the search that found it and makes OpenAI request and report its
found pages and search actions. chatd stores the tag and marks a
search failed on an Anthropic error_code or an OpenAI failed status.
Each search renders as one row with its queries or opened page, a
running or unfinished state, and the pages it found. Cited pages stay
in the answer's source row, which now follows the answer text.

Closes CODAGT-795
2026-10-27 12:46:27 +02:00
Mathias Fredriksson 1aff810443 chore: sync fantasy with upstream v0.45.5 (#31288)
Moves the fantasy pin to the tip of `coder_2_33` (`85ffbbe`). That
commit is coder/fantasy#72, which merges 87 upstream
charmbracelet/fantasy commits, v0.40.0 to v0.45.5. Between the two pins
117 files change. Upstream's own v0.40.0 to v0.45.5 change, without
tests, testdata, examples, `go.sum` and the kronk provider, is 43 files,
+2271/−928.

## What reaches chatd

chatd builds only the `anthropic`, `openai` and `openaicompat` fantasy
clients (`chatprovider.ModelFromConfig`); other vendors reach it through
the AI Gateway on one of those. So the upstream changes to the Google,
Vercel, OpenRouter, Bedrock, Azure and kronk providers, and to the
`fantasy.Agent` loop and its retries, do not run in chatd. These do:

- **Chat Completions and openaicompat streams** keep reasoning on
batched and DeepSeek-shaped chunks, keep `finish_reason: length` when
tool calls are truncated, and stop sending empty reasoning fields.
openaicompat drops assistant turns that hold only reasoning.
- **Token usage.** When an OpenAI-shaped backend reports reasoning
tokens apart from completion tokens (reasoning larger than output),
output tokens now include them, so that backend's reported output tokens
and cost rise. OpenAI's own shape, where reasoning is a subset, is
unchanged.
- **Responses.** A content-filtered turn no longer reads as a tool-call
turn.
- **Context overflow.** Baseten, Fireworks and Parasail prompt-length
errors set `ContextTooLargeErr`, which compaction acts on.
- **Anthropic.** Tool calls whose content block never closes are
reported. For Claude 5 models, thinking defaults to `display:
"summarized"` when the chat config sets none; a configured display still
wins. `TestThinkingDropBlockModel_RetriesWithEffort` now expects the
field in the retry body.
- **Anthropic keepalives.** Fantasy forwards `message_start`,
`message_delta` and unknown events as content-free `keepalive` parts.
The first one arrives before any model output, so this PR makes
`ttft_seconds` and the `time_to_first_token` stage skip them, as they
skip warnings. They still reset the stream silence guard.
`processStepStream` ignores them, and the debug recorder counts them in
its part count but records no content.
- **Tool-result media** now goes through upstream's shared
`ToolRunBuffer`, which replaces coder/fantasy#64 and does the same
thing.

#72 keeps coder's anthropic-sdk-go and openai-go forks, Go 1.26 and
kronk v1.31.5, so upstream's SDK and Go moves do not come along.

`go mod tidy` raises the modules fantasy v0.45.5 requires, among them
otel exporters v1.47, the AWS SDK, `google.golang.org/api` v0.307 and
go-openapi v1. All are upgrades.

The `go.mod` comment now names #72 as the upstream sync.

coder/coder#29861 stacks on this PR to pin coder/fantasy#62.

<details>
<summary>Verification</summary>

- `go build ./...`.
- `go test` passes on every package that imports fantasy
(`coderd/x/chatd/...`), and `go test ./coderd -run
'TestChat|TestExpChat|TestMCP|TestAIBridge'` passes.
-
`TestGuardedStreamTTFTStage/WarningsAndKeepalivesThenContentObservesOnce`
fails without the TTFT change (TTFT observed 0s instead of 1s) and
passes with it.
- `make pre-commit` passes. The generated Prometheus docs pick up the
metric help change.

Not verified against live backends in this PR: the stream fixes, the
token usage change and the context-overflow patterns have no coder test
against a real provider. coder/fantasy#72 reports a local chatd run
against the AI Gateway on Anthropic, Responses, OpenAI Chat and
openaicompat.

</details>

> 🤖 This PR was created with the help of Coder Agents, and _will be_
reviewed by a human. 🏂🏻
2026-10-27 11:29:10 +11:00
Michael Suchacz a33af01605 refactor(coderd/x/chatd/chatloop): create the compaction debug run through StartRun (#31087)
Compaction was the last debug-run caller that did by hand what
`chatdebug.Service.StartRun` does: insert the run on a detached context
with a 5 second limit, rebuild the run context, and finalize it with
`ClassifyError`. This PR moves `startCompactionDebugRun` onto `StartRun`
and deletes `compactionDebugCreateRunTimeout`. #30934 moved the other
callers; compaction waited for #30766, which has merged.

**Why it is safe:**
- The first commit changes only the test, renamed from
`TestStartCompactionDebugRun_DoesNotReportDebugErrors` to
`TestStartCompactionDebugRun`. It now asserts the whole inserted row:
kind `compaction`, status `in_progress`, the parent run's root chat,
parent chat and trigger message, the summary model's provider, model,
model config and history tip when the caller sets them and the parent
run's otherwise, and no summary. It also asserts the returned run
context and the `completed` and `interrupted` finalize statuses. It
passes on the old code and on the new code.
- On the old code, 16 mutants (wrong kind or status, a dropped parent
field, an ignored override or fallback, a seeded summary, a wrong run
context field, a wrong finalize status) all fail the new test. Before
this PR, the chatloop tests caught 1 of them and
`TestActiveServer_CompactionModelOverride` 3 more (kind, model and model
config); the other 12 passed both. On the new code, all 13 mutants of
the `StartRun` call fail the test.
- `StartRun` keeps the rest: a failed insert returns the caller's
context and a no-op finalizer, the insert ignores the caller's
cancellation, and the run still finalizes through `FinalizeRun` with its
default 5 second limit. `TestService_StartRun` pins the first two. A nil
seed summary still stores no summary, because `nullJSON` maps a nil map
to SQL NULL.

**Review focus:** log output. The old code discarded create and finalize
errors. `StartRun` logs them at Warn as `failed to create debug run` and
`failed to finalize debug run`, like the other callers. A debug failure
still never fails compaction.

Validation:
- `go test -count=1 ./coderd/x/chatd/chatloop/...` passes, and so does
`go test -race` on the compaction debug tests.
- `go test -count=1 -p 2 -parallel 4 ./coderd/x/chatd/chatdebug/...`
passes.
- `go test -count=1 -parallel 4 -run 'Debug|Compact' ./coderd/x/chatd/`:
the same 53 tests pass before and after.
- `go vet` is clean on both packages, and the pre-commit hook, which
runs `paralleltestctx`, passed on both commits.

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
2026-10-27 01:13:58 +01:00
Hugo Dutka e75885a37e feat: add ACP subagent waiting and transcript streaming (#30560)
Add `acp_wait_agent` to stream ACP transcripts, including reasoning and
nested tool activity, to users while returning the subagent's text
response to the parent model. Reconnect dropped streams from their
transcript cursor, preserve running sessions on wait timeout or
cancellation, and persist separate user-visible and model-visible
results.

Depends on #31022.

Generated with Codex, carefully reviewed by Hugo.
2026-10-26 17:55:52 +01:00
Mathias Fredriksson 3c36790c86 fix(coderd/x/chatd): coalesce deltas when persisting interrupted turns (#29661)
An interrupted turn was stored with one ChatMessagePart per streamed
delta, while a completed turn gets one part per block. One 28,539
character interrupted response became 3,987 parts and 1 MB of JSON,
and a /messages page with such turns reached 11.7 MB. Whitespace-only
deltas were separate parts too, and prompt replay skips whitespace-only
parts, so words and paragraphs ran together in the next turn's prompt.

Adjacent deltas of the same type are now joined into one part. A text
run ends at another part type, a tool result or a narration change. A
reasoning run ends where CreatedAt, the block's start time, changes.
Adjacent text blocks still merge because text deltas have no block
identity. Rows already stored are not rewritten.
2026-10-23 23:59:37 +03:00
Michael Suchacz a86012f82b refactor(coderd/x/chatd/chatloop): resolve tool name aliases in one toolBatch method (#30939)
Adds `(*toolBatch).resolve` for the `ToolNameAliases` lookup that
`runOne`, `notifyStepToolCallObservers`,
`notifyStepToolResultObservers`, and `isSerialToolCall` each repeated.
The last three become `toolBatch` methods instead of taking the tool map
and the alias map. Where they looked up a tool and then type-asserted
it, they now assert on the map value: a missing name gives a nil
interface, so the assertion fails as the missing-key check did.

It also fixes a latent bug: `runOne` looked up result provider metadata
by the name the model sent, while its other lookups use the resolved
name, so a runner called by an alias lost its metadata. This cannot
happen in production today: the only alias is `close_agent` to
`interrupt_agent` (`subagent.go:45`), and only `computer_use.go:112`
sets `ResultProviderMetadata`.

## Why it is safe

Each mutant below puts the raw name back at one call site, and each
fails a test:

| Call site | Failing test |
|---|---|
| `runOne` tool lookup | `TestExecuteSingleTool_ResolvesToolNameAlias`
and 5 more |
| `runOne` metadata lookup | `TestExecuteLocalTools_ProviderToolRunner`
|
| `notifyStepToolCallObservers` |
`TestExecuteToolsNotifiesStepToolCallObservers` |
| `notifyStepToolResultObservers` (either loop) |
`TestExecuteToolsNotifiesStepToolResultObservers` |
| `isSerialToolCall` | `TestExecuteToolsSerialToolCallOrder` |

No test called a serial tool or a provider runner by an alias, so this
PR adds one aliased call to `TestExecuteToolsSerialToolCallOrder` and
one to `TestExecuteLocalTools_ProviderToolRunner`. With the
`chatloop.go` of #30938, the first test still passes, and the second
fails because the aliased result has no metadata.

## Validation

- `go test -race -count=1 ./coderd/x/chatd/chatloop/` passes on #30938
and here.
- `go test -count=1 -p 4 -parallel 4 ./coderd/x/chatd/...` passes here
(20 packages).
- `golangci-lint run ./coderd/x/chatd/chatloop/...` reports 0 issues,
and the pre-commit hook passed.

## Stack

This stack replaces the 20-parameter `executeTools` and the 15-parameter
`executeSingleTool` in `chatloop` with methods on one `toolBatch` type,
and resolves tool name aliases in one place. Production behavior does
not change.

1. #30937 Cover provider tool runners in `ExecuteLocalTools`
2. #30938 Replace `executeTools` and `executeSingleTool` with
`toolBatch`
3. #30939 Resolve tool name aliases in one `toolBatch` method

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
2026-10-20 09:39:54 +00:00
Michael Suchacz 63c116fddd refactor(coderd/x/chatd): delete the CompactionOptions and compactionOutcome mirrors (#30984)
Deletes two structs that each repeat another struct field for field.
`chatloop.CompactionOptions` repeated 21 fields of
`GenerateCompactionOptions`, and `normalizedCompactionGenerateConfig`
copied them one by one, so each new option had to be added in three
places, as #30766 did for `Logger`. `compactionOutcome` in `chatd`
repeated the 11 fields of `chatloop.CompactionResult` only so that
`generateCompaction` could convert one into the other. The PR also
deletes `TestGenerateCompactionSummaryUsesToolDefinitions`, which
repeats `TestGenerateCompactionSummaryPreservesCallOptions`.

## Why it is safe

- `publishCompactionError`, `startCompactionDebugRun`, and
`generateCompactionSummary` now take `GenerateCompactionOptions`. Every
read of the normalized config, in `GenerateCompaction` and in those
helpers, uses one of the 21 fields that the copier copied without
change. None reads the 7 fields that only `GenerateCompactionOptions`
has.
- `normalizedCompactionGenerateConfig` applies the same four defaults
and the same threshold gate to its parameter, which is already a copy.
So `GenerateCompaction` still reports the requested `ThresholdPercent`
and `ContextLimit` from its own `opts`. If it read
`config.ThresholdPercent` instead,
`TestGenerateCompaction_RecordsRequestedTrigger` would fail.
- Go allows the `compactionOutcome(outcome)` conversion only because the
two structs have identical fields, so `buildCompactionMessagesInput` can
hold the `chatloop.CompactionResult` itself. The four
`TestBuildCompactionMessages_*` tests that build it keep their
assertions.
- Six mutants of the summary call each fail the deleted test on #30766:
no tools, no call options, no summary prompt, one tool dropped, the
caller's history changed, and tool descriptions stripped. The kept test
failed only the first five, because its tool had no description. With
that description moved into it, the kept test fails all six.
- `git grep -w CompactionOptions` and `git grep compactionOutcome` find
no matches.

## Review focus

`generateCompactionSummary` keeps its `model` and `messages` parameters,
so its 15 direct test calls change only the type name. It now also
receives `Model` and `Messages` fields with the same values, and it does
not read them.

## Validation

- `go test -race -count=1 ./coderd/x/chatd/chatloop/` passes.
- `go test -count=1 -p 4 -parallel 4 ./coderd/x/chatd/...` passes (20
packages).
- `golangci-lint run ./coderd/x/chatd/...` reports 0 issues, and the
pre-commit hook passed.

## Stack

#30766 has merged, so this PR now targets `main` by itself.

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
2026-10-20 09:29:57 +00:00
Michael Suchacz 3df3c03288 refactor(coderd/x/chatd): replace executeTools and executeSingleTool with toolBatch (#30938)
Replaces the 20-parameter `executeTools` and the 15-parameter
`executeSingleTool` in `chatloop` with methods on an unexported
`toolBatch`. `newToolBatch` builds it once from
`ExecuteLocalToolsOptions`, `run` replaces `executeTools`, and `runOne`
replaces `executeSingleTool`. The 12 internal test calls set only the
options they use instead of up to 20 positional arguments.

It also deletes two copies:

- `executeTools` filtered out provider-executed calls that
`ExecuteLocalTools` had already removed. Only the `ExecuteLocalTools`
filter remains.
- The `runOne` metric defer repeated the body of
`recordToolResultMetrics`. It now calls that function.

## Why it is safe

- `newToolBatch` builds the tool, runner, and metadata maps with the
lines `executeTools` used, and the byte budget as `ExecuteLocalTools`
did. `run` drops the empty-batch returns: `ExecuteLocalTools` returns
before it when no local call is left.
- `recordToolResultMetrics` labels by `result.ToolName`, which `runOne`
sets to `tc.ToolName` and never changes. It skips a nil `Metrics`.
Production never passes one, because `ExecuteLocalTools` sets
`NopMetrics()` first; only the direct test calls, which omit `Metrics`,
use that path.
- With the duplicate filter gone, removing the remaining
`ProviderExecuted` filter fails
`TestExecuteLocalTools_ProviderToolRunner` (#30937). The other four
mutants from #30937 still fail it here.
- `git grep 'executeTools\|executeSingleTool'` finds one comment, in
`chatd_test.go`, which this PR updates.

## Review focus

The metric defer stays a closure so that it reads `result` when `runOne`
returns.

## Validation

- `go test -race -count=1 ./coderd/x/chatd/chatloop/` passes on #30937
and here, including the 16 tests that reach this code:
`TestExecuteTools*` (5), `TestExecuteSingleTool_*` (5),
`TestExecuteLocalTools*` (6).
- `TestActiveServer_ToolErrorRecordsMetric`,
`TestActiveServer_ToolExecutionAndPolicy`, and
`TestProcessChatPanicRecovery` in `./coderd/x/chatd/` pass on #30937 and
here.
- `golangci-lint run ./coderd/x/chatd/chatloop/...` reports 0 issues,
and the pre-commit hook passed.

## Stack

This stack replaces the 20-parameter `executeTools` and the 15-parameter
`executeSingleTool` in `chatloop` with methods on one `toolBatch` type,
and resolves tool name aliases in one place. Production behavior does
not change.

1. #30937 Cover provider tool runners in `ExecuteLocalTools`
2. #30938 Replace `executeTools` and `executeSingleTool` with
`toolBatch`
3. #30939 Resolve tool name aliases in one `toolBatch` method

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
2026-10-20 09:26:42 +00:00
Michael Suchacz f004c72749 test(coderd/x/chatd/chatloop): cover provider tool runners in ExecuteLocalTools (#30937)
Adds `TestExecuteLocalTools_ProviderToolRunner`. No test ran a
`ProviderTool` runner (the local half of a provider-defined tool, such
as OpenAI computer use) through `ExecuteLocalTools`, so nothing checked
runner dispatch, the active-tool exemption for runners, result metadata,
or the skip of provider-executed calls. #30938 rewrites that code, so
this PR pins the behavior first.

The test sends a runner call and a provider-executed `web_search` call.
The runner is in neither `Tools` nor `ActiveTools`. The test checks that
only the runner call gets a result, and that the result has the runner's
output and its `ResultProviderMetadata`.

## Why it can fail

Each mutant is `chatloop.go` with one change, run with `go test
-overlay`. Each one fails the new test, and all other `chatloop` tests
pass with it.

| Mutant | New test fails with |
|---|---|
| Runners are not added to the tool map | `Tool not found: computer` |
| Runners are not exempt from the active-tool check | `Tool not active
in this turn: computer` |
| `ResultProviderMetadata` is not attached | nil metadata |
| Provider tools without a runner are not skipped | nil pointer panic |
| Both `ProviderExecuted` filters are removed | 2 results |

Removing only one of the two `ProviderExecuted` filters, in
`ExecuteLocalTools` or in `executeTools`, fails no test, because the
other filter still runs. #30938 deletes the one in `executeTools`.

`go test ./coderd/x/chatd/` also passes with runners left out of the
tool map and the metadata attachment removed. Its subtest
`TestActiveServer_ToolExecutionAndPolicy/provider_runner_executes_and_preserves_metadata`
replays a stored computer-use call and result; it does not run the
runner.

## Validation

- `go test -race -count=1 ./coderd/x/chatd/chatloop/` passes.
- `golangci-lint run ./coderd/x/chatd/chatloop/...` reports 0 issues,
and the pre-commit hook passed.

## Stack

This stack replaces the 20-parameter `executeTools` and the 15-parameter
`executeSingleTool` in `chatloop` with methods on one `toolBatch` type,
and resolves tool name aliases in one place. Production behavior does
not change.

1. #30937 Cover provider tool runners in `ExecuteLocalTools`
2. #30938 Replace `executeTools` and `executeSingleTool` with
`toolBatch`
3. #30939 Resolve tool name aliases in one `toolBatch` method

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
2026-10-20 11:02:53 +02:00
Michael Suchacz 8245457c8a fix(coderd/x/chatd): make compaction fit an overflowing context (#30766)
Compaction is the recovery path for a chat near its context window, but
its summary request was larger than the turn that overflowed: the whole
history plus the tool definitions. Its only fallback, a retry without
tools, also missed context-window errors that arrive inside an HTTP 200
stream (OpenAI Responses API, Bedrock Mantle). In the incident behind
this PR, one MCP tool result of about 1 MB moved a 1M-token chat from
67% to 92% of its window in one step. After that, every turn and every
`/compact` failed with "Your input exceeds the context window of this
model."

This PR does not try to predict whether the summary request fits. Each
time the model rejects it for its size, the next request is thinner:

1. The full request: tool definitions and the whole history.
2. The same history without tool definitions (as before).
3. Tool results longer than 32 KiB cut to 32 KiB, keeping their head and
tail.
4. Tool results longer than 4 KiB cut to 4 KiB.

A step that would cut nothing is skipped. Only locally executed text and
error tool results are cut. User messages, attachments,
provider-executed results, and media stay whole. The cuts apply only to
the summary request, not to the stored history.

Other changes:

- **In-stream rejections match.** Fantasy stream errors without a status
code and Responses `error` and `response.failed` events now count as
size rejections. A Responses error code other than
`context_length_exceeded` never matches, so a rate limit stays
retryable. Anthropic's "input length and `max_tokens` exceed context
limit" also matches.
- **A conversation that still does not fit** fails as a non-retryable
config error that tells the user how to recover.
- **Each rejected summary request is logged at Warn** with its tool
count, tool result cap, and the bytes it cut.

Not in this PR: counting tool results that arrive after the measured
step in the compaction trigger, estimating the summary request before
sending it, and a tighter per-result cap for large-window models.
Earlier revisions of this branch did these and were dropped for the
simpler path above.

## Testing

- `go test ./coderd/x/chatd/...` (20 packages), `golangci-lint` on
`chatd` and `chatloop`, and the pre-commit hook pass.
- The real OpenAI Responses client runs against `httptest` for each
in-stream error shape and must retry without tools.
- Each of these breaks a test: removing the cut steps, sending a cut
that cuts nothing, turning error results into text, cutting
provider-executed results, and dropping the in-stream match.

> Xum opened this PR on behalf of @ibetitsmike.
2026-10-20 00:30:05 +02:00
Michael Suchacz 7517301ffa chore: add golangci new-code quality gates (#30622)
Stack context: second layer of the Go lint gate stack, built on the
exclusion fix below.

Why: adds commented-out code detection and a `log/slog` ban to the main
config, plus a new-code-only pass requiring reasons on lint
suppressions, flagging ignored errors (including `_, _ = w.Write(...)`),
and banning `time.Sleep` in tests. Fixing the hits surfaced a real bug:
GitLab warnings went to the wrong logger. The new-code pass resolves
symlinked checkout paths so it does not silently report nothing.

Validation: both configs report 0 issues; each seeded violation is
caught; `make pre-commit` passed.

> Xum acted for Mike to open this PR.
2026-10-13 07:57:56 +00:00
Michael Suchacz 41f5caafec fix: fold web searches, answer reasoning, and OpenAI narration into working blocks (#30529) 2026-10-12 22:53:07 +02:00
Michael Suchacz c28ae0dc9b feat: trigger compaction independently for the chat and compaction models (#28568)
> [!NOTE]
> Xum (an AI agent) authored this pull request on Mike's behalf.

## Summary

Automatic compaction now fires at whichever of two independent triggers
is reached first, and the UI shows when the organization override's
trigger applies.

- **Chat trigger**: the chat owner's personal threshold (or the chat
model's) against the chat model's window. A threshold of 100 turns off
only this trigger.
- **Organization trigger**: a fixed 80% of the organization compaction
override model's window. The override model's own compression threshold
is ignored. The other 20% is a target margin for the summary prompt and
the summary, not a guarantee, because the trigger reads the previous
step's prompt size.

On main, the chat threshold was applied to the smaller of the two
windows, and nothing in the UI showed it. Example: with a 20K chat model
at 70% and an 8K override, main compacts at 5,600 tokens; this PR
compacts at 6,400 (80% of 8K, shown as 32%).

## Changes

Backend (`coderd/x/chatd`):

- The trigger with the lower token point binds (ties go to the chat
trigger). It drives the automatic compaction gate and the
still-over-limit check, and every summary, automatic or manual, records
it.
- An override that is unset, deleted, disabled, on a disabled provider,
or without credentials for the chat owner is treated as absent: the chat
model summarizes and only the chat trigger applies.
- The summary's output cap is bounded by the summarizer's own window,
not the trigger's.
- The still-over-limit error names what can clear it: the compaction
threshold (chat trigger), an override model with a larger window
(organization trigger), or both when usage is over both points.
- The still-over-limit check compacts once more when the latest summary
was recorded under a different threshold or trigger window. The
`chat_summarized` result gains `trigger_context_limit_tokens`, and
`context_limit_tokens` now records the chat model's window (main
recorded the smaller window). On main, lowering a threshold after a
compaction ended every later turn with the error.
- The still-over-limit check skips a 0% threshold ("always compact"),
which on main ended the run with the error after every compaction.
- `ARCHITECTURE.md` documents the two triggers, the summary cap, and the
still-over-limit rules.

Frontend (`site`), through a shared `compactionTriggers` helper that
mirrors the backend rules (no new API):

- Coder Agents settings: the compaction override row warns when the
organization trigger undercuts chat models' default triggers ("the
organization override compacts chats at 80% of {model}'s {N}-token
window").
- Personal thresholds page: an Effective column shows the trigger that
applies, with a popover when the organization override binds. "Compacts
at" follows the same trigger, and a point the chat model's window cannot
reach shows as Off. The header's 100% sentence, and the row warning when
a member types 100, say an organization override may still compact chats
when one is set or its organization's overrides failed to load (the
header also while they load).
- Chat context gauge: the compact-at percent follows the binding trigger
and is annotated when the organization override binds. If the overrides
request fails, it shows the chat-only value with "(organization override
not loaded)", or "Compaction off (organization override not loaded)"
when the chat trigger is off.
- An override whose provider the user cannot use is ignored on the gauge
and the personal page, matching chatd.
- The shared `formatContextLimit` prints counts below 1,000 exactly (an
800-token compaction point showed "~1K") and 999,500 to 999,999 as
"1.0M" instead of "1000K". It also labels the model pickers; their other
values are unchanged.
- `HelpPopoverIconTrigger` lets callers set their own `aria-label`; only
the Effective column's popover does.

Known v1 limits: the Coder Agents settings alert is organization-scoped,
so it can warn about an override that some members fall back from for
lack of provider credentials. The gauge checks the viewer's provider
access, so on a shared chat its label can differ from the owner's
runtime trigger (Codex P2, declined). Compaction itself is unaffected.

## Screenshots

From remote UAT round 13 on 093076f402: an isolated test deployment
with a local fake LLM. The models are test fixtures named for their
context windows: Chat model 20K (own threshold 70%), Chat model 5K
(40%), and Chat model 8K (50%). Chat model 8K is also the organization
compaction override, so the organization trigger is 80% of 8,000 = 6,400
tokens, which is 32% of Chat model 20K's window. Chat model 8K's own 50%
applies only to chats that use it as the chat model; changing it to 10%
or 100% left every value below unchanged.

**Coder Agents settings:** the organization trigger undercuts Chat model
20K's 70% default.

![Override warning: 80% of Chat model 8K's window undercuts Chat model
20K](https://github.com/user-attachments/assets/188c92e6-faaa-4e9a-a563-567cbf98af85)

**Personal thresholds:** the organization trigger binds Chat model 20K
at 32%. Chat model 5K (40%) and Chat model 8K (50%) keep their own chat
triggers.

![Effective column: 32% from the organization trigger on Chat model
20K](https://github.com/user-attachments/assets/9949be4e-9417-4cfc-a683-f1367a1dd77a)
![Organization override popover: 80% of Chat model 8K's window, about
32% of this model's
window](https://github.com/user-attachments/assets/4bc3e002-0be1-46fd-91f0-aa27503034af)

**Personal thresholds, organization settings request failed** (the first
red line is the injected error). Rows fall back to chat-only values.

![Organization settings load failure above usable chat-only
rows](https://github.com/user-attachments/assets/dcf48546-aa47-4f82-a6eb-85f106b4bf45)

**Chat context gauge:** organization trigger binds (Chat model 20K),
chat trigger binds (Chat model 5K), overrides request failed (injected).

![Gauge: Compacts at 32% (organization
override)](https://github.com/user-attachments/assets/cd5402d9-9849-4254-a7a7-6623c4006a2a)
![Gauge: Compacts at
40%](https://github.com/user-attachments/assets/f372c9e9-72ef-41c8-950c-e77d74965f65)
![Gauge: Compacts at 70% (organization override not
loaded)](https://github.com/user-attachments/assets/ab829746-2ab5-4743-848f-dce4e806de3a)

**Still-over-limit errors from real runs:** organization trigger, chat
trigger, both.

![Organization-only over-limit
error](https://github.com/user-attachments/assets/f210807a-a2cc-47a0-807c-f1404373bb28)
![Chat-only over-limit
error](https://github.com/user-attachments/assets/eecec38e-97de-49b6-8670-b32090103bbb)
![Combined over-limit
error](https://github.com/user-attachments/assets/459a6d08-7215-492d-93f5-96a4f767a09e)

## Testing

- Go: `TestBindingCompactionTriggerSource`,
`TestCompactionStillOverLimitError`, `TestCompactionStatusFromHistory`,
and `TestActiveServer_CompactionModelOverride` cover the 80% point
(fails if the override's own threshold is read again), user threshold
100 leaving the organization trigger on, the summary cap for automatic
and manual compaction, the over-limit messages, and recompaction after a
trigger change. The site binding tests mirror the Go table row for row.
- Site: `compactionTriggers` unit tests and Storybook stories for the
three surfaces, including the overrides fetch failure.
- On c6f8750479: `go vet`, the `chatd` and `chatloop` tests, and the
AgentsPage, AISettingsPage, and HelpPopover unit tests (2,274 passed, 2
skipped) pass, and CI is green. On 987cefe399, tsc, the five affected
story files (109 passed), and `make pre-commit` also passed. 093076f402
only changes `ARCHITECTURE.md`; the pre-commit hook's markdown format
and lint checks pass on it.
- On 3cd5b08f1f: tsc, the new `formatContextLimit` test (fails on the
old helper for 800 and 999,999), the `compactionTriggers` and `aiModels`
unit tests (224 passed), and the affected story files pass, and the
pre-commit hook passed on all three commits.

## Review record

- Earlier heads: 12 Codex passes and 3 human reviews, all threads
answered. Rebased onto main several times, most recently ff3fa22f68
(kept #30274, #29892, and #29890 in conflicted files).
- coder-agents-review rounds 1 to 18: findings fixed; approved since
round 15. The round 1 `ARCHITECTURE.md` finding was left for the
maintainer, who asked for the update in the last commit.
- coder-frontend-review (eight roles): all pass. Splitting the PR (RO6)
was declined because it is reviewed as one unit; the
`organizationTriggerFor` story helper stays because its three fields
must match the model.
- Design change after UAT round 11: the organization trigger is a fixed
80% of the override window instead of the override model's own threshold
(171d5f568e, 21d2ce7fce, 987cefe399).
- coder-agents-review round 19: copy, naming, and an unreachable
zero-window gate fixed (6e8efb9054, c6f8750479). A summary-time fit
check with chat-model fallback (CRF-156) was declined: the previous-step
lag applies to every trigger, including main's. Round 20 approved
c6f8750479 with no findings.
- Codex: one P2 on 987cefe399 declined (see known limits); no major
issues on c6f8750479.
- Remote UAT on dogfood (fake LLM, isolated test deployment): rounds 8
to 13 PASS. Round 12 on c6f8750479 first failed one criterion that was
written wrong (the admin alert correctly lists an override model that is
also a chat model at 100%); the reviewer endorsed PASS on the same
evidence after the correction. Round 13 on 093076f402 retook the
screenshots with neutral model names and confirmed that the override
model's own threshold (10%, 50%, 100%) does not move the organization
trigger.
- UAT round 13 notes: the header after a failed or pending overrides
load and the sub-1K labels are fixed in 4e172ab5ff and d5888ce15dc;
coder-agents-review round 21 found the same gap in the row warning
(CRF-167, fixed in 3cd5b08f1f). Left as on main: a summary's
`usage_percent` is measured against the binding trigger's window, so it
can exceed 100 (nothing reads it), and the 100% help text shows only for
an unsaved edit.
- Codex: no major issues on d5888ce15d and 3cd5b08f1f.
coder-agents-review round 22 approved 3cd5b08f1f with no open findings.
- coder-frontend-review on those commits: all roles pass after the
fixes. Declined: moving the formatter fix to its own PR (RO5; the owner
asked for it here), adding "tokens" to sub-1K labels (TN6; "~800" reads
like "~16K"), and splitting the component (RP6).
2026-10-11 12:10:59 +02:00
Michael Suchacz bb9f31d94b fix: apply Anthropic prompt caching through OpenRouter and OpenAI-compatible gateways (#29563)
Anthropic models configured through an `openrouter` provider (and, by
the same code path, `vercel` and generic `openai-compat` providers)
never got prompt caching. Two independent gaps: chatd only emitted
`cache_control` breakpoints when the fantasy client was `anthropic`, and
even when a client sent them, the AI Gateway chat-completions
interceptor rebuilt the upstream body from typed
`openai.ChatCompletionNewParams` and silently dropped the unknown key.
Dogfood UAT on main @ 77479408b4 measured 0 cached tokens on 27/27 turns
for `anthropic/claude-haiku-4.5` via OpenRouter while the direct
Anthropic provider cached 7k to 11k tokens per turn.

## Fix

**AI Gateway** (`aibridge/intercept/chatcompletions`):
`ChatCompletionNewParamsWrapper` records every message-level and
content-part-level `cache_control` from the raw request and re-applies
it at the same JSON path when the upstream body is rebuilt, before the
Google-specific patching. The blocking path now sends the raw body
whenever preserved fields exist; the streaming path already sent the
rebuilt body. Nothing is added unless the client sent it, and at most
four markers are recorded per request (Anthropic's breakpoint limit) so
the per-marker body rewrites stay bounded.

**chatd** (`coderd/x/chatd`): the caching gate becomes a small strategy
resolver keyed on the fantasy client and the model ID. `anthropic` keeps
the existing message-level breakpoints; `openrouter` and `vercel` reuse
them for Anthropic-family IDs since both clients translate them;
`openai-compat` gets content-part-level `ContentExtraFields` markers
(last text part of the last system message plus the final two
text-bearing user or assistant messages, tool messages skipped, at most
3 markers). `chatprovider.IsAnthropicFamilyModelID` recognizes
`anthropic/...`, Bedrock-style `anthropic.`/`.anthropic.` IDs, and bare
`claude` IDs. The `claude` substring match is deliberately broad; a
non-Anthropic model whose ID contains it would receive markers that
OpenAI-compatible upstreams ignore.

Docs: a short "Prompt caching" note in `docs/ai-coder/agents/models.md`.
`coderd/x/chatd/ARCHITECTURE.md` has no caching text to update.

## Dogfood UAT

Remote Coder Agents UAT on dogfood (chat:
https://dogfood.cdr.dev/agents/9f81cc40-add0-4b6f-b76c-f348b04762e4),
OpenRouter provider with real keys:

- Baseline (main @ `77479408b4`): 0 cached tokens on every OpenRouter
Haiku turn; direct Anthropic control cached normally; a hand-added
`cache_control` on the same key cached fine.
- Fix (`1d685f7e3e`, the first three commits):
`anthropic/claude-haiku-4.5` via OpenRouter now sends 2 markers on turn
1 and 3 afterwards (system last part plus the last two text-bearing
messages, never on tool messages). OpenRouter's `/generation` ground
truth reported `native_tokens_cached` 7137 to 7255 on turns 2+, equal to
Coder's `cache_read_tokens` and the AI Gateway's
`cache_read_input_tokens` on all 43 measured rows. Turn-2 prompt cost
dropped about 8x versus baseline. Agentic S3 cache reads grew from 7156
to 13993 as a 2500-line file entered the prompt. `openai/gpt-5-mini`,
`google/gemini-2.5-flash`, `deepseek/deepseek-chat-v3.1`, and
`moonshotai/kimi-k2.5` received 0 markers with unchanged request shapes;
the direct Anthropic control was unchanged. 76 upstream attempts, 0
non-200. The remote agent reported FAIL only because one acceptance
criterion I wrote asked the chat usage indicator to display cached
tokens, which the UI has never done for any provider; the runner
endorsed PASS for the fix itsel... [truncated]
- Smoke re-tests on `20ad259999b` (marker-code simplification) and on
`e31f1806748` (docs reflow plus the four-marker cap; the later head
`92b8f8ad0b8` only renames `google_openai_compat.go` to
`request_body.go` with no content change): S1 cache reads 0 / 7139 /
7161 / 7184 / 7207 equal to OpenRouter ground truth on every row,
markers 2 then 3 and never on tool messages, tool-heavy turn all 200, a
12 KB file read grew cache reads to 13548 with every request body
inspected for marker placement, `openai/gpt-5-mini` still 0 markers.

Known follow-ups surfaced by the UAT and out of scope here:
`provider_options.openrouter.*` never reach the wire on the gateway
path; OpenRouter cache writes are folded into `input_tokens` in chat
message usage (`cache_creation_tokens` stays 0) although the gateway
records them; the AI Gateway sessions UI has no cache columns and labels
every OpenAI-compatible provider "OpenAI"; attachment-only messages (no
text) fail upstream with HTTP 400 on OpenRouter (pre-existing,
reproduced without markers).

> Xum acted on behalf of @ibetitsmike for this pull request.
2026-10-04 22:55:37 +02:00
J. Scott Miller fc0af7fbd8 feat: open a chat_turn span per prompt (#29771)
**TL;DR:** Wraps each prompt in a `chat_turn` root span with every
lifecycle stage parented under it. The span records one outcome per turn
(`completed`, `interrupted`, `error`, `abandoned`), and the new
`coderd_chatd_turn_outcomes_total` counter counts those outcomes. Turns
stay correct when tasks are canceled, replaced, or retried.

Each prompt runs inside a `chat_turn` root span owned by the runner
(`runnerTurnSpan`). The generation step, prepare, MCP connect, stream,
TTFT, provider attempt, commit, thinking, tool call, compaction, and
retry backoff stages are parented under it, activating the stages from
#29770. `chat_turn` spans are standalone roots: the turn runs on a
worker, often on another replica, and no trace context is persisted.

**Lifecycle**

- **Start.** Each generation loop iteration calls `Ensure`, which opens
the turn if none is open and returns a `turnToken`. The span is
backdated to the trigger: the latest of the last user prompt, a dynamic
tool's result message after it, and a pending compaction request. An
`acquisition` stage covers trigger to pickup.
- **Stale trigger / takeover.** A trigger at or before the previous
anchor opens the turn at now with no `acquisition` and counts
`stale_anchor`. A runner that took the chat over from an owner with a
stale heartbeat (`TakenOver`, from acquisition) opens its first turn at
now with no `acquisition`, so the dead owner's run is not reported as
pickup delay.
- **Newer prompt.** `Ensure` on the current turn with a newer trigger
closes it and opens a new one. The closed turn is `abandoned` unless the
task that ran it records an outcome before releasing it.
- **Finish.** After `FinishTurn` or `EnterRequiresAction` commits, the
step calls `Complete`; `Settle` closes the span after the step's own
stage ends, so the finishing step is inside the turn. A chat waiting in
`requires_action` is outside any turn; the generation resumed by its
tool results opens a new one.
- **Emission.** Closing a turn fixes its end time; its span is emitted
once every task that joined it (`Ensure` takes the task ID) has called
`Release`, which `runTask` does after the task returns. A finishing task
whose `FinishTurn` promoted a queued prompt therefore still counts its
turn as `completed` even when the promoted task's `Ensure` runs before
its `Complete`. `End` at runner exit emits whatever is left.
- **Failure.** A step that commits `FinishError`, or fails closed,
invalidates the turn as `error` after the commit. Errors the task runner
retries, including task timeouts, keep the same turn open; an expected,
non-retryable exit records no outcome and leaves the turn open until the
interrupt task, a newer prompt, or runner exit closes it. A canceled
generation task records no outcome and cannot obtain a turn token, so a
replaced task never touches its successor's turn.
- **Interrupt.** The interrupt task marks the open turn `interrupted`
before `FinishInterruption` commits and closes it after. Only a stop (by
the user or a parent agent) is `interrupted`; it ends with unset span
status, since error status is reserved for `error` turns. runner
shutdown, loss of ownership, and message edits close the turn as
`abandoned` with unset status.
- **Promotion.** Every promoting transition (`FinishTurn`,
`FinishInterruption`, a send to an errored chat, `PromoteQueued`)
returns the promoted message's queued time in its result, and one
`recordQueueWait` helper records a standalone `queue_wait` to the
promotion. The promoted turn opens at the promotion time.

`chat_turn` is observed on `coderd_chatd_stage_duration_seconds` only
for completed turns, so the histogram is completed-turn latency; every
turn still emits its span.

`coderd_chatd_turn_outcomes_total{outcome, chat_kind}` counts every
closed turn exactly once (`completed`, `interrupted`, `error`,
`abandoned`), matching the span's `turn_outcome`, under the
`chat-stage-metrics` experiment. Turn time accounting is in draft
#29772.

**Stages and identity**

- `tool_call` is a live stage around each local tool call in `chatloop`;
the tool runs on its context, and it ends errored only when the tool
fails to execute. Calls that never run (inactive, unknown, or rejected
by policy) record no stage.
- `mcp_connect` carries connected and failed server counts and ends
errored only when servers were configured and none connected.
- The compaction stage carries the model that runs the summary.
- `withStageIdentity` puts scope and chat kind on the turn context.
Detached background work runs in `background` scope as its own trace
roots.
- The `advisor` tool's nested model call appears only as its enclosing
`tool_call`.

ARCHITECTURE.md gains a "Lifecycle tracing" section.

<details>
<summary>Review decisions</summary>

- Turn identity stays timestamp-based with targeted fixes (clamped
acquisition, rotation on a newer trigger, invalidate after commit,
fail-closed invalidation, settle closing invalidated turns) plus a
takeover flag; keying turns on message IDs would not detect takeover and
cannot express interrupted/abandoned outcomes.
- Outcome classification: the interrupt task owns `interrupted`, and
cancellation or a fence mismatch alone never records an outcome, so
shutdown, ownership loss, and edits are `abandoned`. Only `error` turns
set error span status. Remaining ordering races between a step and an
interrupt are documented as known limitations in ARCHITECTURE.md.
- Turns are emitted after the tasks that ran them return, not when they
close, so an outcome a replaced task records while unwinding reaches its
own turn; the successor task is not delayed. Chosen over an
after-commit, pre-publish hook on `ChatMachine.Update`, which would fix
only the `FinishTurn` ordering.
- `requires_action` ends the turn rather than pausing it, so `chat_turn`
measures chatd time; a prompt with client tools counts as several turns.
- Promoted turns anchor at promotion time and `queue_wait` is always
standalone, so `chat_turn` never includes time spent queued behind
another turn.
- The former `capacity_wait` stage (#29805) was dropped: it was a
per-worker lower bound, excluded from accounting, and needed its own SQL
and bookkeeping. #29805's remaining helpers are folded into this PR.

</details>

Supersedes #28972, #28973, and #29805.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
2026-09-30 11:53:58 -05:00
J. Scott Miller e988ddb11d feat: trace chatd provider stages and count only output parts as first token (#29770)
**TL;DR:** Traces the model provider call: a `stream` stage, a nested
`time_to_first_token` stage, and one `provider_attempt` per HTTP round
trip, labelled with provider type and model. Behavior change:
`coderd_chatd_ttft_seconds` now counts only a real output part as the
first token, so error, warning, and finish parts no longer count.

Instruments the provider call.

**Stream and time to first token.** `GenerateAssistant` records a
`stream` stage from stream open to close and a nested
`time_to_first_token` stage. The TTFT window closes once, on the first
streamed output part: any part other than warnings, finish, or error,
including start markers such as `text_start`. A closed window observes
the stage and the pre-existing `coderd_chatd_ttft_seconds` with the same
elapsed time. A failed open, an error part, or a release without an
output part ends the TTFT span with the error (a silence timeout is
classified as such) and observes nothing. The `stream` span ends once,
with the error `GenerateAssistant` returns, so content-filter and
no-output attempts end errored. Failed `stream` windows are observed
like successful ones. The attempt is released before the stream span
ends, so the TTFT window always falls inside its stream. The compaction
summary stream passes a nil tracer, so it records neither.

Without #29771, production emits only `provider_attempt`; #29771 passes
the tracer to `GenerateAssistant`.

**Provider attempts.** The provider HTTP transport is wrapped in
`stageSpanRoundTripper`, which emits one `provider_attempt` stage per
round trip, so retries each get a span. The span closes on response
headers; a status >= 400 marks it errored without changing what the
client sees.

`resolveModelCall` resolves the `StageModel` (wire provider, configured
`provider_type`, model, effective reasoning effort) and threads it to
the transport, so span attributes and labels match the call.
`provider_type` is read from `ai_providers.type`, so types outside the
fantasy set (copilot) keep their configured name. `chatd.New` builds the
tracer with the server clock.

**Behavior change to `coderd_chatd_ttft_seconds`:** only an output part
counts as a first token. Error parts no longer close the window as a
success, and warnings and finish parts are skipped; the help text says
so.

`StageSpan.End` returns the span's elapsed window, which `ttft_seconds`
observes. Callers pass `Stages` and `StageModel` through
`GenerateAssistantOptions`; a nil tracer discards (TTFT still records
`ttft_seconds`). `Server` gains the `stages` tracer and a
`TracerProvider` option, wired from `coderd`.

<details>
<summary>Review decisions</summary>

- Start markers close the TTFT window: they are the first sign the model
is producing output, and waiting for a delta would fold hidden reasoning
time into TTFT.
- Failed `stream` and `provider_attempt` windows are observed: that time
was spent. TTFT is the exception because a first token that never
arrived has no duration.
- `http_host` was dropped from `provider_attempt`; every request goes
through the local AI Gateway transport, so it was constant.
- The advisor tool's nested model call is intentionally not instrumented
as `stream`/TTFT.

</details>

Supersedes #28970 and #28971.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
2026-09-30 11:53:58 -05:00
J. Scott Miller 7a9442014e feat: add chatd lifecycle stage tracer (#29769)
**TL;DR:** Adds a stage tracer that turns each chat lifecycle stage into
both an OpenTelemetry span and a Prometheus histogram observation, so
traces and metrics always agree. The metrics are only registered with
the `chat-stage-metrics` experiment. Nothing calls the tracer until the
next PRs.

Adds `chatloop.StageTracer`, which emits each chat lifecycle stage as an
OpenTelemetry span and a `coderd_chatd_stage_duration_seconds{stage,
scope, chat_kind}` observation from one `End` call, so trace and metric
durations cannot disagree. The provider-bound stages (`stream`,
`provider_attempt`) are also observed on
`coderd_chatd_model_stage_duration_seconds{stage, provider_type,
chat_kind, model}`, the only family carrying `provider_type` and
`model`, and only for turn-scoped stages. Time to first token per model
stays on the existing `coderd_chatd_ttft_seconds`. Nothing calls the
tracer yet; the next PRs instrument the provider call and the runner.

- The families are registered only when the `chat-stage-metrics`
experiment is enabled (safe and known, so
`--experiments=chat-stage-metrics` or `--experiments=*` turns it on).
Spans are emitted regardless. `NewMetrics` leaves the families off;
`chatd.New` opts in from the experiment, and a test asserts the
experiment decides whether they reach the server's registry.
- `Stage`, `Scope`, `ChatKind`, and `StageAnomaly` are typed string
constants. `scope` separates latency attributable to a prompt (`turn`)
from detached `background` work; `chat_kind` is `root` or `subagent`,
empty without a known chat.
- `StageModel` carries the wire `provider`, `provider_type`, `model`,
and `reasoning_effort`. All are span attributes; only `provider_type`
and `model` are metric labels, on the model family. No family carries an
organization label.
- Observed stages: `chat_turn`, `queue_wait`, `acquisition`,
`mcp_connect`, `stream`, `time_to_first_token`, `provider_attempt`,
`tool_call`, `commit`, `retry_backoff`. `generation_step`, `prepare`,
`thinking`, and `compaction` are span-only. A test keeps the stage sets
and help text consistent.
- API: `Start` / `StartRootAt` for live spans (span timestamps come from
the tracer clock), `Record` for stages reconstructed from timestamps
(the scope comes from the context). `End` is idempotent;
`EndWithoutObservation` ends the span without a sample. A nil tracer,
provider, or metrics discards.
- `coderd_chatd_stage_anomalies_total{reason}` counts observations
dropped, adjusted, or inconsistent (`negative_elapsed`,
`inverted_window`, `missing_timestamp`, `future_start`, `stale_anchor`).
Span-only stages never feed it.
- Buckets are 12 round edges (`0.1 0.25 0.5 1 2.5 5 10 30 60 300 1800
3600`).
- Moves the body of the `for` loop in `StartGeneration` into
`runGenerationStep` so the step has a function boundary for a deferred
stage end. Control flow and error wrapping are unchanged.

<details>
<summary>Series cost (approximate, whole stack)</summary>

A histogram label set is 12 edges + `+Inf` + `_sum` + `_count`: 15
series. Per replica, each `chat_kind` costs about 150 stage series and 4
turn outcome counters; each `(chat_kind, model)` costs 30 (2 stages x
15). Both chat kinds on `M` models: about `308 + 60*M` per replica, plus
background-scope stage series.

Dogfood `main`, 3 replicas: ~3.4k `coderd_*` series per replica (~10.1k
total).

| Models used per replica | Added series (3 replicas) | vs. baseline |
|---:|---:|---:|
| 1 | ~1.1k | +11% |
| 5 | ~1.8k | +18% |
| 10 | ~2.7k | +27% |
| 35 | ~7.2k | +72% |

Most deployments run four or fewer models, so a typical replica adds
roughly 370 to 550 series. The experimental turn accounting in #29772
would add 88 more per replica (11 categories x 4 outcomes x 2 chat
kinds). Series appear lazily and do not grow with organizations.

</details>

<details>
<summary>Review decisions</summary>

- Experiment gate only; no deployment flag or metric levels. If the
experiment graduates, the families become always on like the other chatd
families; a flag is added then only if dogfood shows a cardinality
problem.
- Organization is on neither metrics nor spans, and `effort` is
span-only; `provider_type` (configured type), not the wire `provider`,
is the model-family label.
- `time_to_first_token` is not on the model family; per-model TTFT is
`ttft_seconds`, which observes the same measured window.
- The model family records only turn-scoped stages.
- Nil guards are kept so a nil or zero tracer is always safe to call.
- HTTP attribute keys stay snake_case like the rest of the chatd span
attributes.

</details>

Supersedes #28968 and #28969.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
2026-09-30 11:53:57 -05:00
Mathias Fredriksson fa5bac6e77 feat: send tool call IDs and cancel unresolved calls on interrupt (#29954)
Retries and replica takeovers ran tool calls again, and an interrupt
recorded the same generic result whether a call had finished, was
still running, or never started.

chatd now derives a tool call ID from the chat, the committed
assistant message, and the provider tool call ID, and sends it with
execute starts, edit_files, and write_file. Every replica and retry
computes the same ID, so the agent returns the first result instead
of running the tool again, and a retried foreground execute waits
only for the rest of its timeout.

On interrupt, chatd cancels the unresolved calls of those tools on
the agent before committing, within 30 seconds, and records what
happened: "tool call was canceled before it ran; no changes were
made", the output of a command canceled while running, or the real
result. The UI shows a canceled command with the existing killed
marker. If the cancel fails, including an old agent's 404, the result
stays the generic interrupted text.

Calls whose provider tool call ID is empty or repeats in the message
get no ID and behave as before.

Fixes CODAGT-757
2026-09-30 14:31:52 +03:00
Michael Suchacz 5a997c471a fix(coderd/x/chatd): tell the model when output hit the token limit mid tool call (#30120)
When an Anthropic response hits the output token limit while the model
is writing a tool call's input, chatd gets `FinishReasonLength` with a
cut-off call. It handled both shapes of that call badly:

- If the provider closed the `tool_use` block, the call carried
truncated JSON and got the generic malformed-input result, which invites
the model to resend the same oversized input and hit the limit again.
- If the block never closed, fantasy emitted no tool call, chatloop
discarded the dangling tool input, and the turn ended with only
reasoning and no explanation.

On a length finish, `GenerateAssistant` now gives each cut-off local
call an error result: "This tool call was not executed because the
response reached the output token limit (N tokens) while writing its
input. Split the content into smaller tool calls." The token count is
included when the call has a configured output limit. A tool input that
started streaming but never became a call first gets a stand-in call
with empty input. The call persists with nil args and replays as `{}`,
so the provider receives a valid `tool_use` and `tool_result` pair, and
generation continues with another step.

With lifecycle hooks enabled, `pre_tool_use` admission skips calls the
step already resolved, so the hook is not dispatched for them and no
second result is added. The duplicate tool ID check still runs on the
full batch, because a resolved call still occupies its ID.

Independent of #30119, which stops the AI Gateway from failing these
streams before chatd sees them.

## Testing

-
`TestGenerateAssistant_SilentNoOutputFinish/LengthFinishResolvesTruncatedToolCalls`
(truncated tool call, dangling tool input): no error, reasoning kept,
and each call paired with the output-limit result.
- `TestPreToolUseHookOutputLimitTruncatedToolCallResolvedOnce` with
hooks enabled, for truncated and unfinished input: no hook dispatch, one
stored error result per call, a second model request, and a replayed
request that pairs `tool_use` input `{}` with its `tool_result`.
`DuplicateIDFailsClosed` checks that a reused ID still fails the chat
with `duplicate tool use ID`.
- Red-green: with `chatloop.go` and `generation.go` restored to the base
code, all four truncation cases fail. `DuplicateIDFailsClosed` passes on
the base code by design and fails on `d02bf4aa89b`, which dropped
resolved calls before the duplicate check.
- `go test ./coderd/x/chatd/... -count=1` passes, and lint on the
touched packages reports no issues. Local pre-push checks were skipped
per standing preference; PR CI covers them.
- Remote UAT: not run yet.

## Review record

- Parent review (Xum) of `d02bf4aa89b`: skipping resolved calls inside
`chathooks.PendingToolCalls` hid them from the duplicate tool ID check
in `admitStepToolCalls`. Fixed in `745f722165b`.
- Codex: not requested yet.

> Xum, an AI coding agent, prepared this PR on behalf of @ibetitsmike.
2026-09-30 12:20:03 +02:00
Michael Suchacz fd5b37576a feat(coderd): record provider response IDs on chat messages (#29763)
Chat messages could only be linked to AI Gateway records per chat, so
nothing recorded which model response produced a given message.

fantasy surfaces the response ID the model endpoint returns on the
stream's finish part: the Anthropic Messages ID as the part ID, and the
OpenAI Responses ID in provider metadata. chatd now stores that ID in
the existing `chat_messages.provider_response_id` column for assistant
steps and the compaction summary row. No migration and no aibridge
changes.

What the stored value is depends on the route:

| Route | Stored value | Joins to |
|---|---|---|
| Anthropic Messages API through the AI Gateway (Anthropic and Bedrock
Claude) | The gateway's interception ID. The gateway replaces the
message ID because injected MCP tool loops can fold several upstream
messages into one response. | `aibridge_interceptions.id` |
| OpenAI Responses API through the AI Gateway | The upstream `resp_` ID,
passed through unchanged | `provider_response_id` on
`aibridge_token_usages`, `aibridge_user_prompts`, and
`aibridge_tool_usages` |
| Direct to the provider | The provider's own ID (`msg_`, `resp_`) |
Nothing in the gateway tables |

To resolve messages to interceptions, then read usage by
`interception_id`:

```sql
SELECT cm.id AS chat_message_id, i.id AS interception_id
FROM chat_messages cm
JOIN aibridge_interceptions i ON i.id::text = cm.provider_response_id
UNION
SELECT cm.id, tu.interception_id
FROM chat_messages cm
JOIN aibridge_token_usages tu ON tu.provider_response_id = cm.provider_response_id;
```

Not covered, and left NULL:
- Partial rows saved on interrupt, because fantasy only reports the ID
in the finish part.
- Chat Completions providers (OpenAI-compatible, OpenRouter, Vercel),
because fantasy does not surface the chunk ID there.
- User, tool, and system rows, and all existing messages.

<details>
<summary>Delivery record</summary>

- `96a896c` replaced the earlier interception-link design (`59fa58f`,
`7f93c18`, `9d04ccd`) in place: the aibridge hook, the
`aibridge_interception_id` column and migration, the message part buffer
tracking, and the `ARCHITECTURE.md` section are all removed. The net
diff against the base touches only `coderd/database` and
`coderd/x/chatd`.
- Local validation on `96a896c`: pre-commit hook green (gen, fmt, lint,
build); `go test ./coderd/x/chatd/... ./coderd/database/` green.
Red-green: removing the finish-part capture fails both new end-to-end
tests (`TestActiveServer_PersistsAnthropicMessageID`,
`TestOpenAIResponsesPersistsProviderResponseID`). CI on `96a896c`:
green.
- Remote UAT round 2 on `96a896c`
([chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)):
**FAIL** against this description's earlier claim that every stored ID
joins the gateway usage rows directly. Anthropic rows routed through the
gateway stored the interception ID, so 0 of 18 joined usage rows
directly, while every checked Anthropic row equaled
`aibridge_interceptions.id`. Both OpenAI Responses rows joined directly.
Interrupted partial rows, Chat Completions rows, and user and tool rows
were NULL, with no empty strings or duplicate IDs. Multi-step turns,
manual compaction, edit and regenerate, a provider switch, and a retried
429 behaved as expected. One unattributed anomaly: a single duplicate
user row after interrupt and continue, which did not reproduce and is
outside this diff.
- `2e889ba` corrects the doc comment on
`PersistedStep.ProviderResponseID` and this description to match;
behavior is unchanged. Pre-commit hook and CI green.
- Remote UAT round 3 on `2e889ba` against the corrected contract ([same
chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)):
**PASS**. The query above resolved all 40 stamped rows to exactly one
interception from the same chat, with none unresolved. Reading usage by
interception recovered every usage row for 38 of them; the last two
resolved but their usage rows were not counted. Fresh Anthropic and
Responses calls, a seven-step tool turn, manual compaction, an Anthropic
to Responses to Anthropic switch in one chat, and a UI reload behaved as
described, and the NULL cases held. Three interrupt and continue
attempts did not reproduce the round-2 duplicate, so it stays
unattributed. Not tested: gateway-injected MCP tool loops (unreachable
from chatd, whose delegated requests skip the gateway's MCP proxy),
direct Anthropic `msg_` IDs, Bedrock, and threshold-triggered
compaction. Both rounds had a verified isolation proof and a clean
dogfood audit.
- Codex rounds 1 to 3 reviewed the old design. Codex round 4 on
`2e889ba`
([request](https://github.com/coder/coder/pull/29763#issuecomment-5796193371),
13:58Z): no major issues
([result](https://github.com/coder/coder/pull/29763#issuecomment-5796267231),
reviewed commit `2e889ba05c`). No unresolved review threads. Review
passes so far: 4 of 6.
- Final head `2e889ba` is the exact commit remote UAT round 3 and Codex
round 4 covered; required checks green.

</details>

> Xum updated this PR on behalf of @ibetitsmike.
2026-09-23 19:01:24 +02:00
Michael Suchacz e35cbe2a7f fix(coderd/x/chatd): stream compaction summaries (#28886)
Chats could get permanently stuck once their context crossed the
compaction threshold on adaptive-thinking Claude models: every
compaction attempt failed with `compaction produced an empty summary`,
and both automatic and manual compaction take the same path, so the chat
could never recover.

## Problem

`compactionSummaryCall` deliberately set `MaxOutputTokens = nil` because
the summary request was non-streaming and the Anthropic SDK rejects
non-streaming requests whose `max_tokens` implies a completion longer
than ten minutes. But a nil cap makes fantasy's Anthropic provider
silently default to 4096 output tokens, and adaptive-thinking Claude
models can spend that entire budget on a thinking block, leaving zero
summary text.

## Fix

- Stream the compaction summary request. Streaming lifts the Anthropic
SDK's non-streaming duration restriction, which was the only reason the
cap was nil'd.
- Keep the resolved output cap (32k default) on the summary call instead
of nil'ing it.
- When the summary comes back empty with truncation evidence (length
finish reason, or reasoning with no text), fail with `compaction summary
was truncated at the output token cap` instead of the generic
empty-summary error, so the next occurrence is attributable at a glance.

Unit tests cover text-block joining and the truncation classification;
the compaction e2e fixtures now serve streamed summaries and assert the
summary request streams with `max_tokens: 32000`. Dogfood UAT on a
deployment built from this branch verified manual compaction with Claude
Opus 5 (adaptive thinking, twice, with reload persistence) and an OpenAI
regression chat.

> 🤖 Opened by [Xum](https://mux.coder.com) on behalf of @ibetitsmike.
2026-09-21 21:28:45 +02:00
Michael SuchaczandPaul Frybarger 19e2b0415a fix(coderd/x/chatd): preserve tool definitions during compaction (#29436)
Supersedes #28883 by @pfrybar. His commit is carried with authorship
intact (rebased onto main as 32d261f94f); the second commit adds the
overflow fallback found while testing it.

## Problem

Compaction rebuilt the summary request without the conversation's tool
definitions, so the summary prompt no longer shared the turn's prefix
and providers could not reuse the cached context (measurements in
#28883).

Sending the tools with the summary makes that request at least as large
as the generation it summarizes. At a model's real context window
(gpt-3.5-turbo-0125, 16385 tokens) the summary request was rejected with
`Your input exceeds the context window of this model.` on the same
overflow compaction exists to recover from. The chat stayed in `error`,
and both manual `/compact` and the automatic retrigger failed the same
way, so the chat was stuck.

## Fix

`generateCompactionSummary` sends the prepared tool definitions with the
summary request. When the provider rejects that request for size, it
regenerates the summary once without tools, which is the request shape
compaction used before and still fits whenever the history alone does.
The context-window check reuses fantasy's classification and also
matches bad-request wordings fantasy does not parse (OpenAI Responses
API, Anthropic, `context_length_exceeded`).

`BuildToolDefinitions` moves into its own file in `chatloop` so the turn
and the summary share one construction path.

On Anthropic the summary prompt now carries the same `cache_control`
breakpoints as the turn; without them Anthropic never reads the cache,
so the shared tool and history prefix would have been pure cost there. A
compaction override that resolves to a different model shares no prompt
cache and may not support the chat model's provider-native tools, so its
summary request carries no tool definitions; an override resolving to
the chat model itself keeps them. HTTP 413 counts as a size rejection
for the tool-less retry.

## Testing

- Unit:
`TestGenerateCompactionSummaryRetriesWithoutToolsWhenContextTooLarge`,
`TestGenerateCompactionSummaryDoesNotRetryOtherFailures`,
`TestGenerateCompactionSummaryAppliesAnthropicPromptCaching`,
`TestIsContextTooLargeError`, `TestBuildToolDefinitions*`.
- End to end: `TestActiveServer_ManualCompaction/retries the summary
without tools when the prompt exceeds the context window` asserts two
summary requests (with tools, then none), the chat returns to `waiting`
with `last_error` cleared, and the summary is persisted.
- Red-green: with the retry disabled the unit test fails on the returned
provider error and the e2e chat never leaves `error`; with the phrase
matching disabled the Responses, Anthropic, and error-code classifier
cases fail.

## Review record

- Local standards and spec review at 99f7e80a0a9 (pre-rebase content of
8799db2f0e): switched `errors.As` to `errors.AsType`, removed two
comments restating test names, trimmed the phrase list to
provider-attested wordings. Rebase onto main resolved one test-file
conflict (both sides deleted different tests) and inlined the
`chattest.AnthropicErrorResponse` helper that main removed in #29008.
- Codex round 1 on 8799db2f0e: one P1, parent tool definitions sent to
a compaction override on a different provider. Mechanism verified
partially (fantasy skips foreign provider-defined tools with a warning;
`web_search` translates across providers and schemas can be rejected).
Fixed in a7fa7a61e4 by dropping tool definitions when the override is a
separate provider instance, with a red-green case in
`TestActiveServer_CompactionModelOverride`. Thread resolved.
- Codex round 2 on a7fa7a61e4: P1 override tools should drop for any
different model (agreed, guard replaced by an unconditional drop); P2
HTTP 413 should take the tool-less retry (agreed); P2 Anthropic summary
request had no cache breakpoints so the tools bought no cache hit
(confirmed in code, matches earlier dogfood `cache_read=0`). All three
fixed in e7bd72390c with red-green unit and e2e assertions. Threads
resolved.
- Codex round 3 on e7bd72390c: P2 keep tools when the override resolves
to the chat model (agreed, fixed in 6d8d8d6f71 with a red-green e2e
case); P1 missing ARCHITECTURE.md TODO items for the changed compaction
flow (agreed, added in 6d8d8d6f71 per the chatd rule); P2 summary tool
set may differ from the triggering request after a `find_tools`
activation (mechanism confirmed, declined as a cache-efficiency edge
case with the overflow path covered by the retry). Threads resolved.
- Codex round 4 on 6d8d8d6f71: "Didn't find any major issues", reviewed
commit `6d8d8d6f71`, zero unresolved threads. CI green on the same head
(Storybook rerun once after Pixel returned HTTP 500 while uploading
snapshots; this branch has no `site/` changes).

> Xum, an AI coding agent, prepared this PR on behalf of @ibetitsmike.

---------

Co-authored-by: Paul Frybarger <pfrybar@frymail.net>
2026-09-18 23:19:00 +02:00
Ethan 77479408b4 feat: make the Coder Agents stream silence timeout configurable (#29394)
The `chatloop` stream silence guard cancels and retries a model attempt
after `10m` of silence, and it was hard-coded. It's armed before the
request opens, so it also bounds time to first token, and slow local
models (e.g. `llama-server` on a large prompt) can spend longer than
that on prompt processing alone, meaning every attempt just gets
cancelled and retried.

This PR adds `--chat-stream-silence-timeout` /
`CODER_CHAT_STREAM_SILENCE_TIMEOUT` (default `10m`) and passes it to
both the main loop and the advisor. Setting it to `0` disables the guard
entirely, and anything above `24h` is rejected, which also keeps the
watchdog arithmetic below from overflowing.

Whilst doing this I ran into the runner's `15m` task watchdog, which
kept counting during an open stream. That silently cancelled any healthy
stream longer than `15m`, and would've stopped a silence timeout above
`15m` from ever firing. To fix, the silence guard now kicks the watchdog
out to `max(15m, silence + 5m)` whenever it arms or sees a part. Silent
streams still fail through the guard, healthy streams can run for as
long as they like, and hang detection is unchanged everywhere else. With
the guard disabled there's nothing to kick against, so arming it just
stops the watchdog for the rest of the attempt. That does mean the DB
write after the stream isn't bounded either when `0` is set; I think
that's fine for an opt-in setting, and a user can still stop the chat
from the UI if it wedges there.

I did consider removing the guard entirely, but the provider HTTP
clients have no timeout, so a provider that holds the connection open
would only ever be caught as a generic task hang, which is what #25782
was fixing.
2026-09-18 15:24:15 +10:00
Michael Suchacz 26e4429362 test: fold near-identical test bodies into tables across eight packages (#29463)
Folds groups of near-identical test bodies (same parent, same parallel
flag, differing only in inputs and expected values) into table-driven
tests in eight packages: `coderd/templatebuilder`, `scripts/develop`,
`agent/agentfiles`, `coderd/x/chatd/chatloop`,
`coderd/x/chatd/chatdebug`, `enterprise/aibridgeproxyd`, `cli`, and
`coderd/database/db2sdk`. One commit per package.

Every case keeps its previous subtest name, `t.Parallel()` placement is
unchanged, and the set of `=== RUN` names is identical except where
standalone `TestX_Y` funcs became `TestX/Y` subtests (chatloop: 9 funcs
into `TestExecuteLocalTools_BatchRuntime`; chatdebug: 15 funcs into two
tables). Members whose bodies differ in more than data stayed as they
were (listed per package in the commit messages). In `agent/agentfiles`
the four `TestFuzzyReplace_*` funcs were already tables, so the shared
request plumbing was hoisted into one helper instead of merging the
funcs and losing their per-rule docs. Net -1035 lines across 9 test
files; runtime is unchanged (all sub-100ms unit tests).

| Package | Diff | Funcs removed | Subtests | RUN lines |
|---|---|---|---|---|
| coderd/templatebuilder | +237 / -291 | 0 | 31 -> 31 | 33 -> 33 |
| scripts/develop | +194 / -246 | 0 | 27 -> 27 | 28 -> 28 |
| agent/agentfiles | +117 / -260 | 0 | unchanged | 58 -> 58 |
| coderd/x/chatd/chatloop | +224 / -306 | 9 | 3 -> 12 | 14 -> 15 |
| coderd/x/chatd/chatdebug | +251 / -537 | 13 | 0 -> 15 | 42 -> 44 |
| enterprise/aibridgeproxyd | +287 / -419 | 0 | 31 -> 31 | 32 -> 32 |
| cli | +143 / -326 | 0 | 15 -> 15 | 16 -> 16 |
| coderd/database/db2sdk | +130 / -233 | 0 | 9 -> 9 | 10 -> 10 |

## Validation

- Per package: `go test -count=1 -tags=testsmallbatch -parallel=8
./<pkg>/ -run <regex> -v` before and after, with the `=== RUN` name sets
diffed
- `go test -race -count=2 -shuffle=on` on each converted test: pass
- `golangci-lint run ./<pkg>/...` with a fresh cache: 0 issues in all
eight packages
- Repository pre-commit hook (including lint/go) passed on every commit

Part of the test-suite speedup series (see #29457); independent of the
other PRs.

## Remote validation

Remote run on a dogfood workspace (separate compute), chat
https://dogfood.cdr.dev/agents/5bbe73d0-866a-4841-9cc3-c5e5b4d97e44,
tested head `d7c55d6b0e8` against base `9d973372774`: all eight packages
pass on the head and under `-race -count=3 -shuffle=on`; package times
are sub-second on both sides (unchanged). `=== RUN` name sets: chatloop
15 to 16 lines with all nine `TestExecuteLocalTools_<X>` leaves
surviving as `TestExecuteLocalTools_BatchRuntime/<X>`; chatdebug 42 to
44; `aibridgeproxyd` `TestNew` 32 to 32, byte-identical sorted list. One
correction to the text above: seven chatdebug leaves are renamed, not
preserved verbatim (`TestRecordingTransport_*UnknownLength*` became
`TestRecordingTransport_UnknownLengthClose/<name without
"UnknownLength">`), so `-run` filters keyed on those old names need
updating. Endorsed verdict: PASS.


> Xum (an AI agent) authored this PR on behalf of @ibetitsmike.
2026-09-17 19:26:18 +02:00
Michael Suchacz 8397680687 fix: keep text and image together in MCP tool results (#29332)
## Problem

MCP tool results that mix text and an image (Playwright's default
`browser_take_screenshot` returns `[text, image]`) were reduced to text
only in both MCP conversion paths, so screenshots never reached the
model. The generic tool renderer also showed media results as a JSON
dump of base64.

## Fix

- `mcpclient` and `chattool`: keep all text plus the first usable binary
item. Error flags survive empty and MIME-less results. Only one binary
payload is represented; additional images are still dropped.
- `chatloop`: at execution time, cap tool media at the 10 MiB attachment
limit and correct or reject declared image types by byte signature
(sniffing, not decoding).
- `chatd` prompt build and `chatprovider`: at generation and compaction,
replace media the destination provider cannot accept with the text plus
an omission note (Anthropic/Bedrock: JPEG, PNG, GIF, WebP under the
inline cap; OpenAI/Azure and Google: their image allowlists, applying
the configured vendor's rules behind `openai-compat`). Persisted history
is unchanged, so the UI keeps the payload and provider switches keep
working.
- `site`: results flagged `is_media` render through `ToolResultMedia`
(image with the existing lightbox, or a short notice for non-image
media, plus the text). Unflagged media-shaped JSON stays in the JSON
viewer.
- `go.mod`: pin [coder/fantasy
`9a3598480a71`](https://github.com/coder/fantasy/commit/9a3598480a71e45901bfcfbf566ffa42c6bbdd63),
main's `a511353f83eb` plus the batch-order fix `460d4fdd113b` (Chat
Completions and compatible transports emit every tool result before
synthetic user images). The fix lives on
`mike/mcp-tool-media-batch-order`; no upstream PR yet.

## Validation

Round 3 dogfood UAT on this head, `522bd76a58a`: **PASS**
([chat](https://dogfood.cdr.dev/agents/5e617636-c6f2-46fc-96e5-97ae35ed822e),
real Playwright MCP, Claude Haiku 4.5 and GPT-4o mini through the AI
gateway). Verified in the UI and on the wire: image plus text render
together, the lightbox opens and closes, both survive a reload; audio
shows a notice instead of base64; media-shaped JSON stays in the JSON
viewer; BMP on OpenAI Responses and Chat Completions reaches the model
as text plus an omission note; SVG and PNG histories switch from
Anthropic to OpenAI; cross-provider compaction through the organization
override works; a result over 10 MiB persists only text plus a note, and
a 5.5 MiB PNG is substituted for Anthropic yet still renders in the UI.
Google is not routable on the dogfood gateway (404), so it was not
exercised. The remote agent's own FAIL was a probe artifact (the code
viewer renders in a shadow root) and was overturned on its screenshots
by the independent runner. Rounds 1 and 2 (`65db84a`, `5001de7`) covered
the same API surface earlier.

Known limits: signature sniffing does not catch a truncated PNG with a
valid header (providers return 400 for it), and only the first binary
item of a result is kept. Pre-existing and out of scope: chatd opens a
new MCP session per generation action, so Playwright's default
per-session browser loses page state between steps (UAT used
`--cdp-endpoint`).

Reviews:

| Review | Head | Outcome |
|---|---|---|
| Codex 1 to 5 | `472914f` to `bcf4c72` | 16 findings (empty binary
promoted over text, no provider caps, SVG on Anthropic, provider-switch
gap, Bedrock labels, unbounded media, unvalidated bytes, renderer
gating, lightbox test), all fixed |
| Codex 6, 7 | `8a9a9e0`, `5001de7` | no major issues; one Responses
audio concern [refuted with a live
probe](https://github.com/coder/coder/pull/29332#discussion_r4018514197)
|
|
[Codex](https://github.com/coder/coder/pull/29332#pullrequestreview-5226106293)
| `83e1a10` | two P2s refuted with serializer captures
([MIME](https://github.com/coder/coder/pull/29332#discussion_r4029188332),
[advisor](https://github.com/coder/coder/pull/29332#discussion_r4029160781));
[reassessment](https://github.com/coder/coder/pull/29332#issuecomment-5702295886)
clean |
| Codex | `522bd76` | [no major
issues](https://github.com/coder/coder/pull/29332#issuecomment-5713663081)
on the rewritten tests |
| @DanielleMaywood | `472914f`, `5001de7` | renderer restructure and
explicit audio mapping done; lazy slice cloning removed; two threads
left open for her re-check |

Local pre-push hooks were skipped per Mike's instruction; CI is the
gate.

> Xum acted on Mike's (@ibetitsmike) behalf to create this PR.
2026-09-17 15:09:19 +02:00
Michael Suchacz 0362e5386c fix(coderd/x/chatd/chatloop): tell the model to gather context before retrying an exclusive tool (#29281)
When the model batches `advisor` (an exclusive tool) with other tool
calls, every call in the batch is rejected and the `advisor` result only
says to retry with `advisor` alone. That nudges the model toward
re-calling `advisor` immediately, even when the sibling calls were there
to gather context for it.

Extend the message so the model is told to run the other tools first
when it still needs their output, then retry with only the exclusive
tool call.

Generated by Coder Agents on behalf of @ibetitsmike.
2026-09-15 17:02:48 +02:00
Michael Suchacz 46c904d181 fix: show estimated context usage after compaction (#28984)
After compaction, the context gauge showed the unused-chat placeholder
until the next response. It now shows a clearly labeled estimate based
on the compacted summary, keeps that estimate after reload, and uses the
selected chat model’s context limit. The next response replaces it with
measured usage.

The estimate stays separate from provider-reported token accounting.
Remote dogfood UAT passed the core workflow on
cb18cbdac596231acf821cd612c5a8c5d1db1c7b. It has not been repeated on
the current head, 8cfc3da83c.

> Xum acted on behalf of @ibetitsmike.

<details>
<summary>Review follow-up and validation</summary>

- DanielleMaywood's review of df01d8b536: simplified both nested label
ternaries and removed the unrelated mock handlers in 8cfc3da83c.
Replies: [accessible
label](https://github.com/coder/coder/pull/28984#discussion_r4003806647),
[panel
label](https://github.com/coder/coder/pull/28984#discussion_r4003807264),
[mock
scope](https://github.com/coder/coder/pull/28984#discussion_r4003807845).
All three threads resolved.
- Local visual review, task ecb1c01d70: six states of the label refactor
inspected with no regressions found. Frontend self-review: FE1 through
FE10 passed.
- Passed frontend format, check, and lint; 50 targeted unit tests; 19
stories; 64 label parity comparisons; and the full pre-commit and
pre-push hooks, including Go tests, frontend tests, and the production
frontend build.
- The pre-push run used a process-local LD_PRELOAD shim to refuse
loopback port 3000, preventing unmocked test requests from reaching the
unrelated host service. No hooks were disabled and the host service was
not changed.
- [Documentation analysis is
blocked](https://github.com/coder/coder/actions/runs/34827361213/job/103922480177):
its Coder workspace is dormant, so the review did not run. The workspace
needs reactivation before retrying that check.

</details>
2026-09-14 20:50:12 +02:00
Michael Suchacz 680b2f59cd refactor: remove obsolete chatd code and tests (#29228)
Chatd still carries an unreachable interruption flush path, unused state
and options, an uncalled workspace MCP cache invalidation callback, and
a stream-buffer-drop counter that production never increments. Remove
these paths and their dedicated assertions, and update the generated
metrics inventory and documentation.

Consolidate duplicate publication and rollback tests and move fixture
writers and test-only inspection helpers into test files. Retain the
interruption persistence, state-machine matrix, legacy message format,
and provider compatibility coverage.
2026-09-14 10:51:22 +02:00
Garrett Delfosse c12355d6e5 fix(coderd/x/chatd): carry unanswered user messages through compaction verbatim (#28645)
Fixes
[CODAGT-737](https://linear.app/codercom/issue/CODAGT-737/compaction-can-treat-the-last-user-message-as-an-assistant-message)

## Problem

Auto-compaction can trigger on a turn whose last history message is a
fresh, unanswered **user** message. Two things go wrong:

1. The summary instruction is appended as another user message.
Providers that merge consecutive same-role messages (verified against
fantasy's Anthropic conversion) fuse it with the user's message, and the
summarizer misattributes the user's instruction as assistant intent.
2. After compaction the original user row falls outside the prompt
window, so the instruction survives only through summary fidelity.

## Fix

Carry unanswered trailing user messages through compaction verbatim
instead of summarizing them. All logic lives in `chatd`; `chatloop` is
unchanged apart from prompt wording.

- **Prepare** (`generation_preparer.go`): `pendingUserSegmentStart`
locates the trailing user rows on the persisted rows, before
provider-switch sanitization (which can drop the assistant row
separating them from an already-answered message). Head and tail are
converted separately so the summarizer input excludes the tail exactly.
Falls back to summarizing everything when no usable segment exists.
- **Commit** (`buildCompactionMessages`): replays the tail after the
boundary trio as uncompressed, model-only user rows. The existing prompt
window query includes them; no SQL, UI, or turn-guard changes. Replay
rows carry no usage, so compaction cannot immediately re-fire.
- **Prompt** (`chatloop/compaction.go`): the summary prefix no longer
claims the assistant "was actively working", which is false in this
case. The summarizer prompt is unchanged.
- `ARCHITECTURE.md` compaction section updated.

## Testing

`go test ./coderd/x/chatd/...` green. New:
`TestBuildCompactionMessages_ReplaysPendingUserMessages`,
`TestPendingUserSegmentStart`,
`TestPendingUserSegmentConversionEquivalence`,
`TestPendingUserSegmentDerivedBeforeProviderSwitchSanitization`,
`TestDecisionGeneratesAfterCompactionWithReplayedPendingUser`.

<details>
<summary>Root-cause repro</summary>

Replaying the reported scenario through fantasy's Anthropic provider
against a capture server:

```
=== fantasy prompt (roles as chatd builds them) ===
[2] role=user      "You'll hit compaction now, we'll need to go through some fin..."
[3] role=user      "You are performing a context compaction. Summarize the conve..."

=== wire request to Anthropic ===
[2] role=user, 2 content block(s)
    block[0]: "You'll hit compaction now, we'll need to go through some final checks..."
    block[1]: "You are performing a context compaction. Summarize the conversation so..."
```

</details>

---

🤖 This PR was generated by Coder Agents on behalf of @f0ssel.
2026-09-08 09:31:42 -04:00
Michael Suchacz ac824cd235 chore(coderd): remove dead code from chatd (#29008)
Removes production code in `coderd/x/chatd` that nothing reaches, the
tests that only existed to exercise it, and option/config fields that
callers filled in but chatd never read. No runtime behaviour changes;
the only edits outside `coderd/x/chatd` drop the now-nonexistent fields
at their three call sites (`coderd/coderd.go`, `coderd/exp_chats.go`,
`coderd/coderdtest`).

Net: 42 files, +147 / -1065.

Verification method for every item: `golang.org/x/tools/cmd/deadcode`
(with and without `-test`) rooted at `cmd/coder`, `enterprise/cmd/coder`
and every package that imports chatd; a `go/packages` pass counting
`TypesInfo.Uses` per declared type, const, var, field and method across
the same roots, split into test and non-test use sites, plus a
read-vs-write split for struct fields; `rg` over the whole repo
including `site/src` for json tag names; then `go build ./...`, `go vet
./coderd/x/chatd/...`, `golangci-lint run ./coderd/x/chatd/... ./coderd/
./coderd/coderdtest/`, and `go test ./coderd/x/chatd/...` (all 20
packages) locally.

<details>
<summary>Evidence per deletion</summary>

| Removed | What it was | Why unused |
|---|---|---|
| `subagent.go` `resolveConfiguredModelOverride`, `parseModelOverride`,
`parsedModelOverride` (~135 lines) +
`TestResolveConfiguredModelOverride_*`, `TestParseModelOverride` |
raw-string model override resolver | Only callers were its two tests;
production resolves overrides through `resolveOrganizationModelOverride`
(`title_override.go`, `compaction_override.go`, `subagent.go`). |
| `chatd.go` `removeSkillIndexMessages`, `isSkillIndexMessage` +
`TestSkillIndexRefreshReplacesStaleAliases` | skill-index refresh
helpers | Zero production references; the test simulated a refresh path
no production code performs. |
| `chatd.go` `chatMessage`, `newChatMessage`, `withCreatedBy`,
`appendMessageFields`, `BuildSingleChatMessageInsertParams` (~90 lines)
| insert-params builder | Exported, but every caller was a `chatd_test`
file (6 sites). Replaced by a 25-line `singleChatMessageInsertParams`
helper in `chatd_helpers_test.go`. |
| `chatd.go` `Server.closeInflightAdmission` | one-line wrapper around
`inflightClosed.Store(true)` | Single call site (`Close`); inlined. |
| `chatd.go` `Server.workerID`, `instructionLookupTimeout`,
`pendingChatAcquireInterval`, `maxChatsPerAcquire`,
`chatHeartbeatInterval` | Server struct fields | Assigned in `New` and
never read anywhere (the values reach the worker through
`chatWorkerOptions`, not these fields). |
| `Config.InstructionLookupTimeout`, `homeInstructionLookupTimeout`,
`coderd.Options.ChatdInstructionLookupTimeout`,
`coderdtest.Options.ChatdInstructionLookupTimeout` | timeout plumbing |
The value only landed in the dead `Server.instructionLookupTimeout`; no
CLI flag sets it and nothing reads it. |
| `PromoteQueuedOptions.CreatedBy` | option field | Set by
`coderd/exp_chats.go` and three tests, never read in chatd (promotion
keeps the queued row's `created_by`). |
| `SubmitToolResultsOptions.DynamicTools` | option field | Set by
`coderd/exp_chats.go` (which re-read `chat.DynamicTools` just to pass
it) and three tests, never read in chatd. |
| `attachments.go` `buildAssistantPartsForPersist` (whole file) |
assistant part builder | Stale copy left by #26270; production uses
`buildAssistantParts` in `message_conversion.go`. Its six tests were
retargeted to the live function (now
`message_conversion_internal_test.go`) and a dead-copy assertion in
`subagent_internal_test.go` was dropped. |
| `chatloop.RunOptions` (94 lines) | old loop options struct | Only two
of 23 fields were ever populated, for one internal call to
`prepareMessagesForRequest`, which now takes `model, logger` directly
and no longer returns the ignored canonical slice.
`GenerateAssistantOptions` / `ExecuteLocalToolsOptions` are the live
types. |
| `chatloop.ErrDynamicToolCall`, `ErrStopAfterTool`, `PendingToolCall`,
`PersistedStep.PendingDynamicToolCalls` | dynamic-tool exit protocol |
Zero references outside their declarations (one stale comment in
`chatdebug/service.go` updated). |
| `chatloop.CompactionOptions.Persist`, `.OnError` | callbacks |
`Persist` had zero references; `OnError` was set only by two tests that
asserted it was never invoked. |
| `chatloop` `stepResult.shouldContinue` + its computation | flag |
Written once, never read. |
| `chatloop.StateWaiting` | metric label const | Zero references. |
| `generation.go` `generationOutcome.InsertedMessages`,
`runnerActionMessage` | outcome field | Built on every commit, never
read. |
| `chatstate` `ExecutionState.IsArchived`, `AllOwnershipStates`,
`OwnershipState`/`StateU`/`StateO`, `TransitionAcquire` | state-machine
leftovers | Zero references (the ownership consts became dead once
`AllOwnershipStates` went). |
| `chatstate` `ExecutionState.QueueNonEmpty` | predicate | Only
referenced by the test asserting its own table; the sibling `IsRunnable`
check in that test is kept. |
| `chatdebug.Attempt.RetryClassification`, `.RetryDelayMs` | json fields
| Never assigned anywhere, `omitempty`, and
`retry_classification`/`retry_delay_ms` do not appear in `site/src` or
`typesGenerated.ts`. |
| `chatdebug.RunContext.Kind`, `StepContext.HistoryTipMessageID`,
`RecorderOptions.Provider`/`.Model`, `stepHandle.opts`/`.usage`/`.err` |
context/handle fields | Written (from `chatd.go`,
`chatloop/compaction.go`, `modelcall.go`, `recorder.go`) and never read.
The DB run row keeps its own `kind` column. |
| `chatprovider.MergeProviderAPIKeys` +
`TestMergeProviderAPIKeys_PreservesProviderRegions` | key merge |
Superseded by `ResolveUserProviderKeys`, whose region test already
covers the same behaviour. |
| `chatprovider` `providerCreationError` | one-line `xerrors.Errorf`
wrapper | Single call site; inlined with identical message. |
| `chatretry.IsRetryable` | classifier wrapper | Zero references. |
| `computeruse.IsTool` + `TestComputerUseTool` | one-line forwarder to
`fantasyopenai.IsComputerUseTool` | No production caller; two test
assertions now call the fantasy function directly, and the dropped test
only asserted the library's own constructor. |
| `chatprompt.ToolResultToPart` | one-line forwarder to
`codersdk.ChatMessageToolResult` | Single call site; inlined. |
| `subagent_catalog.go` `formatSubagentDefinitions` | forwarder with
`nil` overrides | Single call site; inlined. |
| `worker.go` `chatWorker.chatWorkerID` | getter | Only tests called it;
they read `opts.WorkerID` directly now. |
| `chattest.AnthropicErrorResponse`, `OpenAIServerErrorResponse`,
`ChatMessageWithParts` (`messages.go` removed) | test fixtures | Zero
references. |
| `chattest` `AnthropicRequest.Options`, `OpenAIRequest.Options` |
`json:",inline"` map fields | `encoding/json` has no inline tag (the
TODO on each said so); never populated, never read. |

</details>

<details>
<summary>Considered and kept</summary>

- `chatstate.CreateChat` (test-only thin wrapper over `insertChat`,
production uses `CreateChatWithID`): documented package API with 25 test
call sites; not worth the churn.
- `chatprompt.MarshalContent` / `MarshalToolResult`: test-only, but they
deliberately produce legacy-format DB rows so the read path's legacy
handling stays covered.
- `chatstate.AllowedExecutionTransitionsFrom` / `AllowedInputStates` /
`AllowedExecutionTransitionOutputs`, `chatdebug.StepFromContext`,
`WithClock`, `WithStaleThreshold`, `WaitUntilIdleForTest`,
`PublishBuffer.BufferedChannels`, `chattool.NewWorkspaceMCPTool`: test
seams that tests actively use to reach otherwise unexported state.
- `AgentCapacityLimiter` interface (one production implementation):
tests inject fakes.
- `applyHookResultMessages` / `appendHookResultMessages`: a named pair
over one placement enum, both used.
- Nil-receiver guards on `*Metrics`, `*Result`, `*streamSyncPoller`,
`*runnerDebugTurn`: nil is a supported "disabled" value on those paths.
- Tests flagged by heuristics as assertion-light
(`TestNopMetrics_DoesNotPanic`, `TestStepHandleFinish_NilHandle`,
`TestRetry_OnRetryNilDoesNotPanic`, the `chatstate`
synthetic-cancellation tests that assert through helpers): they cover
real nil-safety or delegate assertions to helpers; kept.

</details>

<details>
<summary>Out-of-scope findings (not changed here)</summary>

- `chatdebug.Service.FinalizeStale` has had no production caller since
#26270 removed the background sweep loop; only its tests call it, and
comments in `chatd_debug.go` and `database/querier.go` still describe
the loop. Deleting it would erase the implementation a fix needs, so it
is left for a decision.
- `coderd_chatd_chats` (gauge) and
`coderd_chatd_stream_buffer_dropped_total` (counter) are registered and
documented in `docs/admin/integrations/prometheus.md` but never set or
incremented in production; `Metrics.RecordStreamBufferDropped` is
test-only. Removing them changes the exported metric set and the docs,
so they are left alone.

</details>

## Declined review findings

None yet.

> Xum (AI agent) opened this PR on behalf of @ibetitsmike.
2026-09-05 11:39:10 +02:00
Jaayden Halko b428c8feb6 feat(coderd): bill completed local tool batches in agent runtime (#28360)
**Stack**: #28359 → **#28360 (this PR)** → #28361 → #28362

## Summary

Completed local tool batches now count toward Coder Agent runtime
(`chat_messages.runtime_ms`, the source of truth for
`hb_agent_runtime_v1`), excluding sub-agent orchestration tools to avoid
double counting, and without multiplying runtime for parallel tool
calls.

Second of a four-PR stack splitting #28211. Stacked on #28359. Refs
CODAGT-928
(https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).

## Problem

Agent runtime previously measured only model invocation wall clock
(stream open to fully consumed). Time spent executing local tools
between steps, including file operations, terminal commands, workspace
provisioning, and MCP tools, was deliberately excluded, which
undercounts the product definition of "actively processing a task".

Naive inclusion has two hazards: a batch of parallel tool calls would
bill N windows for one wall-clock wait, and `wait_agent` would re-bill
child agents that already bill their own model and tool time.

## Fix

Each completed local tool batch bills one window: the union of the
billed tools' execution intervals, persisted as `runtime_ms` on a
dedicated usage record appended after the batch's tool-result rows. The
record is a tool-role message with `visibility='model'`, so it never
reaches the API, SSE, or clients, and prompt replay drops it because its
single internal `tool-batch-usage` part converts to no provider content.
Its content carries an audit payload (`billed_ms`, `billed_calls`) so
the billed window is inspectable after the fact; real tool-result rows
never carry batch-level runtime. Concurrent calls all start at batch
start, so 5 parallel 10s reads bill 10s, not 50s; serial calls
(`SerialToolCalls`) count only from their own launch, so unbilled waits
before them do not count.

Sub-agent orchestration tools (`spawn_agent`, `wait_agent`,
`message_agent`, `interrupt_agent`, `list_agents`,
`list_subagent_models`, plus the deprecated `close_agent` alias) never
extend the window: every chat, including children, applies the same
rules to its own runtime, so a parent's `wait_agent` window would double
count. A lone `wait_agent` bills 0 and appends no usage record;
`execute` 10s in parallel with `wait_agent` 60s bills 10s. A test pins
the unbilled set to the registered sub-agent tool catalog so they cannot
drift.

`GetTotalChatMessageRuntimeMsInRange` already sums `runtime_ms`
role-agnostically, so the usage record is picked up with no schema,
query, or cron changes.

Client-executed dynamic tools, external agents, parked/idle time, and
retry backoff remain unbilled. Interrupted batches still bill only the
model window; the rest of the stack adds partial-window billing via the
same usage record.

`ExecuteLocalToolsOptions` also gains an optional `ToolBillingRecorder`
(`RecordStart`/`RecordComplete`) observing per-occurrence dispatch-order
execution stamps. This PR wires no recorder; the next PR in the stack
connects it to the message part buffer for interrupt billing.

**NOTE**: Reported agent runtime (`hb_agent_runtime_v1`) increases from
deploy forward. There is no backfill and no feature flag.
2026-08-25 16:40:41 +07:00
Jaayden Halko d44aae37de refactor(coderd/x/chatd): return PersistedStep directly from ExecuteLocalTools (#28359)
**Stack**: **#28359 (this PR)** → #28360 → #28361 → #28362

`chatloop.ToolExecutionOutcome` only wrapped a `PersistedStep`, so
callers had to reach through `outcome.Step` for every field.
`ExecuteLocalTools` now returns the `PersistedStep` directly and the
wrapper type is deleted. No behavior change.

First of a four-PR stack that makes local tool execution count toward
Coder Agent runtime (splitting #28211). This refactor keeps the rename
noise out of the feature PRs.

Refs CODAGT-928
(https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).
2026-08-25 16:31:06 +07:00
Michael Suchacz a48aedd924 fix(coderd/x): surface Gemini malformed-function-call stream deaths as retryable errors (#28470)
## Problem

When Gemini's OpenAI-compatible endpoint rejects a model-generated
function call server-side, it ends the SSE stream cleanly with the
nonstandard finish reason `function_call_filter:
MALFORMED_FUNCTION_CALL` after streaming only thought summaries; the
rejected call never reaches the wire. chatd treated this as a normal
completion: fantasy maps the unrecognized finish reason to `unknown`,
the reasoning block never closes (the only non-thought delta is the
`</thought>` marker, which the transport seam strips to an empty string,
and the openaicompat hook only ends reasoning on a non-empty content
delta), so the step accumulates no content and the generation loop
finishes the turn as complete. The user sees the model think for ~40
seconds and then nothing: no assistant message, no `last_error`, chat
status `waiting`. Observed twice in a row in production on
gemini-3.7-flash, with "Resume" reproducing it identically.

## Fix

Two independent layers:

- `coderd/x/googleopenai`: the stream rewrite now converts any chunk
whose `finish_reason` starts with `function_call_filter` into an
OpenAI-style SSE `{"error": ...}` event embedding the raw reason.
openai-go turns error-bearing events into stream errors, so the failure
rides the existing stream-error path instead of ending the stream
cleanly.
- `coderd/x/chatd/chaterror`: classifies that injected error as
retryable (kind `generic`, provider `google`) with a clear user-facing
message, so the existing generation retry machinery re-runs the step and
persists a `last_error` if retries exhaust.
- `coderd/x/chatd/chatloop`: provider-agnostic guard: a step that
produced no user-visible content and no tool calls under a finish reason
of `unknown`, `error`, `other`, or `tool-calls` (a tool-calls finish
that delivered zero calls) now returns a retryable error instead of
silently completing the turn. `stop` and `length` finishes keep their
existing semantics.

Tests cover the seam rewrite (live-capture SSE shape plus standard
finish reason passthrough), the new classification, and the chatloop
guard (error cases plus preserved stop, length, text, and tool-call
behavior). Each layer was red-green verified independently.

Remote dogfood UAT ran against this exact commit: normal reasoning and
tool-call chats on a real model complete cleanly with no spurious guard
errors and no retry loops. The Google-side failure itself is not
deterministically triggerable against live Gemini and is owned by the
unit tests.

> Xum acted on Mike's (@ibetitsmike) behalf.

<!-- xum-attribution: model=claude-opus, thinking=enabled -->
2026-08-24 17:09:00 +02:00
Michael Suchacz 9168dbf4d3 refactor(coderd/x/chatd): unify LLM model-call construction behind a resolver (#28145)
Centralizes every LLM client build in chatd behind one resolver and
makes all model calls behave the same, instead of preserving per-flow
quirks.

## Why

`chatd` built LLM clients in five places (standard turn, compaction
override, quickgen, title override, advisor), each repeating config
parsing, route resolution, client construction, and debug wrapping with
slightly different behavior. An earlier iteration of this PR centralized
construction but preserved every historical quirk behind
purpose-specific spec constructors and policy flags. Review concluded
the quirks themselves were the problem.

## What changed

- `Server.resolveModelCall` (`modelcall.go`) is the single pipeline from
a `modelCallSpec` to a ready client plus call metadata. Callers build
spec literals inline; the only spec fields are data: model source (chat
config, explicit config row, or fixed provider/model pair for computer
use), requested reasoning effort, chatd-scoped routing, and build
options.
- Uniform behavior for every call:
- Provider options are always derived. Whole-chat summaries, turn status
labels, and chat-model compaction summaries previously sent none; they
now follow the config like everything else.
- `MaxOutputTokens` always defaults to 32k when the config leaves it
unset. The compaction summary call nulls the cap after copying the
template because it is non-streaming and the Anthropic SDK rejects
non-streaming requests whose cap implies a completion over ten minutes.
- Debug recording always wraps the client when chat debug is enabled,
labeled with the resolved provider/model. The debug-rebuild paths
(`manualTitleDebugSpec`, `quickgenDebugSpec`) are deleted; debug helpers
now only create and finalize run records, since run association flows
through the generate context, not the model.
- Clients are always constructed with the configured model string and
unmutated route, so AI Gateway validation sees the name exactly as
configured. The resolved identity feeds metadata, logs, and debug labels
only.
- `resolvedModelCall.newCall` / `newObjectCall` are the only production
constructors of `fantasy.Call` / `fantasy.ObjectCall`; object calls now
carry the config's sampling parameters.
- Net: -240 lines.

## Validation

- Full `./coderd/x/chatd/...` suite green (PostgreSQL-backed).
- Red-green toggles verified for the three key unifications (option
derivation, token default, summary cap removal).
- ARCHITECTURE.md: prose the refactor made stale is deleted; no new
resolver documentation is added.

> Mux acted on Mike's behalf for this PR.
2026-08-20 20:54:52 +00:00
Danielle Maywood 5480299cef fix: serialize empty MCP tool properties as {} not null for OpenAI (#28380)
## Summary

Fixed a bug where MCP tools with an empty `properties` schema had their
tool definitions sent to OpenAI with `"properties": null`, causing
OpenAI to reject the request.

## Problem

When an MCP server reports a tool with an empty input schema, e.g.:

```json
{
  "type": "object",
  "properties": {},
  "required": []
}
```

Coder was forwarding the tool definition to OpenAI as:

```json
{
  "type": "object",
  "properties": null
}
```

OpenAI's function-calling API rejects `properties: null`, making these
tools unusable with OpenAI models.

## Root cause

The empty `properties` object collapsed to Go `nil` at two layers:

1. `agent/x/agentmcp/manager.go` `toolInputSchemaMap` stripped the
present-but-empty `"properties": {}` (guarded by `len(properties) > 0`),
so the wire copy lost it entirely.
2. `coderd/x/chatd/chatloop/chatloop.go` `buildToolDefinitions` embedded
`info.Parameters` directly into the input schema. When nil,
`encoding/json` marshals it as `null`.

## Fix

- `buildToolDefinitions` now substitutes an empty object when
`Parameters` is nil, mirroring the existing `required` guard (the sink,
protects all tool sources).
- `toolInputSchemaMap` now preserves an empty `properties` object
instead of dropping it (the upstream producer).

## Tests

Added regression tests (failing before, passing after) covering both
layers and asserting the schema serializes to
`{"type":"object","properties":{}}`:

- `TestBuildToolDefinitionsNilPropertiesBecomesEmptyObject`
- `TestToolInputSchemaMapPreservesEmptyProperties`
-
`TestWorkspaceMCPToolInfosFromResources/EmptyPropertiesYieldsEmptySchema`

---

*This PR was generated by Coder Agents.*
2026-08-20 15:56:58 +02:00
Michael Suchacz 7724ee281a feat: defer MCP tool schemas behind a find_tools search (#28225)
## Summary

When the `mcp-tool-search` experiment is enabled, chatd stops inlining
connected MCP tool schemas into every generation. It instead exposes a
built-in `find_tools` tool whose description carries a compact catalog
of the deferred tools, and only ships full JSON schemas for tools the
model has activated by searching or by calling them directly.

Closes [CODAGT-760](https://linear.app/coder/issue/CODAGT-760).

## Problem

Tool-heavy agent configurations (GitHub, Linear, Notion, and dev-tooling
MCP servers) inline over 100k tokens of tool schema definitions into
every generation. Initial uncached requests reached ~216k tokens with
time-to-first-token close to nine minutes, while the model typically
invokes only a handful of tools per turn.

## How it works

- `decideMCPToolSearch` defers external and workspace `.mcp.json` MCP
tools whenever the experiment is enabled. Native, dynamic, provider,
skill, and transport tools are never deferred.
- `find_tools` embeds a server-grouped catalog in its tool description
(degrading to names-only, then counts-only, then a constant-size summary
past a context-scaled size cap) and scores keyword matches across tool
names, descriptions, parameter schemas, and server metadata. Queries can
scope to one server with a `server:` prefix, and exact `names` arguments
always activate.
- Activation state is ephemeral: it is re-derived each generation from
surviving chat history (`find_tools` results and direct calls to
deferred tools), so activations naturally lapse when compaction
summarizes them away. Aggregate activated schema weight is capped at 10%
of the context window, shedding the least recently activated schemas
first; `find_tools` shares that budget across parallel calls in one
step. No new persistence.
- Deferred tools stay registered for execution, so the model can call a
cataloged tool directly without searching first; the schema is activated
for subsequent steps.
- Fail-open: the experiment being disabled, an empty candidate set, or
an MCP tool named `find_tools` all disable deferral, leaving today's
behavior byte-identical on the wire.
- Prometheus counters/histograms track `find_tools` calls, matches,
activations, and deferred token weight.
- The conversation timeline renders `find_tools` calls with a collapsed
search summary and expandable match list, falling back to the generic
renderer on malformed payloads.

## Validation

- Unit tests for the catalog, matcher, experiment-gated decision, and
activation derivation; end-to-end chatd generation tests covering
search-then-call, direct-call activation, experiment-off wire parity,
compaction lapse, and subagent tool gating.
- Storybook interaction tests for the timeline rendering and
malformed-payload fallback.
- Remote dogfood UAT on dev.coder.com passed: deferral with a real MCP
server and Anthropic model, direct calls without prior search,
activation persistence across turns, experiment-off parity, and clean
UI/console.

> Disclosure: Mux (AI agent) authored this PR on Mike's behalf.
2026-08-18 19:12:47 +02:00
4b7494be72 feat: harden chat generation runtime instrumentation for billing (#27451)
Closes CODAGT-835

## Summary

`chat_messages.runtime_ms` becomes the billing source of truth for Coder
Agents runtime (summed hourly by #27312), but it was built for
debugging: the June refactor (#26270) silently stopped recording
tool-step runtime, compaction was never measured, and interrupted turns
lost their partial runtime entirely. This PR defines the billable
metric, closes the paths that dropped it, and documents the definition
where the data lives.

## The billable definition

**`runtime_ms` is the wall-clock duration of the model invocation that
produced the persisted message content**, measured from just before the
provider stream opens until it is fully consumed.

What counts:

- Assistant generation steps, in top-level and sub-agent chats
(sub-agents are ordinary chats on the same generation path).
- Compaction summarization calls, persisted on the compaction assistant
message (**new**).
- Interrupted attempts: the message-part episode's lifetime is persisted
on the partial assistant message committed by `FinishInterruption`, so
partial generation time survives interruption (**new**; measured via a
new `Buffer.EpisodeDuration`, which works even though the generation
goroutine and the interrupt task are different tasks).

What deliberately does not count (each is documented in code and docs):

- **Local tool execution.** Tool wall time includes idle waits, most
importantly `wait_agent` polling a sub-agent chat that already bills its
own model invocations; billing the batch would double count, and
excluding one tool from a concurrent batch's wall time is ill-defined.
Pre-refactor instrumentation did include tool time; this makes the
exclusion an explicit product definition instead of a silent regression.
- **Failed model calls whose output is discarded** (retried attempts,
terminal errors, content-filter refusals). They persist no content, so
they bill nothing; billing errs toward undercounting. Notably a
stream-silence timeout can burn 10 idle minutes before a retry, which
should not be billable "active generation". If product later wants
failed attempts billed, that needs a place to persist runtime on error
turns (`FinishError` inserts no rows today) and is a deliberate
follow-up, not instrumentation drift.
- **Ancillary calls that produce no chat messages** (title generation,
advisor, turn summaries) and all idle/parked time (`requires_action`,
queueing).

The definition is documented as `COMMENT ON COLUMN
chat_messages.runtime_ms` (migration 000551, surfacing as a Go doc
comment on `ChatMessage.RuntimeMs`), on
`chatloop.PersistedStep.Runtime`, in the chatd architecture doc, and in
the Spend Management docs page.

## Index for the hourly scan

None needed: `GetTotalChatMessageRuntimeMsInRange` (#27312) filters an
hour-wide `created_at` range, which the existing
`idx_chat_messages_created_at` b-tree already serves; the residual
`runtime_ms IS NOT NULL` filter applies to one hour of rows. A partial
index would add permanent write amplification for a query that runs once
an hour.

> [!NOTE]
> Migration 000551 is also claimed by #27312; whichever merges second
renumbers via `fix_migration_numbers.sh`.

## Tests

- End-to-end: the existing full-server generation test now asserts
`RuntimeMs.Valid` on the committed assistant row (it previously read
`.Int64` without checking `.Valid`, so it passed on NULL).
- Interrupted turn: full task-level test (real DB, mock clock) asserting
the partial assistant message persists the attempt's runtime.
- Errored stream: asserts a failed invocation yields no step and no
runtime.
- Tool-using turn: asserts runtime lands on the assistant row only and
tool rows stay NULL.
- Compaction: asserts the summarization call duration is recorded and
lands on the compaction assistant message only.
- `messagepartbuffer.EpisodeDuration` unit coverage.

Blocks: CODAGT-843 (B3), CODAGT-838 (D8).

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Hugo Dutka <hugo@coder.com>
2026-08-06 16:09:38 +07:00
Michael Suchacz c17bed25e0 feat: wire chat lifecycle hooks into chatd (#27429)
Wires chat lifecycle hooks into chatd, gated by the
`agent-lifecycle-hooks` experiment. Part of the lifecycle hooks stack
(#27401, #27428, #27430). See `docs/admin/setup/chat-lifecycle-hooks.md`
for the consumer-facing contract.

## Summary

When a hook URL is configured, chatd dispatches `session_start`,
`user_prompt_submit`, `pre_tool_use`, `post_tool_use`, `pre_compact`,
`post_compact`, and `stop` events to the consumer and applies its
responses.

## Design

- **Stateless**: Coder stores no hook dispatch or decision state.
Delivery is at least once; consumers deduplicate on stable payload
identifiers (chat ID, event type, tool-use ID) and answer duplicates
with the same decision.
- **Admission-time prompt effects**: `user_prompt_submit` dispatches
exactly once per submission (create, send, queue, edit, subagent spawn)
and folds its effects into the stored prompt as typed message parts:
original-or-overridden user parts, then model-only `hook-context`, then
a user-visible `hook-notice`. Hook context is stripped from every
client-facing conversion; hook notices are excluded from model prompts.
The server rejects hook parts in client-submitted content.
- **Tool gating**: `pre_tool_use` allow can override tool input; deny
becomes a synthetic denied tool result, with any returned model context
persisted as a model-only transcript row so it never reaches clients.
The denial text identifies an external policy (the deployment's
lifecycle hook) as the source and marks the decision as persistent, so
the model explains the denial instead of retrying it or misreporting it
as an infrastructure failure.
- **Fail closed**: a dispatch failure rejects the triggering request or
moves the chat to the error state in the same transaction as the
affected step, so a runnable state is never published with unapproved
content.
- **Admission before persistence**: `pre_tool_use` is dispatched for the
calls the model produced, before the assistant message is stored. See
"Staged tool admission" below.
- **Fresh dispatch per tool call**: every non-provider-executed tool
call is decided by its own `pre_tool_use` dispatch; Coder never reuses
an earlier decision on the consumer's behalf. Retries re-dispatch the
same logical event.

## Structure

All hook dispatch flows through one seam: entry points build a
`chathooks.Chat` (chat identity) and a `chathooks.Message` (event
details) and call `Trigger.Trigger`, the only component that talks to
the dispatcher. The integration lives in the `coderd/x/chatd/chathooks`
subpackage, split by responsibility:

- `trigger.go`: the trigger seam; builds the wire envelope per event,
normalizes deny into a typed error, and holds the package's single
enabled-check.
- `effects.go`: pure conversion of hook results into transcript rows and
prompt parts.
- `errors.go`: failure classification (dispatch error messages, denial
mapping, tool-result dispatch-failure scanning).
- `tooluse.go`: the tool-call gate (`pre_tool_use` preflight,
`post_tool_use` payloads, applying admitted input to the step).

Server-bound glue stays in `coderd/x/chatd/hook_server.go`: the
chat-parking dispatch error handlers, the step-commit row insertion
wrappers, and the dynamic post-tool-use state loader, which depends on
chatd validation types.

This PR adopts the `codersdk/x/agenthooks` and
`coderd/x/agenthooks/dispatch` import paths introduced at the tip of
#27401; intermediate commits still reference the pre-move paths and are
not individually buildable.

## Staged tool admission

`pre_tool_use` originally ran at tool execution time, which is after the
assistant message carrying the tool call was already committed. An
`input_override` therefore had to rewrite stored message content in
place. @hugodutka pointed out that chatd treats message content as
immutable, and that the rewrite was a shortcut rather than a
requirement.

It was also a correctness problem in its own right: the rewrite only
updated the database, so the transcript could show one input while a
different one had executed.

The hook now runs before the step is persisted:

```text
provider stream ends (tool calls complete, in memory)
  -> pre_tool_use dispatch per call
  -> ONE transaction: assistant row with admitted inputs, synthetic denials, hook rows
  -> execute
```

The step is inserted once, carrying the input the tool runs with.
`UpdateChatMessageContentByID` and `Tx.UpdateMessageContent` are deleted
from #27428, so message content stays immutable.

Two consequences, both intentional:

- **Clients converge rather than wait.** Tool-call parts still stream
live, so a rewritten call briefly shows the model's proposed input
before the committed message replaces it. The chat store already clears
stream state when an assistant message arrives, so the stored input wins
with no frontend change and no added latency before tool cards appear.
- **A call already in history was already admitted.** Execution consumes
the stored input instead of dispatching a second decision, which keeps
one dispatch and one set of hook effects per call. A consumer policy
change between admission and execution applies to later calls, not to
calls already admitted.

The per-chat debug endpoint still records the provider's original tool
input. Its purpose is to report provider behavior, and it requires an
explicit per-chat debug flag; the invariant here covers the transcript.

## Configuration

Adds `chat-hook-url`, `chat-hook-secret`, `chat-hook-timeout`, and
`chat-hook-enabled` deployment options with startup validation. The
flags are hidden from `coder server --help` while the feature is
experimental; the setup guide documents them.

## Tool input validation

Built-in tool arguments reach a consumer as raw JSON with key spelling
preserved, but the tools decode those bytes with Go, which matches
struct fields case-insensitively and keeps the last match. A policy
reading `path` could therefore authorize one value while the tool
executed another, and a lone case variant such as `{"PATH":"/secret"}`
was invisible to a policy checking for `path`.

Coder now rejects a built-in tool call whose input repeats a key or
spells a schema property with different capitalization, before the
`pre_tool_use` dispatch, so a consumer is never asked to authorize bytes
whose meaning depends on the reader. Rejected calls produce an error
result the model can retry; unambiguous calls in the same batch still
run. A consumer-authored `input_override` is rechecked after the
dispatch and fails the turn closed, because the model cannot correct it.
Dynamic and MCP inputs are excluded because the client and the workspace
agent execute those calls rather than coderd.

Two paths needed more than a schema check. Execution resolves a
deprecated tool name to its canonical tool, so validation resolves
aliases first. The `edit_files` decoder also reads `search` and
`replace`, which its schema does not advertise, so those aliases are now
matched exactly and their case variants ignored.

A hook denial now returns a structured 403 carrying `kind:
"hook_denied"`, mirroring the dispatch-failure response that already
carries its own kind. Without it a client cannot tell a policy decision
apart from a generic failure, and the chat UI titled a denial "Request
failed". Adding a kind needs no migration: `ChatErrorKind` is persisted
only inside the JSONB `chats.last_error` column, whose decoder accepts
unknown kinds.

The hook docs also correct the tool-input convergence window. A batch
dispatches sequentially before the assistant row commits, so the
original input stays visible for a span that scales with the number of
tool calls in the step rather than a single hook timeout.

> This PR was written by Mux, an AI coding agent, on Mike's behalf.
2026-07-29 11:39:12 +00:00
Michael Suchacz e9951b07d4 fix(coderd/x/chatd/chatloop): surface reasoning-only content-filter refusals as terminal errors (#27476)
## Problem

Anthropic can end a stream with `stop_reason: "refusal"` after reasoning
content has already streamed. The content-filter guard in
`chatloop.GenerateAssistant` only fired when the step content was
completely empty, so a reasoning-only refusal bypassed it: the turn
finished as `status=waiting` with `last_error=null`, and the user saw
the chat silently stop mid-turn with no explanation. This looked like a
Coder fault when the provider had rejected the response. Observed twice
in dogfood on 2026-07-23 (chat `c72f99fc`, debug steps show
`finish_reason=content-filter` with reasoning-only content).

## Change

Treat a content-filter finish as terminal whenever the step produced no
user-visible output. A new `hasUserVisibleContent` helper counts any
non-reasoning part (text, tool call, tool result) as user-visible;
reasoning-only or empty steps now return the existing
`contentFilterError`, which flows through the established pipeline:
classified `ChatErrorKindContentFilter` (non-retryable, refusal
category/detail when provided), persisted `chats.last_error`, streamed
error event, and the "Response blocked" callout in the chat UI.

Behavior for steps with visible text or tool calls is unchanged, and the
frontend needs no changes.

## Testing

- New regression subtest `ReasoningOnlyContentSurfacesTerminalError`
(reasoning stream then content-filter finish) beside the existing
empty-content and partial-content subtests, which are unchanged.
- `go test ./coderd/x/chatd/...` and lint pass.
- Dogfood UAT against a local dev instance with a mock Anthropic
upstream passed all three scenarios: reasoning-only refusal shows the
"Response blocked" callout with `last_error.kind=content_filter` and no
retry affordance; text-then-refusal still completes normally; empty
refusal still errors.

> This PR was created by Mux acting on Mike's behalf.
2026-07-24 19:40:53 +02:00
Michael Suchacz 3227cac217 feat: add manual chat compaction via /compact (#27081)
Adds a user-triggered `/compact` action for Coder Agents chats: typing
`/compact` in the composer (or picking it from the `/` trigger menu)
summarizes the conversation so far to free up context window space.

## How it works

- New `POST /api/experimental/chats/{chat}/compact` endpoint
(owner-only, RBAC `ActionUpdate`, excluded from the public API reference
via `x-apidocgen skip`). It marks the chat with a durable one-shot
`chats.compaction_requested_at` signal and moves it `waiting -> running`
via a new `RequestCompaction` state transition; no message row is
inserted. AI Gateway attribution needs no per-request key: generation
preparation resolves the owner's synthetic API key (#27170) like any
other turn.
- `RequestCompaction` hands off chat ownership (clears
`worker_id`/`runner_id`) so a worker acquisition hint is published;
since the transition changes no history, the previous runner could
otherwise miss the request under reordered pubsub delivery.
- The background chat worker picks the chat up like any other turn. A
pending manual request takes precedence over turn completion in the
generation decision, and forces compaction even below the automatic
threshold (and when compaction is disabled via threshold=100). The
commit step consumes the request marker in the same transaction; any
transition that ends the turn clears stale markers.
- The summary triplet reuses the automatic-compaction path, now tagged
with a `source` (`automatic` | `manual`) that is plumbed through
streamed progress parts, persisted tool JSON, and the UI label
("Summarized (manual)").
- Validation order: busy chats reject with 409 (state-machine conflict),
empty/already-compacted chats with 409 "nothing to compact", archived
chats with 400; the owner usage-limit check runs last so no-op requests
surface the specific conflict instead of a limit error.
- Web UI: the `/` trigger menu now has a built-in "Commands" group
listing `/compact`; submit intercepts exactly `/compact` and calls the
endpoint instead of sending a message. A personal or workspace skill
named `compact` takes precedence over the built-in command; while skill
collisions are still resolving, an exact `/compact` submission is
blocked with a retryable hint instead of leaking as message text.
History and queued-message edits are never intercepted. After
compaction, the context usage indicator resets to its unknown state
until the next assistant response reports fresh usage, instead of
showing the stale pre-compaction number.
- codersdk: `ExperimentalClient.CompactChat`.

Worker-path execution (rather than compacting synchronously in the
handler) reuses the existing lock fencing, live "Summarizing..."
streaming, retry accounting, restart resilience, and debug-run
observability. Rationale documented in `coderd/x/chatd/ARCHITECTURE.md`.

## Testing

- State machine: transition-matrix coverage for `RequestCompaction`,
marker lifecycle tests (carried by lease renewals/queue appends, cleared
by terminal transitions, consumed by commit), ownership handoff +
acquisition hint assertions.
- Worker: decision-ordering and forced-compaction unit tests;
active-server end-to-end test (manual compact below threshold produces a
`source=manual` summary, returns to `waiting`, no assistant follow-up;
busy chat rejected).
- API: success, archived, non-owner, RBAC-denied, empty-chat, no-daemon
cases; usage-limit ordering (at-limit owners still get
state/nothing-to-compact conflicts for no-op requests, with marker
rollback).
- Frontend: Storybook play tests for the Commands menu group, submit
intercept, skill-name collision, queued-edit passthrough, and
manual/automatic tool rendering; unit tests for command availability
resolution and the post-compaction context usage reset.

> This PR was created by Mux, an AI coding agent, working on Mike's
behalf.
2026-07-21 10:58:08 +02:00
Mathias Fredriksson f997afa220 feat(coderd/x/chatd/chatloop): retain user constraints in compaction summaries (#27230)
Compaction summaries drop or soften user-stated constraints,
corrections, and prohibitions, so post-compaction assistants repeat
behavior the user already corrected. Add a summary prompt bullet that
instructs the summarizer to quote them, treat them as standing until
revoked, and attribute rules to their true source instead of
defaulting to the user.

Validated offline on unseen human chats: holdout P1 delta +0.175
(arbitrated), 13/18 cases improve. This improves per-compaction
retention only; it does not address deep-chain correction loss.
2026-07-16 09:18:43 +00:00
Ethan cc11c8a536 feat: surface model content-filter refusals as a blocked chat error (#27118)
Blocked turns from a provider's content filter (Anthropic's `refusal`
stop reason with empty content) previously ended silently on the
"Thinking" spinner. They now end as a terminal `content_filter` error
that renders as a "Response blocked" message with the provider's
category and explanation.

<img width="888" height="335" alt="image"
src="https://github.com/user-attachments/assets/cef85a59-4091-4e62-9d45-1eb06748db48"
/>


Closes CODAGT-611

Follow-ups will involve implementing fallbacks, but this alone is pretty
important
2026-07-15 16:36:31 +10:00
Michael Suchacz a567f6a89f feat: allow admins to override the chat compaction model (#27151) 2026-07-14 16:57:04 +02:00
Michael Suchacz 0ae4554de9 fix(coderd/x/chatd/chatloop): remove compaction timeout (#27007) 2026-07-07 12:22:59 +02:00