Commit Graph
60 Commits
Author SHA1 Message Date
Michael Suchacz c28ae0dc9b feat: trigger compaction independently for the chat and compaction models (#28568)
> [!NOTE]
> Xum (an AI agent) authored this pull request on Mike's behalf.

## Summary

Automatic compaction now fires at whichever of two independent triggers
is reached first, and the UI shows when the organization override's
trigger applies.

- **Chat trigger**: the chat owner's personal threshold (or the chat
model's) against the chat model's window. A threshold of 100 turns off
only this trigger.
- **Organization trigger**: a fixed 80% of the organization compaction
override model's window. The override model's own compression threshold
is ignored. The other 20% is a target margin for the summary prompt and
the summary, not a guarantee, because the trigger reads the previous
step's prompt size.

On main, the chat threshold was applied to the smaller of the two
windows, and nothing in the UI showed it. Example: with a 20K chat model
at 70% and an 8K override, main compacts at 5,600 tokens; this PR
compacts at 6,400 (80% of 8K, shown as 32%).

## Changes

Backend (`coderd/x/chatd`):

- The trigger with the lower token point binds (ties go to the chat
trigger). It drives the automatic compaction gate and the
still-over-limit check, and every summary, automatic or manual, records
it.
- An override that is unset, deleted, disabled, on a disabled provider,
or without credentials for the chat owner is treated as absent: the chat
model summarizes and only the chat trigger applies.
- The summary's output cap is bounded by the summarizer's own window,
not the trigger's.
- The still-over-limit error names what can clear it: the compaction
threshold (chat trigger), an override model with a larger window
(organization trigger), or both when usage is over both points.
- The still-over-limit check compacts once more when the latest summary
was recorded under a different threshold or trigger window. The
`chat_summarized` result gains `trigger_context_limit_tokens`, and
`context_limit_tokens` now records the chat model's window (main
recorded the smaller window). On main, lowering a threshold after a
compaction ended every later turn with the error.
- The still-over-limit check skips a 0% threshold ("always compact"),
which on main ended the run with the error after every compaction.
- `ARCHITECTURE.md` documents the two triggers, the summary cap, and the
still-over-limit rules.

Frontend (`site`), through a shared `compactionTriggers` helper that
mirrors the backend rules (no new API):

- Coder Agents settings: the compaction override row warns when the
organization trigger undercuts chat models' default triggers ("the
organization override compacts chats at 80% of {model}'s {N}-token
window").
- Personal thresholds page: an Effective column shows the trigger that
applies, with a popover when the organization override binds. "Compacts
at" follows the same trigger, and a point the chat model's window cannot
reach shows as Off. The header's 100% sentence, and the row warning when
a member types 100, say an organization override may still compact chats
when one is set or its organization's overrides failed to load (the
header also while they load).
- Chat context gauge: the compact-at percent follows the binding trigger
and is annotated when the organization override binds. If the overrides
request fails, it shows the chat-only value with "(organization override
not loaded)", or "Compaction off (organization override not loaded)"
when the chat trigger is off.
- An override whose provider the user cannot use is ignored on the gauge
and the personal page, matching chatd.
- The shared `formatContextLimit` prints counts below 1,000 exactly (an
800-token compaction point showed "~1K") and 999,500 to 999,999 as
"1.0M" instead of "1000K". It also labels the model pickers; their other
values are unchanged.
- `HelpPopoverIconTrigger` lets callers set their own `aria-label`; only
the Effective column's popover does.

Known v1 limits: the Coder Agents settings alert is organization-scoped,
so it can warn about an override that some members fall back from for
lack of provider credentials. The gauge checks the viewer's provider
access, so on a shared chat its label can differ from the owner's
runtime trigger (Codex P2, declined). Compaction itself is unaffected.

## Screenshots

From remote UAT round 13 on 093076f402: an isolated test deployment
with a local fake LLM. The models are test fixtures named for their
context windows: Chat model 20K (own threshold 70%), Chat model 5K
(40%), and Chat model 8K (50%). Chat model 8K is also the organization
compaction override, so the organization trigger is 80% of 8,000 = 6,400
tokens, which is 32% of Chat model 20K's window. Chat model 8K's own 50%
applies only to chats that use it as the chat model; changing it to 10%
or 100% left every value below unchanged.

**Coder Agents settings:** the organization trigger undercuts Chat model
20K's 70% default.

![Override warning: 80% of Chat model 8K's window undercuts Chat model
20K](https://github.com/user-attachments/assets/188c92e6-faaa-4e9a-a563-567cbf98af85)

**Personal thresholds:** the organization trigger binds Chat model 20K
at 32%. Chat model 5K (40%) and Chat model 8K (50%) keep their own chat
triggers.

![Effective column: 32% from the organization trigger on Chat model
20K](https://github.com/user-attachments/assets/9949be4e-9417-4cfc-a683-f1367a1dd77a)
![Organization override popover: 80% of Chat model 8K's window, about
32% of this model's
window](https://github.com/user-attachments/assets/4bc3e002-0be1-46fd-91f0-aa27503034af)

**Personal thresholds, organization settings request failed** (the first
red line is the injected error). Rows fall back to chat-only values.

![Organization settings load failure above usable chat-only
rows](https://github.com/user-attachments/assets/dcf48546-aa47-4f82-a6eb-85f106b4bf45)

**Chat context gauge:** organization trigger binds (Chat model 20K),
chat trigger binds (Chat model 5K), overrides request failed (injected).

![Gauge: Compacts at 32% (organization
override)](https://github.com/user-attachments/assets/cd5402d9-9849-4254-a7a7-6623c4006a2a)
![Gauge: Compacts at
40%](https://github.com/user-attachments/assets/f372c9e9-72ef-41c8-950c-e77d74965f65)
![Gauge: Compacts at 70% (organization override not
loaded)](https://github.com/user-attachments/assets/ab829746-2ab5-4743-848f-dce4e806de3a)

**Still-over-limit errors from real runs:** organization trigger, chat
trigger, both.

![Organization-only over-limit
error](https://github.com/user-attachments/assets/f210807a-a2cc-47a0-807c-f1404373bb28)
![Chat-only over-limit
error](https://github.com/user-attachments/assets/eecec38e-97de-49b6-8670-b32090103bbb)
![Combined over-limit
error](https://github.com/user-attachments/assets/459a6d08-7215-492d-93f5-96a4f767a09e)

## Testing

- Go: `TestBindingCompactionTriggerSource`,
`TestCompactionStillOverLimitError`, `TestCompactionStatusFromHistory`,
and `TestActiveServer_CompactionModelOverride` cover the 80% point
(fails if the override's own threshold is read again), user threshold
100 leaving the organization trigger on, the summary cap for automatic
and manual compaction, the over-limit messages, and recompaction after a
trigger change. The site binding tests mirror the Go table row for row.
- Site: `compactionTriggers` unit tests and Storybook stories for the
three surfaces, including the overrides fetch failure.
- On c6f8750479: `go vet`, the `chatd` and `chatloop` tests, and the
AgentsPage, AISettingsPage, and HelpPopover unit tests (2,274 passed, 2
skipped) pass, and CI is green. On 987cefe399, tsc, the five affected
story files (109 passed), and `make pre-commit` also passed. 093076f402
only changes `ARCHITECTURE.md`; the pre-commit hook's markdown format
and lint checks pass on it.
- On 3cd5b08f1f: tsc, the new `formatContextLimit` test (fails on the
old helper for 800 and 999,999), the `compactionTriggers` and `aiModels`
unit tests (224 passed), and the affected story files pass, and the
pre-commit hook passed on all three commits.

## Review record

- Earlier heads: 12 Codex passes and 3 human reviews, all threads
answered. Rebased onto main several times, most recently ff3fa22f68
(kept #30274, #29892, and #29890 in conflicted files).
- coder-agents-review rounds 1 to 18: findings fixed; approved since
round 15. The round 1 `ARCHITECTURE.md` finding was left for the
maintainer, who asked for the update in the last commit.
- coder-frontend-review (eight roles): all pass. Splitting the PR (RO6)
was declined because it is reviewed as one unit; the
`organizationTriggerFor` story helper stays because its three fields
must match the model.
- Design change after UAT round 11: the organization trigger is a fixed
80% of the override window instead of the override model's own threshold
(171d5f568e, 21d2ce7fce, 987cefe399).
- coder-agents-review round 19: copy, naming, and an unreachable
zero-window gate fixed (6e8efb9054, c6f8750479). A summary-time fit
check with chat-model fallback (CRF-156) was declined: the previous-step
lag applies to every trigger, including main's. Round 20 approved
c6f8750479 with no findings.
- Codex: one P2 on 987cefe399 declined (see known limits); no major
issues on c6f8750479.
- Remote UAT on dogfood (fake LLM, isolated test deployment): rounds 8
to 13 PASS. Round 12 on c6f8750479 first failed one criterion that was
written wrong (the admin alert correctly lists an override model that is
also a chat model at 100%); the reviewer endorsed PASS on the same
evidence after the correction. Round 13 on 093076f402 retook the
screenshots with neutral model names and confirmed that the override
model's own threshold (10%, 50%, 100%) does not move the organization
trigger.
- UAT round 13 notes: the header after a failed or pending overrides
load and the sub-1K labels are fixed in 4e172ab5ff and d5888ce15dc;
coder-agents-review round 21 found the same gap in the row warning
(CRF-167, fixed in 3cd5b08f1f). Left as on main: a summary's
`usage_percent` is measured against the binding trigger's window, so it
can exceed 100 (nothing reads it), and the 100% help text shows only for
an unsaved edit.
- Codex: no major issues on d5888ce15d and 3cd5b08f1f.
coder-agents-review round 22 approved 3cd5b08f1f with no open findings.
- coder-frontend-review on those commits: all roles pass after the
fixes. Declined: moving the formatter fix to its own PR (RO5; the owner
asked for it here), adding "tokens" to sub-1K labels (TN6; "~800" reads
like "~16K"), and splitting the component (RP6).
2026-10-11 12:10:59 +02:00
Michael Suchacz bb9f31d94b fix: apply Anthropic prompt caching through OpenRouter and OpenAI-compatible gateways (#29563)
Anthropic models configured through an `openrouter` provider (and, by
the same code path, `vercel` and generic `openai-compat` providers)
never got prompt caching. Two independent gaps: chatd only emitted
`cache_control` breakpoints when the fantasy client was `anthropic`, and
even when a client sent them, the AI Gateway chat-completions
interceptor rebuilt the upstream body from typed
`openai.ChatCompletionNewParams` and silently dropped the unknown key.
Dogfood UAT on main @ 77479408b4 measured 0 cached tokens on 27/27 turns
for `anthropic/claude-haiku-4.5` via OpenRouter while the direct
Anthropic provider cached 7k to 11k tokens per turn.

## Fix

**AI Gateway** (`aibridge/intercept/chatcompletions`):
`ChatCompletionNewParamsWrapper` records every message-level and
content-part-level `cache_control` from the raw request and re-applies
it at the same JSON path when the upstream body is rebuilt, before the
Google-specific patching. The blocking path now sends the raw body
whenever preserved fields exist; the streaming path already sent the
rebuilt body. Nothing is added unless the client sent it, and at most
four markers are recorded per request (Anthropic's breakpoint limit) so
the per-marker body rewrites stay bounded.

**chatd** (`coderd/x/chatd`): the caching gate becomes a small strategy
resolver keyed on the fantasy client and the model ID. `anthropic` keeps
the existing message-level breakpoints; `openrouter` and `vercel` reuse
them for Anthropic-family IDs since both clients translate them;
`openai-compat` gets content-part-level `ContentExtraFields` markers
(last text part of the last system message plus the final two
text-bearing user or assistant messages, tool messages skipped, at most
3 markers). `chatprovider.IsAnthropicFamilyModelID` recognizes
`anthropic/...`, Bedrock-style `anthropic.`/`.anthropic.` IDs, and bare
`claude` IDs. The `claude` substring match is deliberately broad; a
non-Anthropic model whose ID contains it would receive markers that
OpenAI-compatible upstreams ignore.

Docs: a short "Prompt caching" note in `docs/ai-coder/agents/models.md`.
`coderd/x/chatd/ARCHITECTURE.md` has no caching text to update.

## Dogfood UAT

Remote Coder Agents UAT on dogfood (chat:
https://dogfood.cdr.dev/agents/9f81cc40-add0-4b6f-b76c-f348b04762e4),
OpenRouter provider with real keys:

- Baseline (main @ `77479408b4`): 0 cached tokens on every OpenRouter
Haiku turn; direct Anthropic control cached normally; a hand-added
`cache_control` on the same key cached fine.
- Fix (`1d685f7e3e`, the first three commits):
`anthropic/claude-haiku-4.5` via OpenRouter now sends 2 markers on turn
1 and 3 afterwards (system last part plus the last two text-bearing
messages, never on tool messages). OpenRouter's `/generation` ground
truth reported `native_tokens_cached` 7137 to 7255 on turns 2+, equal to
Coder's `cache_read_tokens` and the AI Gateway's
`cache_read_input_tokens` on all 43 measured rows. Turn-2 prompt cost
dropped about 8x versus baseline. Agentic S3 cache reads grew from 7156
to 13993 as a 2500-line file entered the prompt. `openai/gpt-5-mini`,
`google/gemini-2.5-flash`, `deepseek/deepseek-chat-v3.1`, and
`moonshotai/kimi-k2.5` received 0 markers with unchanged request shapes;
the direct Anthropic control was unchanged. 76 upstream attempts, 0
non-200. The remote agent reported FAIL only because one acceptance
criterion I wrote asked the chat usage indicator to display cached
tokens, which the UI has never done for any provider; the runner
endorsed PASS for the fix itsel... [truncated]
- Smoke re-tests on `20ad259999b` (marker-code simplification) and on
`e31f1806748` (docs reflow plus the four-marker cap; the later head
`92b8f8ad0b8` only renames `google_openai_compat.go` to
`request_body.go` with no content change): S1 cache reads 0 / 7139 /
7161 / 7184 / 7207 equal to OpenRouter ground truth on every row,
markers 2 then 3 and never on tool messages, tool-heavy turn all 200, a
12 KB file read grew cache reads to 13548 with every request body
inspected for marker placement, `openai/gpt-5-mini` still 0 markers.

Known follow-ups surfaced by the UAT and out of scope here:
`provider_options.openrouter.*` never reach the wire on the gateway
path; OpenRouter cache writes are folded into `input_tokens` in chat
message usage (`cache_creation_tokens` stays 0) although the gateway
records them; the AI Gateway sessions UI has no cache columns and labels
every OpenAI-compatible provider "OpenAI"; attachment-only messages (no
text) fail upstream with HTTP 400 on OpenRouter (pre-existing,
reproduced without markers).

> Xum acted on behalf of @ibetitsmike for this pull request.
2026-10-04 22:55:37 +02:00
J. Scott Miller fc0af7fbd8 feat: open a chat_turn span per prompt (#29771)
**TL;DR:** Wraps each prompt in a `chat_turn` root span with every
lifecycle stage parented under it. The span records one outcome per turn
(`completed`, `interrupted`, `error`, `abandoned`), and the new
`coderd_chatd_turn_outcomes_total` counter counts those outcomes. Turns
stay correct when tasks are canceled, replaced, or retried.

Each prompt runs inside a `chat_turn` root span owned by the runner
(`runnerTurnSpan`). The generation step, prepare, MCP connect, stream,
TTFT, provider attempt, commit, thinking, tool call, compaction, and
retry backoff stages are parented under it, activating the stages from
#29770. `chat_turn` spans are standalone roots: the turn runs on a
worker, often on another replica, and no trace context is persisted.

**Lifecycle**

- **Start.** Each generation loop iteration calls `Ensure`, which opens
the turn if none is open and returns a `turnToken`. The span is
backdated to the trigger: the latest of the last user prompt, a dynamic
tool's result message after it, and a pending compaction request. An
`acquisition` stage covers trigger to pickup.
- **Stale trigger / takeover.** A trigger at or before the previous
anchor opens the turn at now with no `acquisition` and counts
`stale_anchor`. A runner that took the chat over from an owner with a
stale heartbeat (`TakenOver`, from acquisition) opens its first turn at
now with no `acquisition`, so the dead owner's run is not reported as
pickup delay.
- **Newer prompt.** `Ensure` on the current turn with a newer trigger
closes it and opens a new one. The closed turn is `abandoned` unless the
task that ran it records an outcome before releasing it.
- **Finish.** After `FinishTurn` or `EnterRequiresAction` commits, the
step calls `Complete`; `Settle` closes the span after the step's own
stage ends, so the finishing step is inside the turn. A chat waiting in
`requires_action` is outside any turn; the generation resumed by its
tool results opens a new one.
- **Emission.** Closing a turn fixes its end time; its span is emitted
once every task that joined it (`Ensure` takes the task ID) has called
`Release`, which `runTask` does after the task returns. A finishing task
whose `FinishTurn` promoted a queued prompt therefore still counts its
turn as `completed` even when the promoted task's `Ensure` runs before
its `Complete`. `End` at runner exit emits whatever is left.
- **Failure.** A step that commits `FinishError`, or fails closed,
invalidates the turn as `error` after the commit. Errors the task runner
retries, including task timeouts, keep the same turn open; an expected,
non-retryable exit records no outcome and leaves the turn open until the
interrupt task, a newer prompt, or runner exit closes it. A canceled
generation task records no outcome and cannot obtain a turn token, so a
replaced task never touches its successor's turn.
- **Interrupt.** The interrupt task marks the open turn `interrupted`
before `FinishInterruption` commits and closes it after. Only a stop (by
the user or a parent agent) is `interrupted`; it ends with unset span
status, since error status is reserved for `error` turns. runner
shutdown, loss of ownership, and message edits close the turn as
`abandoned` with unset status.
- **Promotion.** Every promoting transition (`FinishTurn`,
`FinishInterruption`, a send to an errored chat, `PromoteQueued`)
returns the promoted message's queued time in its result, and one
`recordQueueWait` helper records a standalone `queue_wait` to the
promotion. The promoted turn opens at the promotion time.

`chat_turn` is observed on `coderd_chatd_stage_duration_seconds` only
for completed turns, so the histogram is completed-turn latency; every
turn still emits its span.

`coderd_chatd_turn_outcomes_total{outcome, chat_kind}` counts every
closed turn exactly once (`completed`, `interrupted`, `error`,
`abandoned`), matching the span's `turn_outcome`, under the
`chat-stage-metrics` experiment. Turn time accounting is in draft
#29772.

**Stages and identity**

- `tool_call` is a live stage around each local tool call in `chatloop`;
the tool runs on its context, and it ends errored only when the tool
fails to execute. Calls that never run (inactive, unknown, or rejected
by policy) record no stage.
- `mcp_connect` carries connected and failed server counts and ends
errored only when servers were configured and none connected.
- The compaction stage carries the model that runs the summary.
- `withStageIdentity` puts scope and chat kind on the turn context.
Detached background work runs in `background` scope as its own trace
roots.
- The `advisor` tool's nested model call appears only as its enclosing
`tool_call`.

ARCHITECTURE.md gains a "Lifecycle tracing" section.

<details>
<summary>Review decisions</summary>

- Turn identity stays timestamp-based with targeted fixes (clamped
acquisition, rotation on a newer trigger, invalidate after commit,
fail-closed invalidation, settle closing invalidated turns) plus a
takeover flag; keying turns on message IDs would not detect takeover and
cannot express interrupted/abandoned outcomes.
- Outcome classification: the interrupt task owns `interrupted`, and
cancellation or a fence mismatch alone never records an outcome, so
shutdown, ownership loss, and edits are `abandoned`. Only `error` turns
set error span status. Remaining ordering races between a step and an
interrupt are documented as known limitations in ARCHITECTURE.md.
- Turns are emitted after the tasks that ran them return, not when they
close, so an outcome a replaced task records while unwinding reaches its
own turn; the successor task is not delayed. Chosen over an
after-commit, pre-publish hook on `ChatMachine.Update`, which would fix
only the `FinishTurn` ordering.
- `requires_action` ends the turn rather than pausing it, so `chat_turn`
measures chatd time; a prompt with client tools counts as several turns.
- Promoted turns anchor at promotion time and `queue_wait` is always
standalone, so `chat_turn` never includes time spent queued behind
another turn.
- The former `capacity_wait` stage (#29805) was dropped: it was a
per-worker lower bound, excluded from accounting, and needed its own SQL
and bookkeeping. #29805's remaining helpers are folded into this PR.

</details>

Supersedes #28972, #28973, and #29805.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
2026-09-30 11:53:58 -05:00
J. Scott Miller e988ddb11d feat: trace chatd provider stages and count only output parts as first token (#29770)
**TL;DR:** Traces the model provider call: a `stream` stage, a nested
`time_to_first_token` stage, and one `provider_attempt` per HTTP round
trip, labelled with provider type and model. Behavior change:
`coderd_chatd_ttft_seconds` now counts only a real output part as the
first token, so error, warning, and finish parts no longer count.

Instruments the provider call.

**Stream and time to first token.** `GenerateAssistant` records a
`stream` stage from stream open to close and a nested
`time_to_first_token` stage. The TTFT window closes once, on the first
streamed output part: any part other than warnings, finish, or error,
including start markers such as `text_start`. A closed window observes
the stage and the pre-existing `coderd_chatd_ttft_seconds` with the same
elapsed time. A failed open, an error part, or a release without an
output part ends the TTFT span with the error (a silence timeout is
classified as such) and observes nothing. The `stream` span ends once,
with the error `GenerateAssistant` returns, so content-filter and
no-output attempts end errored. Failed `stream` windows are observed
like successful ones. The attempt is released before the stream span
ends, so the TTFT window always falls inside its stream. The compaction
summary stream passes a nil tracer, so it records neither.

Without #29771, production emits only `provider_attempt`; #29771 passes
the tracer to `GenerateAssistant`.

**Provider attempts.** The provider HTTP transport is wrapped in
`stageSpanRoundTripper`, which emits one `provider_attempt` stage per
round trip, so retries each get a span. The span closes on response
headers; a status >= 400 marks it errored without changing what the
client sees.

`resolveModelCall` resolves the `StageModel` (wire provider, configured
`provider_type`, model, effective reasoning effort) and threads it to
the transport, so span attributes and labels match the call.
`provider_type` is read from `ai_providers.type`, so types outside the
fantasy set (copilot) keep their configured name. `chatd.New` builds the
tracer with the server clock.

**Behavior change to `coderd_chatd_ttft_seconds`:** only an output part
counts as a first token. Error parts no longer close the window as a
success, and warnings and finish parts are skipped; the help text says
so.

`StageSpan.End` returns the span's elapsed window, which `ttft_seconds`
observes. Callers pass `Stages` and `StageModel` through
`GenerateAssistantOptions`; a nil tracer discards (TTFT still records
`ttft_seconds`). `Server` gains the `stages` tracer and a
`TracerProvider` option, wired from `coderd`.

<details>
<summary>Review decisions</summary>

- Start markers close the TTFT window: they are the first sign the model
is producing output, and waiting for a delta would fold hidden reasoning
time into TTFT.
- Failed `stream` and `provider_attempt` windows are observed: that time
was spent. TTFT is the exception because a first token that never
arrived has no duration.
- `http_host` was dropped from `provider_attempt`; every request goes
through the local AI Gateway transport, so it was constant.
- The advisor tool's nested model call is intentionally not instrumented
as `stream`/TTFT.

</details>

Supersedes #28970 and #28971.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
2026-09-30 11:53:58 -05:00
Mathias Fredriksson fa5bac6e77 feat: send tool call IDs and cancel unresolved calls on interrupt (#29954)
Retries and replica takeovers ran tool calls again, and an interrupt
recorded the same generic result whether a call had finished, was
still running, or never started.

chatd now derives a tool call ID from the chat, the committed
assistant message, and the provider tool call ID, and sends it with
execute starts, edit_files, and write_file. Every replica and retry
computes the same ID, so the agent returns the first result instead
of running the tool again, and a retried foreground execute waits
only for the rest of its timeout.

On interrupt, chatd cancels the unresolved calls of those tools on
the agent before committing, within 30 seconds, and records what
happened: "tool call was canceled before it ran; no changes were
made", the output of a command canceled while running, or the real
result. The UI shows a canceled command with the existing killed
marker. If the cancel fails, including an old agent's 404, the result
stays the generic interrupted text.

Calls whose provider tool call ID is empty or repeats in the message
get no ID and behave as before.

Fixes CODAGT-757
2026-09-30 14:31:52 +03:00
Michael Suchacz 5a997c471a fix(coderd/x/chatd): tell the model when output hit the token limit mid tool call (#30120)
When an Anthropic response hits the output token limit while the model
is writing a tool call's input, chatd gets `FinishReasonLength` with a
cut-off call. It handled both shapes of that call badly:

- If the provider closed the `tool_use` block, the call carried
truncated JSON and got the generic malformed-input result, which invites
the model to resend the same oversized input and hit the limit again.
- If the block never closed, fantasy emitted no tool call, chatloop
discarded the dangling tool input, and the turn ended with only
reasoning and no explanation.

On a length finish, `GenerateAssistant` now gives each cut-off local
call an error result: "This tool call was not executed because the
response reached the output token limit (N tokens) while writing its
input. Split the content into smaller tool calls." The token count is
included when the call has a configured output limit. A tool input that
started streaming but never became a call first gets a stand-in call
with empty input. The call persists with nil args and replays as `{}`,
so the provider receives a valid `tool_use` and `tool_result` pair, and
generation continues with another step.

With lifecycle hooks enabled, `pre_tool_use` admission skips calls the
step already resolved, so the hook is not dispatched for them and no
second result is added. The duplicate tool ID check still runs on the
full batch, because a resolved call still occupies its ID.

Independent of #30119, which stops the AI Gateway from failing these
streams before chatd sees them.

## Testing

-
`TestGenerateAssistant_SilentNoOutputFinish/LengthFinishResolvesTruncatedToolCalls`
(truncated tool call, dangling tool input): no error, reasoning kept,
and each call paired with the output-limit result.
- `TestPreToolUseHookOutputLimitTruncatedToolCallResolvedOnce` with
hooks enabled, for truncated and unfinished input: no hook dispatch, one
stored error result per call, a second model request, and a replayed
request that pairs `tool_use` input `{}` with its `tool_result`.
`DuplicateIDFailsClosed` checks that a reused ID still fails the chat
with `duplicate tool use ID`.
- Red-green: with `chatloop.go` and `generation.go` restored to the base
code, all four truncation cases fail. `DuplicateIDFailsClosed` passes on
the base code by design and fails on `d02bf4aa89b`, which dropped
resolved calls before the duplicate check.
- `go test ./coderd/x/chatd/... -count=1` passes, and lint on the
touched packages reports no issues. Local pre-push checks were skipped
per standing preference; PR CI covers them.
- Remote UAT: not run yet.

## Review record

- Parent review (Xum) of `d02bf4aa89b`: skipping resolved calls inside
`chathooks.PendingToolCalls` hid them from the duplicate tool ID check
in `admitStepToolCalls`. Fixed in `745f722165b`.
- Codex: not requested yet.

> Xum, an AI coding agent, prepared this PR on behalf of @ibetitsmike.
2026-09-30 12:20:03 +02:00
Michael Suchacz fd5b37576a feat(coderd): record provider response IDs on chat messages (#29763)
Chat messages could only be linked to AI Gateway records per chat, so
nothing recorded which model response produced a given message.

fantasy surfaces the response ID the model endpoint returns on the
stream's finish part: the Anthropic Messages ID as the part ID, and the
OpenAI Responses ID in provider metadata. chatd now stores that ID in
the existing `chat_messages.provider_response_id` column for assistant
steps and the compaction summary row. No migration and no aibridge
changes.

What the stored value is depends on the route:

| Route | Stored value | Joins to |
|---|---|---|
| Anthropic Messages API through the AI Gateway (Anthropic and Bedrock
Claude) | The gateway's interception ID. The gateway replaces the
message ID because injected MCP tool loops can fold several upstream
messages into one response. | `aibridge_interceptions.id` |
| OpenAI Responses API through the AI Gateway | The upstream `resp_` ID,
passed through unchanged | `provider_response_id` on
`aibridge_token_usages`, `aibridge_user_prompts`, and
`aibridge_tool_usages` |
| Direct to the provider | The provider's own ID (`msg_`, `resp_`) |
Nothing in the gateway tables |

To resolve messages to interceptions, then read usage by
`interception_id`:

```sql
SELECT cm.id AS chat_message_id, i.id AS interception_id
FROM chat_messages cm
JOIN aibridge_interceptions i ON i.id::text = cm.provider_response_id
UNION
SELECT cm.id, tu.interception_id
FROM chat_messages cm
JOIN aibridge_token_usages tu ON tu.provider_response_id = cm.provider_response_id;
```

Not covered, and left NULL:
- Partial rows saved on interrupt, because fantasy only reports the ID
in the finish part.
- Chat Completions providers (OpenAI-compatible, OpenRouter, Vercel),
because fantasy does not surface the chunk ID there.
- User, tool, and system rows, and all existing messages.

<details>
<summary>Delivery record</summary>

- `96a896c` replaced the earlier interception-link design (`59fa58f`,
`7f93c18`, `9d04ccd`) in place: the aibridge hook, the
`aibridge_interception_id` column and migration, the message part buffer
tracking, and the `ARCHITECTURE.md` section are all removed. The net
diff against the base touches only `coderd/database` and
`coderd/x/chatd`.
- Local validation on `96a896c`: pre-commit hook green (gen, fmt, lint,
build); `go test ./coderd/x/chatd/... ./coderd/database/` green.
Red-green: removing the finish-part capture fails both new end-to-end
tests (`TestActiveServer_PersistsAnthropicMessageID`,
`TestOpenAIResponsesPersistsProviderResponseID`). CI on `96a896c`:
green.
- Remote UAT round 2 on `96a896c`
([chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)):
**FAIL** against this description's earlier claim that every stored ID
joins the gateway usage rows directly. Anthropic rows routed through the
gateway stored the interception ID, so 0 of 18 joined usage rows
directly, while every checked Anthropic row equaled
`aibridge_interceptions.id`. Both OpenAI Responses rows joined directly.
Interrupted partial rows, Chat Completions rows, and user and tool rows
were NULL, with no empty strings or duplicate IDs. Multi-step turns,
manual compaction, edit and regenerate, a provider switch, and a retried
429 behaved as expected. One unattributed anomaly: a single duplicate
user row after interrupt and continue, which did not reproduce and is
outside this diff.
- `2e889ba` corrects the doc comment on
`PersistedStep.ProviderResponseID` and this description to match;
behavior is unchanged. Pre-commit hook and CI green.
- Remote UAT round 3 on `2e889ba` against the corrected contract ([same
chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)):
**PASS**. The query above resolved all 40 stamped rows to exactly one
interception from the same chat, with none unresolved. Reading usage by
interception recovered every usage row for 38 of them; the last two
resolved but their usage rows were not counted. Fresh Anthropic and
Responses calls, a seven-step tool turn, manual compaction, an Anthropic
to Responses to Anthropic switch in one chat, and a UI reload behaved as
described, and the NULL cases held. Three interrupt and continue
attempts did not reproduce the round-2 duplicate, so it stays
unattributed. Not tested: gateway-injected MCP tool loops (unreachable
from chatd, whose delegated requests skip the gateway's MCP proxy),
direct Anthropic `msg_` IDs, Bedrock, and threshold-triggered
compaction. Both rounds had a verified isolation proof and a clean
dogfood audit.
- Codex rounds 1 to 3 reviewed the old design. Codex round 4 on
`2e889ba`
([request](https://github.com/coder/coder/pull/29763#issuecomment-5796193371),
13:58Z): no major issues
([result](https://github.com/coder/coder/pull/29763#issuecomment-5796267231),
reviewed commit `2e889ba05c`). No unresolved review threads. Review
passes so far: 4 of 6.
- Final head `2e889ba` is the exact commit remote UAT round 3 and Codex
round 4 covered; required checks green.

</details>

> Xum updated this PR on behalf of @ibetitsmike.
2026-09-23 19:01:24 +02:00
Michael Suchacz e35cbe2a7f fix(coderd/x/chatd): stream compaction summaries (#28886)
Chats could get permanently stuck once their context crossed the
compaction threshold on adaptive-thinking Claude models: every
compaction attempt failed with `compaction produced an empty summary`,
and both automatic and manual compaction take the same path, so the chat
could never recover.

## Problem

`compactionSummaryCall` deliberately set `MaxOutputTokens = nil` because
the summary request was non-streaming and the Anthropic SDK rejects
non-streaming requests whose `max_tokens` implies a completion longer
than ten minutes. But a nil cap makes fantasy's Anthropic provider
silently default to 4096 output tokens, and adaptive-thinking Claude
models can spend that entire budget on a thinking block, leaving zero
summary text.

## Fix

- Stream the compaction summary request. Streaming lifts the Anthropic
SDK's non-streaming duration restriction, which was the only reason the
cap was nil'd.
- Keep the resolved output cap (32k default) on the summary call instead
of nil'ing it.
- When the summary comes back empty with truncation evidence (length
finish reason, or reasoning with no text), fail with `compaction summary
was truncated at the output token cap` instead of the generic
empty-summary error, so the next occurrence is attributable at a glance.

Unit tests cover text-block joining and the truncation classification;
the compaction e2e fixtures now serve streamed summaries and assert the
summary request streams with `max_tokens: 32000`. Dogfood UAT on a
deployment built from this branch verified manual compaction with Claude
Opus 5 (adaptive thinking, twice, with reload persistence) and an OpenAI
regression chat.

> 🤖 Opened by [Xum](https://mux.coder.com) on behalf of @ibetitsmike.
2026-09-21 21:28:45 +02:00
Michael SuchaczandPaul Frybarger 19e2b0415a fix(coderd/x/chatd): preserve tool definitions during compaction (#29436)
Supersedes #28883 by @pfrybar. His commit is carried with authorship
intact (rebased onto main as 32d261f94f); the second commit adds the
overflow fallback found while testing it.

## Problem

Compaction rebuilt the summary request without the conversation's tool
definitions, so the summary prompt no longer shared the turn's prefix
and providers could not reuse the cached context (measurements in
#28883).

Sending the tools with the summary makes that request at least as large
as the generation it summarizes. At a model's real context window
(gpt-3.5-turbo-0125, 16385 tokens) the summary request was rejected with
`Your input exceeds the context window of this model.` on the same
overflow compaction exists to recover from. The chat stayed in `error`,
and both manual `/compact` and the automatic retrigger failed the same
way, so the chat was stuck.

## Fix

`generateCompactionSummary` sends the prepared tool definitions with the
summary request. When the provider rejects that request for size, it
regenerates the summary once without tools, which is the request shape
compaction used before and still fits whenever the history alone does.
The context-window check reuses fantasy's classification and also
matches bad-request wordings fantasy does not parse (OpenAI Responses
API, Anthropic, `context_length_exceeded`).

`BuildToolDefinitions` moves into its own file in `chatloop` so the turn
and the summary share one construction path.

On Anthropic the summary prompt now carries the same `cache_control`
breakpoints as the turn; without them Anthropic never reads the cache,
so the shared tool and history prefix would have been pure cost there. A
compaction override that resolves to a different model shares no prompt
cache and may not support the chat model's provider-native tools, so its
summary request carries no tool definitions; an override resolving to
the chat model itself keeps them. HTTP 413 counts as a size rejection
for the tool-less retry.

## Testing

- Unit:
`TestGenerateCompactionSummaryRetriesWithoutToolsWhenContextTooLarge`,
`TestGenerateCompactionSummaryDoesNotRetryOtherFailures`,
`TestGenerateCompactionSummaryAppliesAnthropicPromptCaching`,
`TestIsContextTooLargeError`, `TestBuildToolDefinitions*`.
- End to end: `TestActiveServer_ManualCompaction/retries the summary
without tools when the prompt exceeds the context window` asserts two
summary requests (with tools, then none), the chat returns to `waiting`
with `last_error` cleared, and the summary is persisted.
- Red-green: with the retry disabled the unit test fails on the returned
provider error and the e2e chat never leaves `error`; with the phrase
matching disabled the Responses, Anthropic, and error-code classifier
cases fail.

## Review record

- Local standards and spec review at 99f7e80a0a9 (pre-rebase content of
8799db2f0e): switched `errors.As` to `errors.AsType`, removed two
comments restating test names, trimmed the phrase list to
provider-attested wordings. Rebase onto main resolved one test-file
conflict (both sides deleted different tests) and inlined the
`chattest.AnthropicErrorResponse` helper that main removed in #29008.
- Codex round 1 on 8799db2f0e: one P1, parent tool definitions sent to
a compaction override on a different provider. Mechanism verified
partially (fantasy skips foreign provider-defined tools with a warning;
`web_search` translates across providers and schemas can be rejected).
Fixed in a7fa7a61e4 by dropping tool definitions when the override is a
separate provider instance, with a red-green case in
`TestActiveServer_CompactionModelOverride`. Thread resolved.
- Codex round 2 on a7fa7a61e4: P1 override tools should drop for any
different model (agreed, guard replaced by an unconditional drop); P2
HTTP 413 should take the tool-less retry (agreed); P2 Anthropic summary
request had no cache breakpoints so the tools bought no cache hit
(confirmed in code, matches earlier dogfood `cache_read=0`). All three
fixed in e7bd72390c with red-green unit and e2e assertions. Threads
resolved.
- Codex round 3 on e7bd72390c: P2 keep tools when the override resolves
to the chat model (agreed, fixed in 6d8d8d6f71 with a red-green e2e
case); P1 missing ARCHITECTURE.md TODO items for the changed compaction
flow (agreed, added in 6d8d8d6f71 per the chatd rule); P2 summary tool
set may differ from the triggering request after a `find_tools`
activation (mechanism confirmed, declined as a cache-efficiency edge
case with the overflow path covered by the retry). Threads resolved.
- Codex round 4 on 6d8d8d6f71: "Didn't find any major issues", reviewed
commit `6d8d8d6f71`, zero unresolved threads. CI green on the same head
(Storybook rerun once after Pixel returned HTTP 500 while uploading
snapshots; this branch has no `site/` changes).

> Xum, an AI coding agent, prepared this PR on behalf of @ibetitsmike.

---------

Co-authored-by: Paul Frybarger <pfrybar@frymail.net>
2026-09-18 23:19:00 +02:00
Ethan 77479408b4 feat: make the Coder Agents stream silence timeout configurable (#29394)
The `chatloop` stream silence guard cancels and retries a model attempt
after `10m` of silence, and it was hard-coded. It's armed before the
request opens, so it also bounds time to first token, and slow local
models (e.g. `llama-server` on a large prompt) can spend longer than
that on prompt processing alone, meaning every attempt just gets
cancelled and retried.

This PR adds `--chat-stream-silence-timeout` /
`CODER_CHAT_STREAM_SILENCE_TIMEOUT` (default `10m`) and passes it to
both the main loop and the advisor. Setting it to `0` disables the guard
entirely, and anything above `24h` is rejected, which also keeps the
watchdog arithmetic below from overflowing.

Whilst doing this I ran into the runner's `15m` task watchdog, which
kept counting during an open stream. That silently cancelled any healthy
stream longer than `15m`, and would've stopped a silence timeout above
`15m` from ever firing. To fix, the silence guard now kicks the watchdog
out to `max(15m, silence + 5m)` whenever it arms or sees a part. Silent
streams still fail through the guard, healthy streams can run for as
long as they like, and hang detection is unchanged everywhere else. With
the guard disabled there's nothing to kick against, so arming it just
stops the watchdog for the rest of the attempt. That does mean the DB
write after the stream isn't bounded either when `0` is set; I think
that's fine for an opt-in setting, and a user can still stop the chat
from the UI if it wedges there.

I did consider removing the guard entirely, but the provider HTTP
clients have no timeout, so a provider that holds the connection open
would only ever be caught as a generic task hang, which is what #25782
was fixing.
2026-09-18 15:24:15 +10:00
Michael Suchacz 8397680687 fix: keep text and image together in MCP tool results (#29332)
## Problem

MCP tool results that mix text and an image (Playwright's default
`browser_take_screenshot` returns `[text, image]`) were reduced to text
only in both MCP conversion paths, so screenshots never reached the
model. The generic tool renderer also showed media results as a JSON
dump of base64.

## Fix

- `mcpclient` and `chattool`: keep all text plus the first usable binary
item. Error flags survive empty and MIME-less results. Only one binary
payload is represented; additional images are still dropped.
- `chatloop`: at execution time, cap tool media at the 10 MiB attachment
limit and correct or reject declared image types by byte signature
(sniffing, not decoding).
- `chatd` prompt build and `chatprovider`: at generation and compaction,
replace media the destination provider cannot accept with the text plus
an omission note (Anthropic/Bedrock: JPEG, PNG, GIF, WebP under the
inline cap; OpenAI/Azure and Google: their image allowlists, applying
the configured vendor's rules behind `openai-compat`). Persisted history
is unchanged, so the UI keeps the payload and provider switches keep
working.
- `site`: results flagged `is_media` render through `ToolResultMedia`
(image with the existing lightbox, or a short notice for non-image
media, plus the text). Unflagged media-shaped JSON stays in the JSON
viewer.
- `go.mod`: pin [coder/fantasy
`9a3598480a71`](https://github.com/coder/fantasy/commit/9a3598480a71e45901bfcfbf566ffa42c6bbdd63),
main's `a511353f83eb` plus the batch-order fix `460d4fdd113b` (Chat
Completions and compatible transports emit every tool result before
synthetic user images). The fix lives on
`mike/mcp-tool-media-batch-order`; no upstream PR yet.

## Validation

Round 3 dogfood UAT on this head, `522bd76a58a`: **PASS**
([chat](https://dogfood.cdr.dev/agents/5e617636-c6f2-46fc-96e5-97ae35ed822e),
real Playwright MCP, Claude Haiku 4.5 and GPT-4o mini through the AI
gateway). Verified in the UI and on the wire: image plus text render
together, the lightbox opens and closes, both survive a reload; audio
shows a notice instead of base64; media-shaped JSON stays in the JSON
viewer; BMP on OpenAI Responses and Chat Completions reaches the model
as text plus an omission note; SVG and PNG histories switch from
Anthropic to OpenAI; cross-provider compaction through the organization
override works; a result over 10 MiB persists only text plus a note, and
a 5.5 MiB PNG is substituted for Anthropic yet still renders in the UI.
Google is not routable on the dogfood gateway (404), so it was not
exercised. The remote agent's own FAIL was a probe artifact (the code
viewer renders in a shadow root) and was overturned on its screenshots
by the independent runner. Rounds 1 and 2 (`65db84a`, `5001de7`) covered
the same API surface earlier.

Known limits: signature sniffing does not catch a truncated PNG with a
valid header (providers return 400 for it), and only the first binary
item of a result is kept. Pre-existing and out of scope: chatd opens a
new MCP session per generation action, so Playwright's default
per-session browser loses page state between steps (UAT used
`--cdp-endpoint`).

Reviews:

| Review | Head | Outcome |
|---|---|---|
| Codex 1 to 5 | `472914f` to `bcf4c72` | 16 findings (empty binary
promoted over text, no provider caps, SVG on Anthropic, provider-switch
gap, Bedrock labels, unbounded media, unvalidated bytes, renderer
gating, lightbox test), all fixed |
| Codex 6, 7 | `8a9a9e0`, `5001de7` | no major issues; one Responses
audio concern [refuted with a live
probe](https://github.com/coder/coder/pull/29332#discussion_r4018514197)
|
|
[Codex](https://github.com/coder/coder/pull/29332#pullrequestreview-5226106293)
| `83e1a10` | two P2s refuted with serializer captures
([MIME](https://github.com/coder/coder/pull/29332#discussion_r4029188332),
[advisor](https://github.com/coder/coder/pull/29332#discussion_r4029160781));
[reassessment](https://github.com/coder/coder/pull/29332#issuecomment-5702295886)
clean |
| Codex | `522bd76` | [no major
issues](https://github.com/coder/coder/pull/29332#issuecomment-5713663081)
on the rewritten tests |
| @DanielleMaywood | `472914f`, `5001de7` | renderer restructure and
explicit audio mapping done; lazy slice cloning removed; two threads
left open for her re-check |

Local pre-push hooks were skipped per Mike's instruction; CI is the
gate.

> Xum acted on Mike's (@ibetitsmike) behalf to create this PR.
2026-09-17 15:09:19 +02:00
Michael Suchacz 0362e5386c fix(coderd/x/chatd/chatloop): tell the model to gather context before retrying an exclusive tool (#29281)
When the model batches `advisor` (an exclusive tool) with other tool
calls, every call in the batch is rejected and the `advisor` result only
says to retry with `advisor` alone. That nudges the model toward
re-calling `advisor` immediately, even when the sibling calls were there
to gather context for it.

Extend the message so the model is told to run the other tools first
when it still needs their output, then retry with only the exclusive
tool call.

Generated by Coder Agents on behalf of @ibetitsmike.
2026-09-15 17:02:48 +02:00
Michael Suchacz 680b2f59cd refactor: remove obsolete chatd code and tests (#29228)
Chatd still carries an unreachable interruption flush path, unused state
and options, an uncalled workspace MCP cache invalidation callback, and
a stream-buffer-drop counter that production never increments. Remove
these paths and their dedicated assertions, and update the generated
metrics inventory and documentation.

Consolidate duplicate publication and rollback tests and move fixture
writers and test-only inspection helpers into test files. Retain the
interruption persistence, state-machine matrix, legacy message format,
and provider compatibility coverage.
2026-09-14 10:51:22 +02:00
Michael Suchacz ac824cd235 chore(coderd): remove dead code from chatd (#29008)
Removes production code in `coderd/x/chatd` that nothing reaches, the
tests that only existed to exercise it, and option/config fields that
callers filled in but chatd never read. No runtime behaviour changes;
the only edits outside `coderd/x/chatd` drop the now-nonexistent fields
at their three call sites (`coderd/coderd.go`, `coderd/exp_chats.go`,
`coderd/coderdtest`).

Net: 42 files, +147 / -1065.

Verification method for every item: `golang.org/x/tools/cmd/deadcode`
(with and without `-test`) rooted at `cmd/coder`, `enterprise/cmd/coder`
and every package that imports chatd; a `go/packages` pass counting
`TypesInfo.Uses` per declared type, const, var, field and method across
the same roots, split into test and non-test use sites, plus a
read-vs-write split for struct fields; `rg` over the whole repo
including `site/src` for json tag names; then `go build ./...`, `go vet
./coderd/x/chatd/...`, `golangci-lint run ./coderd/x/chatd/... ./coderd/
./coderd/coderdtest/`, and `go test ./coderd/x/chatd/...` (all 20
packages) locally.

<details>
<summary>Evidence per deletion</summary>

| Removed | What it was | Why unused |
|---|---|---|
| `subagent.go` `resolveConfiguredModelOverride`, `parseModelOverride`,
`parsedModelOverride` (~135 lines) +
`TestResolveConfiguredModelOverride_*`, `TestParseModelOverride` |
raw-string model override resolver | Only callers were its two tests;
production resolves overrides through `resolveOrganizationModelOverride`
(`title_override.go`, `compaction_override.go`, `subagent.go`). |
| `chatd.go` `removeSkillIndexMessages`, `isSkillIndexMessage` +
`TestSkillIndexRefreshReplacesStaleAliases` | skill-index refresh
helpers | Zero production references; the test simulated a refresh path
no production code performs. |
| `chatd.go` `chatMessage`, `newChatMessage`, `withCreatedBy`,
`appendMessageFields`, `BuildSingleChatMessageInsertParams` (~90 lines)
| insert-params builder | Exported, but every caller was a `chatd_test`
file (6 sites). Replaced by a 25-line `singleChatMessageInsertParams`
helper in `chatd_helpers_test.go`. |
| `chatd.go` `Server.closeInflightAdmission` | one-line wrapper around
`inflightClosed.Store(true)` | Single call site (`Close`); inlined. |
| `chatd.go` `Server.workerID`, `instructionLookupTimeout`,
`pendingChatAcquireInterval`, `maxChatsPerAcquire`,
`chatHeartbeatInterval` | Server struct fields | Assigned in `New` and
never read anywhere (the values reach the worker through
`chatWorkerOptions`, not these fields). |
| `Config.InstructionLookupTimeout`, `homeInstructionLookupTimeout`,
`coderd.Options.ChatdInstructionLookupTimeout`,
`coderdtest.Options.ChatdInstructionLookupTimeout` | timeout plumbing |
The value only landed in the dead `Server.instructionLookupTimeout`; no
CLI flag sets it and nothing reads it. |
| `PromoteQueuedOptions.CreatedBy` | option field | Set by
`coderd/exp_chats.go` and three tests, never read in chatd (promotion
keeps the queued row's `created_by`). |
| `SubmitToolResultsOptions.DynamicTools` | option field | Set by
`coderd/exp_chats.go` (which re-read `chat.DynamicTools` just to pass
it) and three tests, never read in chatd. |
| `attachments.go` `buildAssistantPartsForPersist` (whole file) |
assistant part builder | Stale copy left by #26270; production uses
`buildAssistantParts` in `message_conversion.go`. Its six tests were
retargeted to the live function (now
`message_conversion_internal_test.go`) and a dead-copy assertion in
`subagent_internal_test.go` was dropped. |
| `chatloop.RunOptions` (94 lines) | old loop options struct | Only two
of 23 fields were ever populated, for one internal call to
`prepareMessagesForRequest`, which now takes `model, logger` directly
and no longer returns the ignored canonical slice.
`GenerateAssistantOptions` / `ExecuteLocalToolsOptions` are the live
types. |
| `chatloop.ErrDynamicToolCall`, `ErrStopAfterTool`, `PendingToolCall`,
`PersistedStep.PendingDynamicToolCalls` | dynamic-tool exit protocol |
Zero references outside their declarations (one stale comment in
`chatdebug/service.go` updated). |
| `chatloop.CompactionOptions.Persist`, `.OnError` | callbacks |
`Persist` had zero references; `OnError` was set only by two tests that
asserted it was never invoked. |
| `chatloop` `stepResult.shouldContinue` + its computation | flag |
Written once, never read. |
| `chatloop.StateWaiting` | metric label const | Zero references. |
| `generation.go` `generationOutcome.InsertedMessages`,
`runnerActionMessage` | outcome field | Built on every commit, never
read. |
| `chatstate` `ExecutionState.IsArchived`, `AllOwnershipStates`,
`OwnershipState`/`StateU`/`StateO`, `TransitionAcquire` | state-machine
leftovers | Zero references (the ownership consts became dead once
`AllOwnershipStates` went). |
| `chatstate` `ExecutionState.QueueNonEmpty` | predicate | Only
referenced by the test asserting its own table; the sibling `IsRunnable`
check in that test is kept. |
| `chatdebug.Attempt.RetryClassification`, `.RetryDelayMs` | json fields
| Never assigned anywhere, `omitempty`, and
`retry_classification`/`retry_delay_ms` do not appear in `site/src` or
`typesGenerated.ts`. |
| `chatdebug.RunContext.Kind`, `StepContext.HistoryTipMessageID`,
`RecorderOptions.Provider`/`.Model`, `stepHandle.opts`/`.usage`/`.err` |
context/handle fields | Written (from `chatd.go`,
`chatloop/compaction.go`, `modelcall.go`, `recorder.go`) and never read.
The DB run row keeps its own `kind` column. |
| `chatprovider.MergeProviderAPIKeys` +
`TestMergeProviderAPIKeys_PreservesProviderRegions` | key merge |
Superseded by `ResolveUserProviderKeys`, whose region test already
covers the same behaviour. |
| `chatprovider` `providerCreationError` | one-line `xerrors.Errorf`
wrapper | Single call site; inlined with identical message. |
| `chatretry.IsRetryable` | classifier wrapper | Zero references. |
| `computeruse.IsTool` + `TestComputerUseTool` | one-line forwarder to
`fantasyopenai.IsComputerUseTool` | No production caller; two test
assertions now call the fantasy function directly, and the dropped test
only asserted the library's own constructor. |
| `chatprompt.ToolResultToPart` | one-line forwarder to
`codersdk.ChatMessageToolResult` | Single call site; inlined. |
| `subagent_catalog.go` `formatSubagentDefinitions` | forwarder with
`nil` overrides | Single call site; inlined. |
| `worker.go` `chatWorker.chatWorkerID` | getter | Only tests called it;
they read `opts.WorkerID` directly now. |
| `chattest.AnthropicErrorResponse`, `OpenAIServerErrorResponse`,
`ChatMessageWithParts` (`messages.go` removed) | test fixtures | Zero
references. |
| `chattest` `AnthropicRequest.Options`, `OpenAIRequest.Options` |
`json:",inline"` map fields | `encoding/json` has no inline tag (the
TODO on each said so); never populated, never read. |

</details>

<details>
<summary>Considered and kept</summary>

- `chatstate.CreateChat` (test-only thin wrapper over `insertChat`,
production uses `CreateChatWithID`): documented package API with 25 test
call sites; not worth the churn.
- `chatprompt.MarshalContent` / `MarshalToolResult`: test-only, but they
deliberately produce legacy-format DB rows so the read path's legacy
handling stays covered.
- `chatstate.AllowedExecutionTransitionsFrom` / `AllowedInputStates` /
`AllowedExecutionTransitionOutputs`, `chatdebug.StepFromContext`,
`WithClock`, `WithStaleThreshold`, `WaitUntilIdleForTest`,
`PublishBuffer.BufferedChannels`, `chattool.NewWorkspaceMCPTool`: test
seams that tests actively use to reach otherwise unexported state.
- `AgentCapacityLimiter` interface (one production implementation):
tests inject fakes.
- `applyHookResultMessages` / `appendHookResultMessages`: a named pair
over one placement enum, both used.
- Nil-receiver guards on `*Metrics`, `*Result`, `*streamSyncPoller`,
`*runnerDebugTurn`: nil is a supported "disabled" value on those paths.
- Tests flagged by heuristics as assertion-light
(`TestNopMetrics_DoesNotPanic`, `TestStepHandleFinish_NilHandle`,
`TestRetry_OnRetryNilDoesNotPanic`, the `chatstate`
synthetic-cancellation tests that assert through helpers): they cover
real nil-safety or delegate assertions to helpers; kept.

</details>

<details>
<summary>Out-of-scope findings (not changed here)</summary>

- `chatdebug.Service.FinalizeStale` has had no production caller since
#26270 removed the background sweep loop; only its tests call it, and
comments in `chatd_debug.go` and `database/querier.go` still describe
the loop. Deleting it would erase the implementation a fix needs, so it
is left for a decision.
- `coderd_chatd_chats` (gauge) and
`coderd_chatd_stream_buffer_dropped_total` (counter) are registered and
documented in `docs/admin/integrations/prometheus.md` but never set or
incremented in production; `Metrics.RecordStreamBufferDropped` is
test-only. Removing them changes the exported metric set and the docs,
so they are left alone.

</details>

## Declined review findings

None yet.

> Xum (AI agent) opened this PR on behalf of @ibetitsmike.
2026-09-05 11:39:10 +02:00
Jaayden Halko b428c8feb6 feat(coderd): bill completed local tool batches in agent runtime (#28360)
**Stack**: #28359 → **#28360 (this PR)** → #28361 → #28362

## Summary

Completed local tool batches now count toward Coder Agent runtime
(`chat_messages.runtime_ms`, the source of truth for
`hb_agent_runtime_v1`), excluding sub-agent orchestration tools to avoid
double counting, and without multiplying runtime for parallel tool
calls.

Second of a four-PR stack splitting #28211. Stacked on #28359. Refs
CODAGT-928
(https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).

## Problem

Agent runtime previously measured only model invocation wall clock
(stream open to fully consumed). Time spent executing local tools
between steps, including file operations, terminal commands, workspace
provisioning, and MCP tools, was deliberately excluded, which
undercounts the product definition of "actively processing a task".

Naive inclusion has two hazards: a batch of parallel tool calls would
bill N windows for one wall-clock wait, and `wait_agent` would re-bill
child agents that already bill their own model and tool time.

## Fix

Each completed local tool batch bills one window: the union of the
billed tools' execution intervals, persisted as `runtime_ms` on a
dedicated usage record appended after the batch's tool-result rows. The
record is a tool-role message with `visibility='model'`, so it never
reaches the API, SSE, or clients, and prompt replay drops it because its
single internal `tool-batch-usage` part converts to no provider content.
Its content carries an audit payload (`billed_ms`, `billed_calls`) so
the billed window is inspectable after the fact; real tool-result rows
never carry batch-level runtime. Concurrent calls all start at batch
start, so 5 parallel 10s reads bill 10s, not 50s; serial calls
(`SerialToolCalls`) count only from their own launch, so unbilled waits
before them do not count.

Sub-agent orchestration tools (`spawn_agent`, `wait_agent`,
`message_agent`, `interrupt_agent`, `list_agents`,
`list_subagent_models`, plus the deprecated `close_agent` alias) never
extend the window: every chat, including children, applies the same
rules to its own runtime, so a parent's `wait_agent` window would double
count. A lone `wait_agent` bills 0 and appends no usage record;
`execute` 10s in parallel with `wait_agent` 60s bills 10s. A test pins
the unbilled set to the registered sub-agent tool catalog so they cannot
drift.

`GetTotalChatMessageRuntimeMsInRange` already sums `runtime_ms`
role-agnostically, so the usage record is picked up with no schema,
query, or cron changes.

Client-executed dynamic tools, external agents, parked/idle time, and
retry backoff remain unbilled. Interrupted batches still bill only the
model window; the rest of the stack adds partial-window billing via the
same usage record.

`ExecuteLocalToolsOptions` also gains an optional `ToolBillingRecorder`
(`RecordStart`/`RecordComplete`) observing per-occurrence dispatch-order
execution stamps. This PR wires no recorder; the next PR in the stack
connects it to the message part buffer for interrupt billing.

**NOTE**: Reported agent runtime (`hb_agent_runtime_v1`) increases from
deploy forward. There is no backfill and no feature flag.
2026-08-25 16:40:41 +07:00
Jaayden Halko d44aae37de refactor(coderd/x/chatd): return PersistedStep directly from ExecuteLocalTools (#28359)
**Stack**: **#28359 (this PR)** → #28360 → #28361 → #28362

`chatloop.ToolExecutionOutcome` only wrapped a `PersistedStep`, so
callers had to reach through `outcome.Step` for every field.
`ExecuteLocalTools` now returns the `PersistedStep` directly and the
wrapper type is deleted. No behavior change.

First of a four-PR stack that makes local tool execution count toward
Coder Agent runtime (splitting #28211). This refactor keeps the rename
noise out of the feature PRs.

Refs CODAGT-928
(https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).
2026-08-25 16:31:06 +07:00
Michael Suchacz a48aedd924 fix(coderd/x): surface Gemini malformed-function-call stream deaths as retryable errors (#28470)
## Problem

When Gemini's OpenAI-compatible endpoint rejects a model-generated
function call server-side, it ends the SSE stream cleanly with the
nonstandard finish reason `function_call_filter:
MALFORMED_FUNCTION_CALL` after streaming only thought summaries; the
rejected call never reaches the wire. chatd treated this as a normal
completion: fantasy maps the unrecognized finish reason to `unknown`,
the reasoning block never closes (the only non-thought delta is the
`</thought>` marker, which the transport seam strips to an empty string,
and the openaicompat hook only ends reasoning on a non-empty content
delta), so the step accumulates no content and the generation loop
finishes the turn as complete. The user sees the model think for ~40
seconds and then nothing: no assistant message, no `last_error`, chat
status `waiting`. Observed twice in a row in production on
gemini-3.7-flash, with "Resume" reproducing it identically.

## Fix

Two independent layers:

- `coderd/x/googleopenai`: the stream rewrite now converts any chunk
whose `finish_reason` starts with `function_call_filter` into an
OpenAI-style SSE `{"error": ...}` event embedding the raw reason.
openai-go turns error-bearing events into stream errors, so the failure
rides the existing stream-error path instead of ending the stream
cleanly.
- `coderd/x/chatd/chaterror`: classifies that injected error as
retryable (kind `generic`, provider `google`) with a clear user-facing
message, so the existing generation retry machinery re-runs the step and
persists a `last_error` if retries exhaust.
- `coderd/x/chatd/chatloop`: provider-agnostic guard: a step that
produced no user-visible content and no tool calls under a finish reason
of `unknown`, `error`, `other`, or `tool-calls` (a tool-calls finish
that delivered zero calls) now returns a retryable error instead of
silently completing the turn. `stop` and `length` finishes keep their
existing semantics.

Tests cover the seam rewrite (live-capture SSE shape plus standard
finish reason passthrough), the new classification, and the chatloop
guard (error cases plus preserved stop, length, text, and tool-call
behavior). Each layer was red-green verified independently.

Remote dogfood UAT ran against this exact commit: normal reasoning and
tool-call chats on a real model complete cleanly with no spurious guard
errors and no retry loops. The Google-side failure itself is not
deterministically triggerable against live Gemini and is owned by the
unit tests.

> Xum acted on Mike's (@ibetitsmike) behalf.

<!-- xum-attribution: model=claude-opus, thinking=enabled -->
2026-08-24 17:09:00 +02:00
Michael Suchacz 9168dbf4d3 refactor(coderd/x/chatd): unify LLM model-call construction behind a resolver (#28145)
Centralizes every LLM client build in chatd behind one resolver and
makes all model calls behave the same, instead of preserving per-flow
quirks.

## Why

`chatd` built LLM clients in five places (standard turn, compaction
override, quickgen, title override, advisor), each repeating config
parsing, route resolution, client construction, and debug wrapping with
slightly different behavior. An earlier iteration of this PR centralized
construction but preserved every historical quirk behind
purpose-specific spec constructors and policy flags. Review concluded
the quirks themselves were the problem.

## What changed

- `Server.resolveModelCall` (`modelcall.go`) is the single pipeline from
a `modelCallSpec` to a ready client plus call metadata. Callers build
spec literals inline; the only spec fields are data: model source (chat
config, explicit config row, or fixed provider/model pair for computer
use), requested reasoning effort, chatd-scoped routing, and build
options.
- Uniform behavior for every call:
- Provider options are always derived. Whole-chat summaries, turn status
labels, and chat-model compaction summaries previously sent none; they
now follow the config like everything else.
- `MaxOutputTokens` always defaults to 32k when the config leaves it
unset. The compaction summary call nulls the cap after copying the
template because it is non-streaming and the Anthropic SDK rejects
non-streaming requests whose cap implies a completion over ten minutes.
- Debug recording always wraps the client when chat debug is enabled,
labeled with the resolved provider/model. The debug-rebuild paths
(`manualTitleDebugSpec`, `quickgenDebugSpec`) are deleted; debug helpers
now only create and finalize run records, since run association flows
through the generate context, not the model.
- Clients are always constructed with the configured model string and
unmutated route, so AI Gateway validation sees the name exactly as
configured. The resolved identity feeds metadata, logs, and debug labels
only.
- `resolvedModelCall.newCall` / `newObjectCall` are the only production
constructors of `fantasy.Call` / `fantasy.ObjectCall`; object calls now
carry the config's sampling parameters.
- Net: -240 lines.

## Validation

- Full `./coderd/x/chatd/...` suite green (PostgreSQL-backed).
- Red-green toggles verified for the three key unifications (option
derivation, token default, summary cap removal).
- ARCHITECTURE.md: prose the refactor made stale is deleted; no new
resolver documentation is added.

> Mux acted on Mike's behalf for this PR.
2026-08-20 20:54:52 +00:00
Danielle Maywood 5480299cef fix: serialize empty MCP tool properties as {} not null for OpenAI (#28380)
## Summary

Fixed a bug where MCP tools with an empty `properties` schema had their
tool definitions sent to OpenAI with `"properties": null`, causing
OpenAI to reject the request.

## Problem

When an MCP server reports a tool with an empty input schema, e.g.:

```json
{
  "type": "object",
  "properties": {},
  "required": []
}
```

Coder was forwarding the tool definition to OpenAI as:

```json
{
  "type": "object",
  "properties": null
}
```

OpenAI's function-calling API rejects `properties: null`, making these
tools unusable with OpenAI models.

## Root cause

The empty `properties` object collapsed to Go `nil` at two layers:

1. `agent/x/agentmcp/manager.go` `toolInputSchemaMap` stripped the
present-but-empty `"properties": {}` (guarded by `len(properties) > 0`),
so the wire copy lost it entirely.
2. `coderd/x/chatd/chatloop/chatloop.go` `buildToolDefinitions` embedded
`info.Parameters` directly into the input schema. When nil,
`encoding/json` marshals it as `null`.

## Fix

- `buildToolDefinitions` now substitutes an empty object when
`Parameters` is nil, mirroring the existing `required` guard (the sink,
protects all tool sources).
- `toolInputSchemaMap` now preserves an empty `properties` object
instead of dropping it (the upstream producer).

## Tests

Added regression tests (failing before, passing after) covering both
layers and asserting the schema serializes to
`{"type":"object","properties":{}}`:

- `TestBuildToolDefinitionsNilPropertiesBecomesEmptyObject`
- `TestToolInputSchemaMapPreservesEmptyProperties`
-
`TestWorkspaceMCPToolInfosFromResources/EmptyPropertiesYieldsEmptySchema`

---

*This PR was generated by Coder Agents.*
2026-08-20 15:56:58 +02:00
Michael Suchacz 7724ee281a feat: defer MCP tool schemas behind a find_tools search (#28225)
## Summary

When the `mcp-tool-search` experiment is enabled, chatd stops inlining
connected MCP tool schemas into every generation. It instead exposes a
built-in `find_tools` tool whose description carries a compact catalog
of the deferred tools, and only ships full JSON schemas for tools the
model has activated by searching or by calling them directly.

Closes [CODAGT-760](https://linear.app/coder/issue/CODAGT-760).

## Problem

Tool-heavy agent configurations (GitHub, Linear, Notion, and dev-tooling
MCP servers) inline over 100k tokens of tool schema definitions into
every generation. Initial uncached requests reached ~216k tokens with
time-to-first-token close to nine minutes, while the model typically
invokes only a handful of tools per turn.

## How it works

- `decideMCPToolSearch` defers external and workspace `.mcp.json` MCP
tools whenever the experiment is enabled. Native, dynamic, provider,
skill, and transport tools are never deferred.
- `find_tools` embeds a server-grouped catalog in its tool description
(degrading to names-only, then counts-only, then a constant-size summary
past a context-scaled size cap) and scores keyword matches across tool
names, descriptions, parameter schemas, and server metadata. Queries can
scope to one server with a `server:` prefix, and exact `names` arguments
always activate.
- Activation state is ephemeral: it is re-derived each generation from
surviving chat history (`find_tools` results and direct calls to
deferred tools), so activations naturally lapse when compaction
summarizes them away. Aggregate activated schema weight is capped at 10%
of the context window, shedding the least recently activated schemas
first; `find_tools` shares that budget across parallel calls in one
step. No new persistence.
- Deferred tools stay registered for execution, so the model can call a
cataloged tool directly without searching first; the schema is activated
for subsequent steps.
- Fail-open: the experiment being disabled, an empty candidate set, or
an MCP tool named `find_tools` all disable deferral, leaving today's
behavior byte-identical on the wire.
- Prometheus counters/histograms track `find_tools` calls, matches,
activations, and deferred token weight.
- The conversation timeline renders `find_tools` calls with a collapsed
search summary and expandable match list, falling back to the generic
renderer on malformed payloads.

## Validation

- Unit tests for the catalog, matcher, experiment-gated decision, and
activation derivation; end-to-end chatd generation tests covering
search-then-call, direct-call activation, experiment-off wire parity,
compaction lapse, and subagent tool gating.
- Storybook interaction tests for the timeline rendering and
malformed-payload fallback.
- Remote dogfood UAT on dev.coder.com passed: deferral with a real MCP
server and Anthropic model, direct calls without prior search,
activation persistence across turns, experiment-off parity, and clean
UI/console.

> Disclosure: Mux (AI agent) authored this PR on Mike's behalf.
2026-08-18 19:12:47 +02:00
4b7494be72 feat: harden chat generation runtime instrumentation for billing (#27451)
Closes CODAGT-835

## Summary

`chat_messages.runtime_ms` becomes the billing source of truth for Coder
Agents runtime (summed hourly by #27312), but it was built for
debugging: the June refactor (#26270) silently stopped recording
tool-step runtime, compaction was never measured, and interrupted turns
lost their partial runtime entirely. This PR defines the billable
metric, closes the paths that dropped it, and documents the definition
where the data lives.

## The billable definition

**`runtime_ms` is the wall-clock duration of the model invocation that
produced the persisted message content**, measured from just before the
provider stream opens until it is fully consumed.

What counts:

- Assistant generation steps, in top-level and sub-agent chats
(sub-agents are ordinary chats on the same generation path).
- Compaction summarization calls, persisted on the compaction assistant
message (**new**).
- Interrupted attempts: the message-part episode's lifetime is persisted
on the partial assistant message committed by `FinishInterruption`, so
partial generation time survives interruption (**new**; measured via a
new `Buffer.EpisodeDuration`, which works even though the generation
goroutine and the interrupt task are different tasks).

What deliberately does not count (each is documented in code and docs):

- **Local tool execution.** Tool wall time includes idle waits, most
importantly `wait_agent` polling a sub-agent chat that already bills its
own model invocations; billing the batch would double count, and
excluding one tool from a concurrent batch's wall time is ill-defined.
Pre-refactor instrumentation did include tool time; this makes the
exclusion an explicit product definition instead of a silent regression.
- **Failed model calls whose output is discarded** (retried attempts,
terminal errors, content-filter refusals). They persist no content, so
they bill nothing; billing errs toward undercounting. Notably a
stream-silence timeout can burn 10 idle minutes before a retry, which
should not be billable "active generation". If product later wants
failed attempts billed, that needs a place to persist runtime on error
turns (`FinishError` inserts no rows today) and is a deliberate
follow-up, not instrumentation drift.
- **Ancillary calls that produce no chat messages** (title generation,
advisor, turn summaries) and all idle/parked time (`requires_action`,
queueing).

The definition is documented as `COMMENT ON COLUMN
chat_messages.runtime_ms` (migration 000551, surfacing as a Go doc
comment on `ChatMessage.RuntimeMs`), on
`chatloop.PersistedStep.Runtime`, in the chatd architecture doc, and in
the Spend Management docs page.

## Index for the hourly scan

None needed: `GetTotalChatMessageRuntimeMsInRange` (#27312) filters an
hour-wide `created_at` range, which the existing
`idx_chat_messages_created_at` b-tree already serves; the residual
`runtime_ms IS NOT NULL` filter applies to one hour of rows. A partial
index would add permanent write amplification for a query that runs once
an hour.

> [!NOTE]
> Migration 000551 is also claimed by #27312; whichever merges second
renumbers via `fix_migration_numbers.sh`.

## Tests

- End-to-end: the existing full-server generation test now asserts
`RuntimeMs.Valid` on the committed assistant row (it previously read
`.Int64` without checking `.Valid`, so it passed on NULL).
- Interrupted turn: full task-level test (real DB, mock clock) asserting
the partial assistant message persists the attempt's runtime.
- Errored stream: asserts a failed invocation yields no step and no
runtime.
- Tool-using turn: asserts runtime lands on the assistant row only and
tool rows stay NULL.
- Compaction: asserts the summarization call duration is recorded and
lands on the compaction assistant message only.
- `messagepartbuffer.EpisodeDuration` unit coverage.

Blocks: CODAGT-843 (B3), CODAGT-838 (D8).

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Hugo Dutka <hugo@coder.com>
2026-08-06 16:09:38 +07:00
Michael Suchacz c17bed25e0 feat: wire chat lifecycle hooks into chatd (#27429)
Wires chat lifecycle hooks into chatd, gated by the
`agent-lifecycle-hooks` experiment. Part of the lifecycle hooks stack
(#27401, #27428, #27430). See `docs/admin/setup/chat-lifecycle-hooks.md`
for the consumer-facing contract.

## Summary

When a hook URL is configured, chatd dispatches `session_start`,
`user_prompt_submit`, `pre_tool_use`, `post_tool_use`, `pre_compact`,
`post_compact`, and `stop` events to the consumer and applies its
responses.

## Design

- **Stateless**: Coder stores no hook dispatch or decision state.
Delivery is at least once; consumers deduplicate on stable payload
identifiers (chat ID, event type, tool-use ID) and answer duplicates
with the same decision.
- **Admission-time prompt effects**: `user_prompt_submit` dispatches
exactly once per submission (create, send, queue, edit, subagent spawn)
and folds its effects into the stored prompt as typed message parts:
original-or-overridden user parts, then model-only `hook-context`, then
a user-visible `hook-notice`. Hook context is stripped from every
client-facing conversion; hook notices are excluded from model prompts.
The server rejects hook parts in client-submitted content.
- **Tool gating**: `pre_tool_use` allow can override tool input; deny
becomes a synthetic denied tool result, with any returned model context
persisted as a model-only transcript row so it never reaches clients.
The denial text identifies an external policy (the deployment's
lifecycle hook) as the source and marks the decision as persistent, so
the model explains the denial instead of retrying it or misreporting it
as an infrastructure failure.
- **Fail closed**: a dispatch failure rejects the triggering request or
moves the chat to the error state in the same transaction as the
affected step, so a runnable state is never published with unapproved
content.
- **Admission before persistence**: `pre_tool_use` is dispatched for the
calls the model produced, before the assistant message is stored. See
"Staged tool admission" below.
- **Fresh dispatch per tool call**: every non-provider-executed tool
call is decided by its own `pre_tool_use` dispatch; Coder never reuses
an earlier decision on the consumer's behalf. Retries re-dispatch the
same logical event.

## Structure

All hook dispatch flows through one seam: entry points build a
`chathooks.Chat` (chat identity) and a `chathooks.Message` (event
details) and call `Trigger.Trigger`, the only component that talks to
the dispatcher. The integration lives in the `coderd/x/chatd/chathooks`
subpackage, split by responsibility:

- `trigger.go`: the trigger seam; builds the wire envelope per event,
normalizes deny into a typed error, and holds the package's single
enabled-check.
- `effects.go`: pure conversion of hook results into transcript rows and
prompt parts.
- `errors.go`: failure classification (dispatch error messages, denial
mapping, tool-result dispatch-failure scanning).
- `tooluse.go`: the tool-call gate (`pre_tool_use` preflight,
`post_tool_use` payloads, applying admitted input to the step).

Server-bound glue stays in `coderd/x/chatd/hook_server.go`: the
chat-parking dispatch error handlers, the step-commit row insertion
wrappers, and the dynamic post-tool-use state loader, which depends on
chatd validation types.

This PR adopts the `codersdk/x/agenthooks` and
`coderd/x/agenthooks/dispatch` import paths introduced at the tip of
#27401; intermediate commits still reference the pre-move paths and are
not individually buildable.

## Staged tool admission

`pre_tool_use` originally ran at tool execution time, which is after the
assistant message carrying the tool call was already committed. An
`input_override` therefore had to rewrite stored message content in
place. @hugodutka pointed out that chatd treats message content as
immutable, and that the rewrite was a shortcut rather than a
requirement.

It was also a correctness problem in its own right: the rewrite only
updated the database, so the transcript could show one input while a
different one had executed.

The hook now runs before the step is persisted:

```text
provider stream ends (tool calls complete, in memory)
  -> pre_tool_use dispatch per call
  -> ONE transaction: assistant row with admitted inputs, synthetic denials, hook rows
  -> execute
```

The step is inserted once, carrying the input the tool runs with.
`UpdateChatMessageContentByID` and `Tx.UpdateMessageContent` are deleted
from #27428, so message content stays immutable.

Two consequences, both intentional:

- **Clients converge rather than wait.** Tool-call parts still stream
live, so a rewritten call briefly shows the model's proposed input
before the committed message replaces it. The chat store already clears
stream state when an assistant message arrives, so the stored input wins
with no frontend change and no added latency before tool cards appear.
- **A call already in history was already admitted.** Execution consumes
the stored input instead of dispatching a second decision, which keeps
one dispatch and one set of hook effects per call. A consumer policy
change between admission and execution applies to later calls, not to
calls already admitted.

The per-chat debug endpoint still records the provider's original tool
input. Its purpose is to report provider behavior, and it requires an
explicit per-chat debug flag; the invariant here covers the transcript.

## Configuration

Adds `chat-hook-url`, `chat-hook-secret`, `chat-hook-timeout`, and
`chat-hook-enabled` deployment options with startup validation. The
flags are hidden from `coder server --help` while the feature is
experimental; the setup guide documents them.

## Tool input validation

Built-in tool arguments reach a consumer as raw JSON with key spelling
preserved, but the tools decode those bytes with Go, which matches
struct fields case-insensitively and keeps the last match. A policy
reading `path` could therefore authorize one value while the tool
executed another, and a lone case variant such as `{"PATH":"/secret"}`
was invisible to a policy checking for `path`.

Coder now rejects a built-in tool call whose input repeats a key or
spells a schema property with different capitalization, before the
`pre_tool_use` dispatch, so a consumer is never asked to authorize bytes
whose meaning depends on the reader. Rejected calls produce an error
result the model can retry; unambiguous calls in the same batch still
run. A consumer-authored `input_override` is rechecked after the
dispatch and fails the turn closed, because the model cannot correct it.
Dynamic and MCP inputs are excluded because the client and the workspace
agent execute those calls rather than coderd.

Two paths needed more than a schema check. Execution resolves a
deprecated tool name to its canonical tool, so validation resolves
aliases first. The `edit_files` decoder also reads `search` and
`replace`, which its schema does not advertise, so those aliases are now
matched exactly and their case variants ignored.

A hook denial now returns a structured 403 carrying `kind:
"hook_denied"`, mirroring the dispatch-failure response that already
carries its own kind. Without it a client cannot tell a policy decision
apart from a generic failure, and the chat UI titled a denial "Request
failed". Adding a kind needs no migration: `ChatErrorKind` is persisted
only inside the JSONB `chats.last_error` column, whose decoder accepts
unknown kinds.

The hook docs also correct the tool-input convergence window. A batch
dispatches sequentially before the assistant row commits, so the
original input stays visible for a span that scales with the number of
tool calls in the step rather than a single hook timeout.

> This PR was written by Mux, an AI coding agent, on Mike's behalf.
2026-07-29 11:39:12 +00:00
Michael Suchacz e9951b07d4 fix(coderd/x/chatd/chatloop): surface reasoning-only content-filter refusals as terminal errors (#27476)
## Problem

Anthropic can end a stream with `stop_reason: "refusal"` after reasoning
content has already streamed. The content-filter guard in
`chatloop.GenerateAssistant` only fired when the step content was
completely empty, so a reasoning-only refusal bypassed it: the turn
finished as `status=waiting` with `last_error=null`, and the user saw
the chat silently stop mid-turn with no explanation. This looked like a
Coder fault when the provider had rejected the response. Observed twice
in dogfood on 2026-07-23 (chat `c72f99fc`, debug steps show
`finish_reason=content-filter` with reasoning-only content).

## Change

Treat a content-filter finish as terminal whenever the step produced no
user-visible output. A new `hasUserVisibleContent` helper counts any
non-reasoning part (text, tool call, tool result) as user-visible;
reasoning-only or empty steps now return the existing
`contentFilterError`, which flows through the established pipeline:
classified `ChatErrorKindContentFilter` (non-retryable, refusal
category/detail when provided), persisted `chats.last_error`, streamed
error event, and the "Response blocked" callout in the chat UI.

Behavior for steps with visible text or tool calls is unchanged, and the
frontend needs no changes.

## Testing

- New regression subtest `ReasoningOnlyContentSurfacesTerminalError`
(reasoning stream then content-filter finish) beside the existing
empty-content and partial-content subtests, which are unchanged.
- `go test ./coderd/x/chatd/...` and lint pass.
- Dogfood UAT against a local dev instance with a mock Anthropic
upstream passed all three scenarios: reasoning-only refusal shows the
"Response blocked" callout with `last_error.kind=content_filter` and no
retry affordance; text-then-refusal still completes normally; empty
refusal still errors.

> This PR was created by Mux acting on Mike's behalf.
2026-07-24 19:40:53 +02:00
Michael Suchacz 3227cac217 feat: add manual chat compaction via /compact (#27081)
Adds a user-triggered `/compact` action for Coder Agents chats: typing
`/compact` in the composer (or picking it from the `/` trigger menu)
summarizes the conversation so far to free up context window space.

## How it works

- New `POST /api/experimental/chats/{chat}/compact` endpoint
(owner-only, RBAC `ActionUpdate`, excluded from the public API reference
via `x-apidocgen skip`). It marks the chat with a durable one-shot
`chats.compaction_requested_at` signal and moves it `waiting -> running`
via a new `RequestCompaction` state transition; no message row is
inserted. AI Gateway attribution needs no per-request key: generation
preparation resolves the owner's synthetic API key (#27170) like any
other turn.
- `RequestCompaction` hands off chat ownership (clears
`worker_id`/`runner_id`) so a worker acquisition hint is published;
since the transition changes no history, the previous runner could
otherwise miss the request under reordered pubsub delivery.
- The background chat worker picks the chat up like any other turn. A
pending manual request takes precedence over turn completion in the
generation decision, and forces compaction even below the automatic
threshold (and when compaction is disabled via threshold=100). The
commit step consumes the request marker in the same transaction; any
transition that ends the turn clears stale markers.
- The summary triplet reuses the automatic-compaction path, now tagged
with a `source` (`automatic` | `manual`) that is plumbed through
streamed progress parts, persisted tool JSON, and the UI label
("Summarized (manual)").
- Validation order: busy chats reject with 409 (state-machine conflict),
empty/already-compacted chats with 409 "nothing to compact", archived
chats with 400; the owner usage-limit check runs last so no-op requests
surface the specific conflict instead of a limit error.
- Web UI: the `/` trigger menu now has a built-in "Commands" group
listing `/compact`; submit intercepts exactly `/compact` and calls the
endpoint instead of sending a message. A personal or workspace skill
named `compact` takes precedence over the built-in command; while skill
collisions are still resolving, an exact `/compact` submission is
blocked with a retryable hint instead of leaking as message text.
History and queued-message edits are never intercepted. After
compaction, the context usage indicator resets to its unknown state
until the next assistant response reports fresh usage, instead of
showing the stale pre-compaction number.
- codersdk: `ExperimentalClient.CompactChat`.

Worker-path execution (rather than compacting synchronously in the
handler) reuses the existing lock fencing, live "Summarizing..."
streaming, retry accounting, restart resilience, and debug-run
observability. Rationale documented in `coderd/x/chatd/ARCHITECTURE.md`.

## Testing

- State machine: transition-matrix coverage for `RequestCompaction`,
marker lifecycle tests (carried by lease renewals/queue appends, cleared
by terminal transitions, consumed by commit), ownership handoff +
acquisition hint assertions.
- Worker: decision-ordering and forced-compaction unit tests;
active-server end-to-end test (manual compact below threshold produces a
`source=manual` summary, returns to `waiting`, no assistant follow-up;
busy chat rejected).
- API: success, archived, non-owner, RBAC-denied, empty-chat, no-daemon
cases; usage-limit ordering (at-limit owners still get
state/nothing-to-compact conflicts for no-op requests, with marker
rollback).
- Frontend: Storybook play tests for the Commands menu group, submit
intercept, skill-name collision, queued-edit passthrough, and
manual/automatic tool rendering; unit tests for command availability
resolution and the post-compaction context usage reset.

> This PR was created by Mux, an AI coding agent, working on Mike's
behalf.
2026-07-21 10:58:08 +02:00
Ethan cc11c8a536 feat: surface model content-filter refusals as a blocked chat error (#27118)
Blocked turns from a provider's content filter (Anthropic's `refusal`
stop reason with empty content) previously ended silently on the
"Thinking" spinner. They now end as a terminal `content_filter` error
that renders as a "Response blocked" message with the provider's
category and explanation.

<img width="888" height="335" alt="image"
src="https://github.com/user-attachments/assets/cef85a59-4091-4e62-9d45-1eb06748db48"
/>


Closes CODAGT-611

Follow-ups will involve implementing fallbacks, but this alone is pretty
important
2026-07-15 16:36:31 +10:00
Michael Suchacz a567f6a89f feat: allow admins to override the chat compaction model (#27151) 2026-07-14 16:57:04 +02:00
Michael Suchacz 0ae4554de9 fix(coderd/x/chatd/chatloop): remove compaction timeout (#27007) 2026-07-07 12:22:59 +02:00
Cian Johnston b21e0717d5 feat: remove chat chain mode (#26980)
Removes OpenAI Responses "chain mode" from chatd. Closes CODAGT-445.

- Deletes `chatopenai/responses.go` (chain detection, activation, prompt filtering, response ID extraction) and its tests.
- Deletes the `ChainBroken` classification in `chaterror` and the chatloop retry bookkeeping that disabled chain mode mid-generation.
- Drops the `chain_broken` label from the `coderd_chatd_stream_retries_total` metric.
- Stops reading and writing `chat_messages.provider_response_id`
- Deletes the dead `ClearChatMessageProviderResponseIDsByChatID` query. Dropping the column is a follow-up migration.
- Deletes three chatloop hooks no caller sets (`ReloadMessages`, `DisableChainMode`, `PrepareMessages`), the dead `const AgentChatContextSentinelPath`, and stale chain-mode comments.

🤖 Generated by Coder Agents on behalf of @johnstcn.
2026-07-06 11:57:12 +01:00
Mathias Fredriksson 59fcc9c0ad feat: improve sub-agent orchestration tools (#26673)
Tool errors caused orchestrators to abandon spawned agents. Bare error
responses and the close_agent name framed delegation as one-shot: one
transient failure or timeout ended the work, and the orchestrator had no
way to recover or reuse agents.

Renames close_agent to interrupt_agent with a hidden backward-compatible
alias. wait_agent and message_agent return structured payloads instead
of bare errors, so the orchestrator can retry after a timeout, recover
from an error status, or redirect an idle agent. Adds list_agents so
orchestrators can rediscover spawned agents. Adds root-only
orchestration guidance for error recovery.
2026-06-26 13:41:43 +03:00
Kyle Carberry 32217259b7 feat: cap tool output to fit the model context window (#26637)
## Problem

Local tool results were persisted and replayed to the model verbatim,
with no size cap. A single oversized result, most often a multi-megabyte
response from an MCP tool, overflows the prompt on the next request.
Every retry rebuilds the same history and fails the same way, leaving
the chat wedged in `error`. Auto-compaction is reactive (token usage is
only known after a response), so it can't catch a single result that
blows the very next request.

## Fix

Cap every locally-executed tool result at its single choke point,
`executeSingleTool` in `chatloop`, so the cap covers built-in tools,
**global (deployment-pinned) MCP**, **workspace MCP**, and provider
runners uniformly. Because this runs before the result is published to
the live stream and before it is committed, the SSE preview, the
persisted message, and the model replay all see the same bounded output.

The budget is token-aware: a single tool result may use at most half the
model's context window (`~4 bytes/token`), with a `16KB` floor and a
`64KB` default when the window is unknown. Truncation keeps the head and
tail of the output and replaces the middle with a marker telling the
model how much was removed and to narrow its query; it is UTF-8 safe and
never exceeds the budget. Binary media `Data` is passed through
untouched (only the text payload is bounded).

A `coderd_chatd_tool_result_truncated_total{provider,model,tool_name}`
counter and a warning log record each truncation.

## Out of scope

- Provider-executed results (e.g. web search) arrive via the stream, not
`executeSingleTool`.
- Dynamic/external tool results submitted through the `/tool-results`
API are validated as JSON elsewhere.
- Cumulative growth across many results is still handled by context
compaction; this change only bounds any single result.

<details>
<summary>Implementation notes</summary>

- New `coderd/x/chatd/chatloop/tooltruncate.go`:
`toolResultByteBudget(contextLimitTokens)` and
`truncateToolResultText(text, maxBytes)` (pure, unit-tested).
- `chatloop.go`: added `ContextLimit` to `ExecuteLocalToolsOptions`;
threaded a computed byte budget through `executeTools` into
`executeSingleTool`, where `resp.Content` is capped for the text,
media-text, and error branches.
- `generation.go`: passes `ContextLimit: prepared.ContextLimitFallback`
(the model's configured context limit).
- `metrics.go`: new `ToolResultTruncatedTotal` counter +
`RecordToolResultTruncated`.
- Tunable knobs live as constants in `tooltruncate.go`
(`toolResultContextDivisor = 2`, `bytesPerTokenEstimate`,
`minToolResultBytes`, `defaultToolResultBytes`).

Verified: `go build ./coderd/x/chatd/...`, `go test
./coderd/x/chatd/chatloop/...`, and the `chatd` test binary compiles.

</details>

---

Resolves CODAGT-678

Generated by Coder Agents on behalf of @kylecarbs.
2026-06-24 09:16:38 -06:00
Cian Johnston 21a2652343 fix(coderd/x/chatd): show correct provider and clean detail for Bedrock errors (#26338)
## Problem
Bedrock errors (routed through aibridge) showed the wrong provider
("Anthropic ...")
and a doubly-wrapped detail string instead of the clean message.

## Fix (chatd only)
- **Provider label:** thread the configured provider to error
classification via
`GenerateAssistantOptions.ErrorProvider`. Transport provider still
drives prompt
  prep, sanitization, and metric labels (unchanged).
- **Detail:** unwrap the SDK transport wrapper (`METHOD "URL": NNN
{body}`) to surface
the inner message; handles top-level `{"message":...}` and nested
`{"error":{"message":...}}`.

## Notes
- Surfacing a top-level `message` now applies to all providers
(intentional; nested wins when both present).
- The advisor path keeps the transport label; accepted as-is (it returns
`err.Error()`, not the classification).
- Fixes display in chatd only; the wrapper originates in aibridge (out
of scope here).

🤖 Generated by Coder Agents.
2026-06-16 11:14:37 +01:00
Hugo Dutka 4debd23cbb fix: chatd refactor (#26270)
Implements the chatd stabilization RFC.

Combines:
- https://github.com/coder/coder/pull/25908
- https://github.com/coder/coder/pull/25923
- https://github.com/coder/coder/pull/26109
- https://github.com/coder/coder/pull/26110
- https://github.com/coder/coder/pull/26111
- https://github.com/coder/coder/pull/26112
2026-06-12 13:33:12 +02:00
Ethan 3ab1323bc9 fix!: rename chat stream silence timeout error (#25973)
Renames the Agents chat stream-silence error from `startup_timeout` to
`stream_silence_timeout` now that the timeout applies to any gap between
provider stream parts, not just first-token startup.

Updates the SDK enum, generated API docs/types, chat error copy, and
Agents UI stories/status labels so the user-facing wording describes a
stalled provider response instead of startup delay.

> **Breaking change:** This is a very minor breaking change for the
Coder Agents API: the public chat error kind enum no longer includes
`startup_timeout`, so clients matching that specific value should handle
`stream_silence_timeout` instead.
2026-06-04 18:36:02 +10:00
Ethan 7e2f7198dd fix(coderd/x/chatd/chatloop): use stream silence timeout (#25782)
Replaces the 60 second first-token timeout in the chat loop with a 10
minute stream-silence timeout.

Previously, the guard bounded only the gap before the first stream part.
Once any part arrived the attempt could hang indefinitely if the
provider stopped streaming without closing the connection, and even
normal long-running responses could be killed after 60 seconds if the
provider was slow to emit the first token.

The guard now arms when a model attempt opens its stream, resets on
every received stream part, and fires after 10 minutes of complete
silence. The existing retry path still handles the timeout, and the
public `startup_timeout` error kind is preserved to avoid API and
frontend churn.

10 minutes matches the default request timeout used by the Anthropic and
OpenAI Python SDKs.


Closes CODAGT-493
2026-05-28 21:02:40 +10:00
Kyle CarberryandCoder Agent 385146000b feat: record created_at/completed_at on reasoning ChatMessageParts (#24789)
Records reasoning start and end times on persisted reasoning
`ChatMessagePart`s so reasoning duration can be computed for stored
chats. Backend-only: no SSE changes and no frontend rendering ship in
this PR.

The `created_at` field on `ChatMessagePart` is extended to also be
present on `reasoning` parts (it previously appeared only on `tool-call`
and `tool-result`), and a new `completed_at` field is added for
`reasoning` parts.

### How timestamps are recorded

- `StreamPartTypeReasoningStart`: stamp `startedAt = dbtime.Now()` on
the active reasoning state.
- `StreamPartTypeReasoningEnd`: stamp `completedAt = dbtime.Now()` and
append both into parallel `[]time.Time` slices on `stepResult`.
- Persistence reads the slices in occurrence order (reasoning has no
provider-side ID) and applies them to the matching `ChatMessagePart` via
`buildAssistantPartsForPersist`. The first reasoning block's stamps go
onto the first reasoning part, and so on.
- `flushActiveState` flushes partial reasoning interrupted before
`StreamPartTypeReasoningEnd` with `startedAt` from the active state and
`completedAt = dbtime.Now()` at the interruption.

### Why two fields, not one?

Tool calls and results are point events. The frontend computes their
duration by subtracting the call's `created_at` from the result's
`created_at`. Reasoning is one assistant part that brackets a span, so
we record both endpoints on the part itself.

### Why not stamp in `PartFromContent`?

Same rationale as #24101: `PartFromContent` is called during both SSE
publishing and persistence. Stamping there would yield incorrect
persistence-time timestamps for reasoning blocks that finished much
earlier in the step. Instead we capture in the chatloop and apply during
persistence.

<details><summary>Implementation plan</summary>

- `codersdk/chats.go`: extend `CreatedAt`'s `variants` to include
`reasoning?`; add `CompletedAt *time.Time` with `variants:"reasoning?"`.
- `coderd/x/chatd/chatloop/chatloop.go`: extend `reasoningState` with
`startedAt`; extend `stepResult` and `PersistedStep` with parallel
`[]time.Time` reasoning slices; stamp on
`ReasoningStart`/`ReasoningEnd`; thread the slices through all
`PersistStep` call sites including the interrupt-safe path; record
partial reasoning in `flushActiveState`.
- `coderd/x/chatd/attachments.go`: walk reasoning parts in occurrence
order and apply `step.ReasoningStartedAt[i]` to `part.CreatedAt` and
`step.ReasoningCompletedAt[i]` to `part.CompletedAt`.

### Tests

- `codersdk/chats_test.go` round-trips `created_at` + `completed_at` on
reasoning parts and verifies omission when absent and partial
interrupted parts.
- `coderd/x/chatd/chatprompt/chatprompt_test.go` asserts
`PartFromContent(ReasoningContent{})` does NOT stamp timestamps.
- `coderd/x/chatd/chatloop/chatloop_test.go`
`TestRun_ReasoningTimestamps` drives a stream with two reasoning blocks
and verifies parallel slices, monotonicity, ordering, non-zero values,
and content-block ordering.
`TestRun_InterruptedReasoningFlushesTimestamps` cancels mid-reasoning
and verifies `flushActiveState` records a non-zero pair.
- `coderd/x/chatd/attachments_test.go` covers
`buildAssistantPartsForPersist` for normal interleaved reasoning,
partial (zero `completed_at`), and missing slices.

</details>

> Generated by Coder Agents.

Co-authored-by: Coder Agent <agent@coder.com>
2026-05-18 12:30:30 -04:00
Ethan e75bd3aca4 fix: preserve Anthropic replay fidelity (#25377)
Anthropic is strict about replaying the latest assistant turn once it
contains signed or redacted reasoning. We were still mutating that turn
in a few Coder-owned places: dropping empty reasoning blocks on replay,
rewriting provider-tool history during sanitization, and in the worst
case sending a prompt we already knew Anthropic would reject.

This patch keeps the latest signed assistant immutable through Coder's
replay and sanitization paths, preserves empty signed or redacted
reasoning anywhere Coder owns the ledger, and fails before the provider
call if the prompt is still unsafe.

It also bumps the existing `coder/fantasy` `coder_2_33` fork that `main`
already uses to the commit containing coder/fantasy#35. These fixes have
also been upstreamed to charmbracelet/fantasy.

Closes CODAGT-409.
2026-05-18 15:20:33 +10:00
Kyle Carberry 376fc80451 fix(coderd/x/chatd): discover workspace MCP tools mid-turn after create_workspace (#25169)
## Problem

In `coderd/x/chatd/chatd.go` `runChat`, workspace MCP discovery is gated
on `chat.WorkspaceID.Valid` at the start of each turn. New chats that
bind their workspace mid-turn (via `create_workspace` or
`start_workspace`) get an empty workspace tool list on the first step,
and the model falls back to `execute` (bash) because no workspace MCP
tools are advertised.

**Repro:** new chat → "create a workspace and use MCP tools". No
`/api/v0/mcp/tools` request hits the agent on turn 1; turn 2 in the same
chat works fine.

## Fix

- Add a `PrepareTools` callback to `chatloop.RunOptions`, analogous to
`PrepareMessages`. It is invoked once before each LLM step with the
current tool list. When it returns non-nil, the chatloop replaces
`opts.Tools`, rebuilds the per-step tool definitions, and appends new
tool names to `opts.ActiveTools` so newly injected tools are callable
immediately.
- Wire `PrepareTools` in `runChat` to trigger workspace MCP discovery
the first time the chat snapshot reports a valid `WorkspaceID`. The
previous top-of-turn discovery path is unchanged for chats that start
with a workspace.
- Extract the discovery logic into `Server.discoverWorkspaceMCPTools` so
the top-of-turn and mid-turn paths share identical behavior (cache,
agent resolution, `ListMCPTools` timeout, invalidation).

Mid-turn discovery stays disabled in plan-mode turns and Explore
subagents, matching the existing top-of-turn gate. The
`workspaceMCPDiscovered` flag prevents redundant dials after the first
successful discovery.

## Tests

- `coderd/x/chatd/chatloop/chatloop_test.go`: two new
`TestRun_PrepareTools*` cases covering injection on the next step and
active-set merging when `ActiveTools` is non-empty.
- `coderd/x/chatd/chatd_test.go`:
`TestRunChat_WorkspaceMCPDiscoveryAfterMidTurnCreateWorkspace` drives
`runChat` through a `create_workspace` tool call against a real Postgres
+ mocked agent conn and asserts the second streamed LLM request
advertises the workspace MCP tool. Verified that the test fails (and
pinpoints the missing tool) when the `PrepareTools` wiring is disabled.

## Validation

```
go test ./coderd/x/chatd/chatloop/... -count=1
go test ./coderd/x/chatd/... -count=1
make lint/emdash
```

<details>
<summary>Decision log</summary>

- Chose a per-step `PrepareTools` callback over mutating `opts.Tools` in
place because `chatloop.Run` builds the `fantasy.Tool` definitions once
at start; a hook is required to let the LLM see new tools on the next
step.
- Returned `[]fantasy.AgentTool` (not also active-tool-names) and let
the chatloop derive name merges via `mergeNewToolNames`. This avoids
leaking plan-mode gating decisions into the callback contract.
- Kept the existing top-of-turn discovery path so chats that already
have a workspace at turn start pay no extra latency.
- Skipped reusing `ReloadMessages` (history reload) since this is purely
a tool-availability concern; coupling it to a history reload would
defeat the chatloop cache prefix optimizations.

</details>

---

_This pull request was generated by Coder Agents._
2026-05-12 00:30:56 -04:00
Cian Johnston e8508b2d90 fix: recover chatd from poisoned chain anchor on retry (#25097)
When OpenAI's Responses API returns `Previous response with id ... not
found` for a chained turn, classify it as a `ChainBroken` retry, clear
`previous_response_id`, exit chain mode, reload full history, and let
`chatretry` retry. Self-heals chats whose anchor was poisoned before
#25074 stopped truncated streams from being persisted as a successful
turn with a stored response id.

The new state is exposed via the existing
`coderd_chatd_stream_retries_total` counter as a
`chain_broken="true"|"false"` label. Aggregating queries (`sum`, `rate`
over `provider`/`model`/`kind`) keep working without changes; raw-series
matchers without aggregation will now see two series per `(provider,
model, kind)` where they previously saw one. The metric is internal-only
so the blast radius should be small, but if you have dashboards that
index by exact label matchers without aggregation they will need an
extra `sum` or an explicit `chain_broken` selector.

> 🤖 This PR was created with the help of Coder Agents, and was reviewed by a human 🧑‍💻
2026-05-11 17:43:40 +01:00
Ethan 46a60e6d5d refactor: move chat error kinds into codersdk (#24955)
Moves the chat error kind taxonomy from `coderd/x/chatd/chaterror` into
`codersdk.ChatErrorKind` and types `ChatError.Kind` /
`ChatStreamRetry.Kind` so generated TypeScript exposes an SDK-owned
union, including `usage_limit`. Backend chat classification now
references the SDK constants directly while preserving the existing JSON
string values.

Keeps chat usage-limit admission failures on their existing 409 response
shape. The frontend maps structured usage-limit responses to the
SDK-owned `usage_limit` kind, uses generated `TypesGen.ChatErrorKind`
directly, and removes the local string union and alias.
2026-05-06 11:57:48 +10:00
Michael Suchacz 0bb09935bc feat: add computer-use provider selection for AI agents (#24772)
Adds a deployment-wide setting to select the computer-use provider
(Anthropic or OpenAI) for AI agents, plus the OpenAI computer-use runner
needed to honor that selection.

The setting is stored in `site_configs` under
`agents_computer_use_provider`, defaults to Anthropic when unset, and is
exposed via experimental GET/PUT endpoints under
`/api/experimental/chats/config/computer-use-provider`. The chatd
computer-use tool now dispatches to either `runAnthropicComputerUse` or
`runOpenAIComputerUse` based on the resolved provider, with
provider-specific result metadata for OpenAI screenshots.

Frontend adds a provider dropdown to the Agents Experiments settings
page nested under the virtual desktop toggle, with disabled state
handling while virtual desktop is off and skeleton loaders while config
queries are in flight.

Hugo and Codex review follow-up:
- Uses shared provider validation and clearer computer-use constant
names.
- Removes stale OpenAI pending-safety-checks commentary.
- Documents why provider result metadata is needed for OpenAI
screenshots.
- Keeps the computer-use subagent visible when provider credentials are
missing, then returns a clear spawn-time configuration error.
- Uses OpenAI's recommended 1600x900 screenshot geometry to preserve the
native 16:9 aspect ratio.
- Moves OpenAI-specific computer-use helpers into
`coderd/x/chatd/chatopenai/computeruse` after rebasing onto the provider
package refactor in `main`.
- Converts OpenAI pixel scroll deltas to Coder desktop wheel-click
amounts.
- Preserves OpenAI pointer modifiers with key down/up desktop actions
and rejects unsupported non-left double-click buttons explicitly.
- Maps OpenAI back/forward side-button clicks to browser navigation key
actions.
- Defaults omitted OpenAI click buttons to left-click.
- Retries mouse release cleanup if the final OpenAI drag release fails.
- Keeps computer-use subagent availability messages stable when provider
config cannot be loaded, while logging the backend error.
- Releases remaining OpenAI modifier keys if a synthetic key-up cleanup
action fails.
- Updates Storybook interaction stories so provider snapshots show the
selected final provider.

> Mux updated this PR description on behalf of Mike.
2026-05-04 20:30:50 +02:00
Michael Suchacz 203b0a9df8 refactor(coderd/x/chatd): extract OpenAI logic into chatopenai package (#24788)
Extracts OpenAI-specific logic from `coderd/x/chatd` into
`coderd/x/chatd/chatopenai` so the main chat path no longer references
`fantasyopenai` directly for chain mode info, response IDs, web search
tooling, or option mapping.

Structural refactor. The only deliberate behavioral narrowing is
consolidating Responses store checks and related keyed option or
metadata access on `opts[fantasyopenai.Name]`. That is documented by
`TestIsResponsesStoreEnabledIgnoresMalformedNonOpenAIKey` and is
unreachable in production where Responses options always live under
`fantasyopenai.Name`.

Summary:

- Moves OpenAI Responses chain mode info, response ID helpers, web
search tool construction, and provider option conversion into
`chatopenai`.
- Keeps Anthropic, Google, OpenRouter, and Vercel provider branches as
thin, existing code paths.
- `chatopenai` only imports `chatprompt` from chatd subpackages. It does
not import `chatd`, `chatloop`, `chatprovider`, or `chaterror`.
- Follow-up review fixes align helper names, keyed provider option
access, map cloning behavior, and PR documentation with the extracted
package boundary.
- Final sweep trims unused chain-mode state, removes a duplicate
store-check test case, drops an unused provider-tool parameter, and
shares the chat-message test helper through `chattest`.

> Mux is updating this PR on Mike's behalf.
2026-05-04 11:17:19 +02:00
Thomas Kosiewski a84534315c feat(coderd/x/chatd/chatloop): add exclusive tool execution policy (#24619)
## Summary

Introduce an **exclusive-tool execution policy** in `chatloop.Run()` so that a designated "planning-only" tool can never execute in the same batch as any other locally executed tool. This is the generic foundation for the advisor tool landed in the stacked PRs above.

## Motivation

The advisor tool (stacked on top) is intentionally a pre-action planning step. It must be consulted *alone*, not interleaved with side-effecting tools, so the model has to reason strategically before committing to actions. Rather than building advisor-specific plumbing into `chatloop`, this PR adds a small, general-purpose policy hook that any future tool can opt into.

## Changes

- `RunOptions.ExclusiveToolNames map[string]bool` declares which tool names must run exclusively within a single tool-execution batch.
- `executeTools()` detects mixed batches: if any exclusive tool name appears next to any other locally executed tool, no tool runs. Instead, structured `ToolResultOutputContentError` entries are synthesized for every tool in the batch so the model can cleanly retry.
- Deterministic, model-facing error copy:
  - Exclusive tool gets: "must be called by itself before action tools".
  - Sibling tools get: "skipped because `<exclusive>` must run alone".
- New tests in `chatloop_test.go` cover: single exclusive batch runs normally, mixed batch returns policy errors and executes nothing, non-exclusive batches are untouched.

## Stack context

This is **PR 1 of 6** in the advisor feature stack.

```
main
 └─ feat/advisor-01-chatloop-exclusive-policy   ← this PR
     └─ feat/advisor-02-chatadvisor-pkg
         └─ feat/advisor-03-config-api
             ├─ feat/advisor-04-chatd-runtime
             │   └─ feat/advisor-05-chat-tool-renderer
             └─ feat/advisor-06-admin-settings-ui
```

## Scope / non-goals

- No advisor-specific code lives in this PR.
- Zero behavior change when `ExclusiveToolNames` is empty (current default for every existing caller).

## Validation

- `go test ./coderd/x/chatd/chatloop/... -run TestExclusive`
- `make lint`

---

<details>
<summary>📋 Implementation Plan (shared across the advisor stack)</summary>

# Plan: Add a Mux-style advisor tool to coder agents/chatd

## Outcome

Add a first-class `advisor` tool to agent chats in `coderd/x/chatd` that feels native to Coder:

- it is a built-in server-side tool, not an MCP/dynamic-tool workaround;
- it performs a nested **tool-less** model call for strategic advice;
- it is exposed only when eligible, and the prompt mentions it only when it is actually available;
- it is treated as a **planning-only** tool so it does not run alongside action tools in the same batch;
- it tracks usage/cost separately enough for operators to reason about it;
- it has a minimally polished UI in the Agents page;
- and it ships with explicit dogfooding evidence, including screenshots and repro videos.

## Design decisions to lock before coding

1. **Primary architecture:** native built-in tool in `chattool/`, backed by a small `chatadvisor` package.
2. **Nested model execution:** reuse chatd's existing model/provider stack for a one-step, tool-less advisor call rather than inventing a new provider pathway.
3. **Execution policy:** treat `advisor` as an exclusive/planning-only tool; mixed batches must return structured policy errors and force the model to retry cleanly.
4. **Availability:** initial rollout is for root agent chats only; disable for child/sub-agent chats until recursion/cost policy is proven.
5. **Prompt sync:** use one eligibility boolean to drive both tool registration and advisor guidance injection.
6. **Persistence/cost split:** MVP should keep advisor usage visible in result metadata and server metrics; only add DB schema if product/billing explicitly needs queryable advisor-specific cost.
7. **UI scope:** generic tool rendering is an acceptable temporary milestone during backend bring-up, but the release candidate should include a dedicated lightweight advisor renderer.

## Delivery model

The work should be executed as coordinated workstreams with one integration owner and parallel contributors for low-conflict areas. The integration owner should own `coderd/x/chatd/chatd.go` because prompt assembly, tool registration, and model resolution all converge there.

## Detailed workstreams

### Repo evidence used for this plan

<details>
<summary>Mux reference and current chatd seams</summary>

**Mux reference implementation**

- `src/node/services/tools/advisor.ts` — native advisor tool implementation.
- `src/common/constants/advisor.ts` — advisor prompt/constants and truncation policy.
- `src/common/utils/tools/tools.ts` — conditional tool registration.
- `src/node/services/streamContextBuilder.ts` — injects advisor guidance only when the tool is available.

**Current chatd seams**

- `coderd/x/chatd/chatd.go`
  - `processChat()` — tool assembly, prompt assembly, and chatloop invocation.
  - `resolveChatModel()` — current model/provider/key resolution seam.
  - `type Config struct` — server-level chatd configuration surface.
- `coderd/x/chatd/chatloop/chatloop.go`
  - `Run()` — main streaming/model loop.
  - `executeTools()` — built-in tool execution/batching seam.
- `coderd/x/chatd/chattool/` — built-in tool implementations.
- `site/src/pages/AgentsPage/components/ChatElements/tools/Tool.tsx` — tool renderer dispatch.
- `site/src/pages/AgentsPage/components/ChatConversation/messageParsing.ts` and `ConversationTimeline.tsx` — tool/result merge and rendering flow.

</details>

### Workstream map and ownership

| Workstream | Primary owner | Main files | Can run in parallel? | Done when |
|---|---|---|---|---|
| 0. Integration + gating | Integration lead | `coderd/x/chatd/chatd.go` | No; central merge lane | Tool registration, prompt sync, and model selection are wired together |
| 1. Advisor runtime + tool | Backend agent | new `coderd/x/chatd/chatadvisor/`, new `coderd/x/chatd/chattool/advisor.go` | Yes | Tool can perform a tool-less advisor call in memory and return structured results |
| 2. Planning-only execution policy | Chatloop agent | `coderd/x/chatd/chatloop/chatloop.go`, related tests | Yes | Mixed `advisor` + action-tool batches are rejected cleanly and deterministically |
| 3. Metrics/usage/config | Backend/telemetry agent | `chatd.go`, `chatloop/metrics.go`, optional config plumbing | Partially; coordinate with integration lead | Advisor usage is separately visible in metadata/metrics and limits are enforced |
| 4. Frontend rendering | Frontend agent | `site/.../tools/Tool.tsx`, new `AdvisorTool.tsx`, stories | Yes after result schema stabilizes | Advisor renders as a readable card and story tests pass |
| 5. Dogfood + QA evidence | QA agent | dev server, Storybook, dogfood output | After backend + UI are usable | Repro videos, screenshots, and a concise QA report exist |

### Parallelization rules

- **Do not split `coderd/x/chatd/chatd.go` across multiple execution agents without an integration lead.** That file owns prompt building, tool registration, model resolution, and cost persistence.
- Workstreams 1 and 2 can be developed in parallel and then stacked onto the integration branch.
- Workstream 4 should begin once the backend result schema is agreed on, even if the backend is still behind a feature flag.
- Any agent that needs to re-check Mux behavior should clone `coder/mux` into a temporary directory (for example, `$(mktemp -d)/mux`) and inspect it read-only; do not vendor or copy code from Mux directly.

## Phase 0 — Preflight and guardrails

### Goals

- Align the team on the smallest shippable architecture.
- Prevent scope creep into MCP/dynamic-tool/sub-agent variants.
- Decide upfront what is MVP vs. follow-up.

### Tasks

1. **Confirm the MVP boundary.**
   - Ship a built-in advisor tool first.
   - Do **not** make MCP, dynamic tools, or sub-agents the primary implementation.
   - Do **not** add transient streaming phases in the first backend PR unless they fall out almost for free.

2. **Confirm local workflow hygiene before coding.**
   - Ensure the repo is using the project git hooks from `scripts/githooks`.
   - Do not bypass hooks with `--no-verify`.
   - Use `./scripts/develop.sh` for the full dev server rather than manual build/run commands.

3. **Lock the model-selection policy.**
   - **Recommended MVP:** advisor uses the same resolved provider/model/cost config as the current chat, with advisor-specific max-output and usage caps.
   - **Follow-up only if required:** add a separate `AdvisorModelConfigID`-style override that resolves through the existing `configCache`/model-config path. Do not invent a new free-form `provider:model` parser if chatd already stores provider/model separately.

4. **Lock the persistence policy.**
   - **Recommended MVP:** no DB migration. Persist advisor-visible metadata in the tool result and record separate metrics in memory/Prometheus.
   - **Only if product/billing explicitly asks for queryable advisor cost:** add a later DB migration or usage table, following the normal `queries/*.sql` + `make gen` workflow.

5. **Create an execution ADR note in the work item or tracking doc.**
   - Capture: built-in tool, tool-less nested call, root-chat-only rollout, exclusive execution policy, MVP no-DB-migration default.

### Quality gate

- Everyone on the team can state the same answers to these questions:
  - Is advisor a built-in tool? **Yes.**
  - Can advisor run with action tools in the same batch? **No.**
  - Does advisor get tools of its own? **No.**
  - Is a DB migration required for MVP? **No, unless billing insists.**

## Phase 1 — Build the advisor runtime and tool wrapper

### Goals

Create the core advisor implementation in a way that is easy to test and keeps `chattool/` thin.

### Files to add

- `coderd/x/chatd/chatadvisor/types.go`
- `coderd/x/chatd/chatadvisor/guidance.go`
- `coderd/x/chatd/chatadvisor/handoff.go`
- `coderd/x/chatd/chatadvisor/runtime.go`
- `coderd/x/chatd/chatadvisor/runner.go`
- `coderd/x/chatd/chattool/advisor.go`

### Responsibilities by file

1. **`types.go`**
   - Define the input/result schema used by the tool and UI.
   - Keep the result shape close to Mux so the UI and model both have predictable cases.
   - Recommended result variants:
     - `advice`
     - `limit_reached`
     - `error`

   Recommended shape:

   ```go
   type AdvisorArgs struct {
       Question string `json:"question"`
   }

   type AdvisorResult struct {
       Type          string              `json:"type"`
       Advice        string              `json:"advice,omitempty"`
       Error         string              `json:"error,omitempty"`
       AdvisorModel  string              `json:"advisor_model,omitempty"`
       RemainingUses int                 `json:"remaining_uses,omitempty"`
       Usage         *AdvisorUsageResult `json:"usage,omitempty"`
   }
   ```

2. **`guidance.go`**
   - Hold two strings:
     - the nested advisor system prompt;
     - the parent-agent guidance block to inject into the outer system prompt.
   - The nested advisor prompt must say, in plain language:
     - you are advising the parent agent;
     - you do not address the end user directly;
     - you do not claim actions happened;
     - you return concise strategic guidance and tradeoffs.

3. **`runtime.go`**
   - Define the per-run runtime state.
   - Recommended fields:
     - resolved model + model config;
     - provider keys/options reused from the outer chat;
     - `MaxUsesPerRun`;
     - `MaxOutputTokens`;
     - atomic/current call counter;
     - callback(s) to obtain the current prompt snapshot and current-step snapshot;
     - optional metrics/usage hook.
   - Add fail-fast validation for impossible config: nil model, non-positive limits, empty prompt builders, etc.

4. **`handoff.go`**
   - Build the advisor handoff message from:
     - the explicit question;
     - the exact prompt/messages the parent model just used;
     - the current step's text/reasoning snapshot, if available;
     - the most recent relevant tool outputs, if they are already in the prompt snapshot.
   - **Important:** use the already-prepared outer prompt tail, not a fresh DB reload. That keeps the advisor aligned with compaction and the exact context the outer model saw.
   - Apply hard truncation budgets with recent-context bias.

5. **`runner.go`**
   - Execute the nested advisor call.
   - **Recommended implementation:** call `chatloop.Run()` in an in-memory, one-step mode:
     - `Tools: nil`
     - `ProviderTools: nil`
     - `MaxSteps: 1`
     - `PersistStep`: capture the assistant output in memory instead of writing DB rows
   - Reuse the existing provider/model/cost path instead of building a second provider runner.
   - Assert that no tool definitions are passed to the nested call.

6. **`chattool/advisor.go`**
   - Keep this file thin and consistent with other built-ins.
   - Responsibilities:
     - decode `AdvisorArgs`;
     - validate `Question` is non-empty and bounded;
     - call the `chatadvisor` runner;
     - return a structured tool response.

### Defensive programming requirements

- Assert `Question` is non-empty after trimming.
- Assert runtime limits are positive.
- Assert the nested advisor call runs with zero tools/provider tools.
- Assert `AdvisorResult.Type` is one of the known variants before returning.
- Assert remaining uses never goes negative.

### Acceptance criteria

- A unit test can call the advisor tool with a fake model and receive a stable `advice` result.
- The nested advisor call is impossible to run with tools accidentally attached.
- The core logic lives in `chatadvisor/`, not embedded inside `chatd.go`.

## Phase 2 — Wire advisor into chatd and keep prompt/tool availability in sync

### Goals

Register the tool in the right place, expose it only when eligible, and inject system guidance only when the tool is present.

### Files to modify

- `coderd/x/chatd/chatd.go`
- optionally a small helper file if `chatd.go` becomes too crowded

### Tasks

1. **Compute one eligibility boolean in `processChat()`.**
   Recommended inputs:
   - server-level advisor enabled flag;
   - root chat only (`chat.ParentChatID == uuid.Nil` or equivalent existing root/child check);
   - a usable resolved model/provider exists;
   - optional experiment/workspace/org gate if product wants staged rollout.

2. **Create the runtime once per outer chat run.**
   - Use the model/config/keys resolved by `resolveChatModel()`.
   - Reuse provider options from the current chat's `ChatModelCallConfig`.
   - Set `MaxUsesPerRun` and `MaxOutputTokens` from advisor config defaults.

3. **Register the tool in the built-in tool block.**
   - Insert after the skill tools and before MCP tools in `processChat()`.
   - Record `builtinToolNames["advisor"] = true` so metrics stay bounded.

4. **Inject advisor guidance into the outer system prompt using the same boolean.**
   - Use `chatprompt.InsertSystem()` in the same prompt assembly path that already injects user/system instructions.
   - Place the block near the existing instruction insertion, before plan-path/skill context blocks.
   - Wrap the guidance in an explicit tag like `<advisor-guidance>` so it is easy to spot in tests and future refactors.

5. **Keep advisor out of child chats for the first release.**
   - That avoids recursion/cost blowups with `spawn_agent` / `wait_agent` flows.
   - Document this explicitly in the rollout notes and tests.

### Acceptance criteria

- If advisor is disabled, neither the tool nor the prompt guidance appears.
- If advisor is enabled, both the tool and the prompt guidance appear.
- Root chats can use advisor; child chats cannot.
- Built-in tool names include `advisor` so metrics do not collapse it into the generic `mcp` label.

## Phase 3 — Enforce planning-only execution policy in `chatloop`

### Goals

Prevent the model from calling `advisor` and action tools in the same execution batch.

### Files to modify

- `coderd/x/chatd/chatloop/chatloop.go`
- related chatloop tests

### Recommended implementation

Keep the MVP small; do **not** build a general policy engine yet.

1. Add a minimal field to `chatloop.RunOptions`, for example:

   ```go
   ExclusiveToolName *string
   ```

2. In `Run()` / `executeTools()`, detect the case where the exclusive tool appears in the same local-tool batch as any other locally executed tool.

3. When that happens, synthesize structured tool-result errors for the affected calls instead of executing anything in the batch.
   - `advisor` should receive a clear error like: _advisor must be called by itself before action tools_.
   - The sibling action tools should receive a paired policy error like: _this tool was skipped because advisor must run alone_.

4. Let the outer model see those tool errors and retry cleanly.
   - This is simpler and safer than partial execution or hidden deferral.
   - It preserves deterministic transcript history for debugging.

5. Pass the just-finished step snapshot into the tool execution context.
   - The advisor runtime should be able to see the current step's text/reasoning content, because that is often the best hint about what the outer model is trying to decide.

### Why this is the right fit

- It matches the intended semantics: advisor is consulted **before** taking action.
- It avoids subtle race conditions caused by concurrent built-in tool execution.
- It keeps the behavior easy to test with fake models.

### Acceptance criteria

- A model-emitted batch containing only `advisor` succeeds.
- A model-emitted batch containing `advisor` plus any other locally executed tool returns deterministic policy errors and executes nothing.
- Non-advisor tool execution stays unchanged for normal chats.

## Phase 4 — Usage limits, metrics, and configuration

### Goals

Make advisor safe to operate without over-designing billing/storage in the first release.

### Files to modify

- `coderd/x/chatd/chatd.go`
- `coderd/x/chatd/chatloop/metrics.go` as needed
- `coderd/x/chatd/chatd.go` `Config` struct and constructor path
- optional follow-up config/db files only if a separate advisor model or persistent billing is required

### Tasks

1. **Add explicit server config knobs for MVP.**
   Recommended fields on `chatd.Config` or a nested advisor config struct:
   - `AdvisorEnabled bool`
   - `AdvisorMaxUsesPerRun int`
   - `AdvisorMaxOutputTokens int64`

2. **Track usage per outer run.**
   - Reset the counter for each `processChat()` invocation.
   - Return `remaining_uses` in the tool result.
   - Return `limit_reached` when the cap is exhausted.

3. **Expose advisor usage metadata in the tool result.**
   - Include model name and token/cost summary if available.
   - Use the same `callConfig.Cost` calculation path as the outer chat for MVP if advisor reuses the same model.

4. **Record server-side metrics.**
   - Count advisor invocations, failures, and latency.
   - Ensure they show up under the built-in tool label `advisor`.

5. **Optional decision gate: separate advisor model.**
   - If product insists on a stronger/different advisor model, add a follow-up config hook that resolves another existing chat model config through the same `configCache` path.
   - Keep that out of the first landing PR unless it is required for acceptance.

6. **Optional decision gate: queryable advisor cost.**
   - If this becomes required, spin a follow-up DB task:
     - update `coderd/database/queries/*.sql`;
     - add migration files;
     - run `make gen`;
     - update audit mappings if a new auditable type/field is introduced.

### Acceptance criteria

- Advisor calls are capped per outer run.
- Limit exhaustion is user-visible in the tool result.
- Metrics distinguish advisor calls from other built-in tools.
- MVP does not require a schema migration unless explicitly approved.

## Phase 5 — Frontend rendering and Storybook coverage

### Goals

Make advisor feel intentional in the Agents UI without blocking the backend on fancy streaming UI.

### Files to modify

- `site/src/pages/AgentsPage/components/ChatElements/tools/Tool.tsx`
- new `site/src/pages/AgentsPage/components/ChatElements/tools/AdvisorTool.tsx`
- Storybook story file(s) in the same tools directory

### Delivery strategy

1. **Intermediate milestone during backend bring-up:** rely on the existing generic tool renderer if needed.
   - This is acceptable only as a short-lived integration checkpoint.

2. **Release milestone:** add a dedicated lightweight `AdvisorTool` renderer.
   - Reuse existing primitives:
     - `ToolCollapsible`
     - `ToolIcon`
     - `Response` for markdown/prose rendering
     - `ScrollArea` if the advice can be long
   - Keep styling light and consistent with the Agents page.
   - Do not add unnecessary React memoization in `site/src/pages/AgentsPage/`; that area is already React-Compiler aware.

3. **Render the structured result states cleanly.**
   - `advice` — readable prose/markdown with optional metadata footer.
   - `limit_reached` — warning-style message.
   - `error` — error state with visible fallback text.
   - `running` — existing tool loading state/spinner is enough for MVP.

4. **Add Storybook coverage instead of ad-hoc component tests.**
   Recommended stories:
   - successful advice;
   - running/loading;
   - limit reached;
   - error.

5. **Keep the UI contract narrow.**
   - Prefer one text field like `advice` plus small metadata rather than a deeply nested schema.
   - That keeps the UI resilient to prompt iteration.

### Acceptance criteria

- The advisor tool card renders readable content rather than raw quoted JSON in the final release branch.
- Running, limit, and error states are visibly distinct.
- Storybook stories and play assertions cover the new states.
- Existing tool rendering flows remain unchanged.

## Phase 6 — Automated tests and validation gates

### Backend tests to add

1. **Advisor runtime/tool tests**
   - question validation;
   - tool-less nested execution assertion;
   - success result shaping;
   - limit-reached result shaping;
   - error result shaping.

2. **Prompt/gating tests in chatd**
   - advisor disabled ⇒ no tool, no guidance;
   - advisor enabled/root chat ⇒ tool + guidance;
   - child chat ⇒ advisor absent.

3. **Chatloop policy tests**
   - advisor alone runs;
   - advisor + action tool mixed batch returns deterministic policy errors;
   - non-advisor tools still execute normally.

4. **Usage/metrics tests**
   - per-run cap resets correctly;
   - builtin tool labeling includes `advisor`;
   - returned metadata includes model/usage summary when available.

### Frontend tests to add

- Storybook `play()` assertions for the advisor renderer states.
- Verify expand/collapse behavior and visible fallback text.
- Verify the message timeline still renders adjacent tools correctly.

### Recommended command sequence

Run these as the implementation matures, not only at the end:

1. Backend-focused gate after phases 1–4:
   - `make test RUN=TestAdvisor`
   - `make test RUN=TestChatloopAdvisor`
   - `make lint`

2. Frontend-focused gate after phase 5:
   - `pnpm test:storybook src/pages/AgentsPage/components/ChatElements/tools/AdvisorTool.stories.tsx`
   - `pnpm lint`
   - `pnpm format`

3. Final repo gate before handoff:
   - `make pre-commit`
   - run any additional targeted `make test RUN=...` selections covering touched chatd paths

> Use the exact new test names the implementing agents create; the names above are recommended anchors, not existing tests.

## Dogfooding plan

### Principle

Dogfood the change as a real agent feature, not just a unit-tested backend. Per the dogfood and `agent-browser` skills, the reviewer should get **watchable repro videos** plus screenshots that make the behavior obvious without reading logs.

### Required setup

1. Start the full dev environment with:
   - `./scripts/develop.sh`
2. If the frontend renderer changes, also start Storybook from `site/` with:
   - `pnpm storybook --no-open`
3. Use `agent-browser` directly — **never `npx agent-browser`**.
4. Use named browser sessions and an output folder such as:
   - `./dogfood-output/advisor/`
   - with subfolders `screenshots/` and `videos/`

### Evidence protocol

For every interactive scenario below:

1. Start video recording **before** the action.
2. Capture step-by-step screenshots at human pace.
3. Capture one annotated screenshot of the final state.
4. Stop the recording.
5. Note the exact pass/fail observation in the QA report.

For static UI states (for example Storybook error/limit cards), an annotated screenshot is sufficient; video is optional but still encouraged by this project’s review preference.

### Dogfood scenarios

#### Scenario A — Happy path in the real Agents UI

**Goal:** prove that a root agent chat can invoke advisor and produce a readable recommendation before taking further action.

Steps:

1. Open the Agents page with an advisor-enabled root chat.
2. Start a repro video.
3. Send a prompt that should reasonably trigger strategic planning, such as an architecture or multi-tradeoff question.
4. Capture screenshots of:
   - the prompt before send;
   - the running advisor state;
   - the completed advisor card and the assistant’s follow-up response.
5. Stop recording.

Pass criteria:

- advisor appears in the timeline;
- the rendered result is readable;
- the assistant can continue after consuming the advisor output.

#### Scenario B — Advisor unavailable path

**Goal:** prove the feature is truly gated.

Suggested variants (at least one is required, both are better):

- feature flag/config off;
- child/sub-agent chat.

Evidence:

- annotated screenshot of the chat/tool state showing advisor is absent;
- short video if toggling the gate live is part of the repro.

Pass criteria:

- no advisor tool is available;
- no advisor-specific prompt behavior leaks through.

#### Scenario C — UI states in Storybook

**Goal:** prove the renderer handles non-happy states cleanly.

Required story states:

- success/advice;
- running;
- limit reached;
- error.

Evidence:

- one screenshot per state;
- at least one short video showing collapse/expand behavior.

Pass criteria:

- success renders readable advice;
- limit/error have visible fallback text;
- the component behaves like the other tool cards.

#### Scenario D — Regression sweep of nearby tools

**Goal:** ensure advisor does not break the surrounding chat timeline.

Check at minimum:

- another existing built-in tool still renders correctly near advisor;
- sub-agent/tool cards still expand/collapse normally;
- no obvious console errors appear in the Agents page during the advisor flow.

Evidence:

- screenshots of adjacent tool cards;
- console/error capture if anything suspicious appears.

### `agent-browser` usage notes for the QA agent

- Prefer `agent-browser batch` for 2+ sequential commands when no intermediate parsing is needed.
- Use `snapshot -i` to discover interactive refs.
- Re-snapshot after navigation or major DOM changes.
- Avoid `wait --load networkidle` unless the page is known to go idle; prefer explicit element/text waits or short fixed waits.
- Record videos at human pace and include pauses that a reviewer can follow.

## Rollout plan

### Initial rollout

- Gate behind a server-side advisor-enabled flag.
- Enable only for selected internal/root agent chats first.
- Watch metrics for:
  - invocation count;
  - failure rate;
  - latency;
  - obvious retry loops.

### Expansion conditions

Expand beyond the initial rollout only after the following are true:

- mixed-batch policy behavior is stable;
- cost impact is understood;
- frontend UX is readable in production-like dogfood;
- no recursion surprises have appeared with sub-agent flows.

### Explicit non-goals for the first release

- advisor inside child/sub-agent chats;
- provider-agnostic streaming phase UI;
- MCP-based external advisor implementation;
- mandatory DB-backed advisor cost reporting.

## Final acceptance checklist

- [ ] `advisor` is a built-in chatd tool, not an MCP/dynamic-tool substitute.
- [ ] The nested advisor call is tool-less and bounded to one in-memory step.
- [ ] One eligibility boolean controls both tool registration and prompt guidance injection.
- [ ] Root chats can use advisor; child chats cannot in the initial rollout.
- [ ] Mixed advisor/action batches produce deterministic policy errors instead of partial execution.
- [ ] Per-run usage caps and limit-reached behavior work.
- [ ] Advisor usage is visible in metadata/metrics without forcing a DB migration for MVP.
- [ ] The Agents UI has a readable advisor card and Storybook coverage.
- [ ] Dogfooding produced screenshots and repro videos for the required scenarios.
- [ ] Validation commands (`make lint`, targeted `make test`, Storybook tests, `make pre-commit`) passed before handoff.

## Suggested PR split

1. **PR 1 — Backend foundation**
   - `chatadvisor/` package
   - `chattool/advisor.go`
   - `chatloop` exclusive policy
   - chatd gating/prompt sync
   - backend tests

2. **PR 2 — Frontend + QA**
   - advisor renderer
   - stories/play assertions
   - dogfood artifacts and QA notes

3. **PR 3 — Optional follow-ups only if demanded by stakeholders**
   - separate advisor model override
   - persistent advisor billing/queryability
   - transient phase-stream UX


</details>

---
_Generated with [`mux`](https://github.com/coder/mux) • Model: `anthropic:claude-opus-4-7` • Thinking: `max`_
2026-04-30 14:08:59 +02:00
Michael Suchacz 99eb46dac1 fix(coderd/x/chatd): repair Anthropic provider tool history (#24744)
## Problem

Anthropic returns HTTP 400 when an assistant message contains a
`web_search_tool_result` block whose `tool_use_id` has no matching
earlier `server_tool_use` block in the same assistant message. A
previous fix (#24706) sanitized provider-executed tool calls without
matching results, but the opposite direction, orphaned or misordered
provider-executed results, could still slip through both the prompt
sanitizer and the persistence path.

## Fix

Tighten Anthropic provider-executed tool history handling while
preserving the useful result payload as normal assistant text when the
provider-tool metadata is unsafe.

1. Extract Anthropic provider-tool sanitization into
`coderd/x/chatd/chatsanitize` so provider-specific repair logic is no
longer spread through `chatprompt` and `chatloop`.

2. `chatsanitize.SanitizeAnthropicProviderToolHistory` removes invalid
provider-executed tool structure for Anthropic prompts: orphans in
either direction, result-before-call, duplicate IDs, invalid JSON
inputs, empty IDs and tool names, unsupported tool names, mismatched
`ProviderExecuted` flags, provider-executed blocks outside assistant
messages, and web-search results without serializable Anthropic result
metadata. Provider-executed result payloads are textified instead of
being discarded when there is text to preserve.

3. `chatsanitize.SanitizeAnthropicProviderToolContent` mirrors the same
rule at the streamed step content level. Persisted history no longer
carries invalid provider-tool blocks forward, but it keeps the result
text for future turns.

4. `chatsanitize.ApplyAnthropicProviderToolGuard` only repairs
structurally invalid Anthropic provider-tool history. It no longer
strips otherwise-valid historical `web_search` blocks just because web
search is disabled for the current request. The fail-closed fallback
also textifies provider results before removing provider-tool metadata.

Tests cover prompt sanitization, validation reason strings, result
payload textification, content-level persistence sanitization, disabled
web-search history preservation, direct pre-request guard behavior, and
the fallback strip path.

> Mux is acting on Mike's behalf.
2026-04-28 12:45:23 +02:00
Michael Suchacz 0211448d09 fix(coderd): sanitize Anthropic provider tool history (#24706)
Anthropic can reject replayed chat histories when a provider-executed
tool call, such as `web_search`, is present without its matching
provider result block.

This sanitizes unpaired Anthropic provider-executed tool calls during
prompt reconstruction, before Anthropic requests, and before persistence
so existing poisoned histories can continue and new malformed turns are
not stored.

Resolves: CODAGT-259

> Mux is acting on Mike's behalf.
2026-04-24 23:57:30 +02:00
Cian Johnston a02339c66a fix(coderd/x/chatd): prevent invalid tool results from poisoning chat history (#24663)
- **computeruse.go**: Decode base64 screenshot data before storing in
`ToolResponse.Data` (was casting base64 string to bytes without
decoding)
- **chatloop.go**: Re-encode `ToolResponse.Data` to base64 via
`base64.StdEncoding.EncodeToString` instead of `string()` cast
- **mcpclient.go**: UTF-8 validate all text from MCP responses in
`convertCallResult()` using `strings.ToValidUTF8`
- **chatprompt.go (persist)**: Defense-in-depth UTF-8 sanitization of
text and media Text fields before database storage
- **chatprompt.go (replay)**: Antivenom layer that validates base64 and
UTF-8 at read time, auto-healing already-poisoned chats without
requiring a migration
- `TestToolResultAntivenom`: 4 subtests covering poisoned text, poisoned
media, valid media round-trip, and media with invalid UTF-8 text
-  Adds `TestConvertCallResult_UTF8Sanitization`: 4 subtests covering invalid
UTF-8 in TextContent, EmbeddedResource, valid passthrough, and
multi-part
- Adds `TestComputerUseTool_Run_ScreenshotDataIsDecodedBinary`: Verifies no
double-encode in the computer-use path
- Updated existing computer-use tests for the new decoded-binary
contract

> 🤖
2026-04-23 19:58:38 +01:00
Cian Johnston 72e3ae9c5f feat: add chatd tool call error metrics and logging (#24559)
- Add `coderd_chatd_tool_errors_total` prometheus counter (labels:
provider, model, tool_name)
- Log tool call errors at warn level with correlation fields: chat_id,
owner_id, organization_id, workspace_id, agent_id, parent_chat_id,
trigger_message_id, tool_name, tool_call_id, provider, model
- Thread enriched logger from chatd.go into chatloop via
`RunOptions.Logger`
- Remove squashing of all MCP tool calls to the `mcp` bucket

> 🤖
2026-04-22 16:19:56 +00:00
Ethan cc4e04afde feat(site): display file attachments in chat UI (#24281)
Renders the durable file attachments introduced in #24280 in the chat
interface. Without this, attachments were stored and served correctly
but the UI showed raw file parts with no previews or download UX.

Every attachment gets a download affordance, split into three rendering
tiers:

- **Images** — thumbnail with a hover/focus overlay containing a
download link. `onFocusCapture`/`onBlurCapture` with
`contains(relatedTarget)` keeps the overlay open while tabbing between
the image and its download link.
- **Text-like files** (`text/*`, `application/json`) — expandable
preview button with loading + error-with-retry states and the same
download overlay. Preview fetches throw a typed
`FetchTextAttachmentError` with a `.status` field instead of a
stringly-typed error.
- **Everything else** — compact `FileCard` with extension badge,
filename, and download link.

User-side and assistant-side rendering now share `AttachmentBlocks.tsx`
(`AttachmentPreviewFrame`, `TextAttachmentButton`,
`ImageAttachmentButton`, `FileCard`, plus
`getAttachmentHref`/`getAttachmentName`) instead of two near-duplicate
implementations. The text-attachment overlay anchors to the preview
surface so the download button stays pinned even when a loading/error
status line widens the row below.

`ComputerRenderer` detects when a screenshot was stored as a durable
attachment (`attachment_file_id`) and suppresses the stale base64
rendering — the screenshot appears as a proper file part instead.
`ToolLabel` shows the attached filename for `attach_file` tool calls.

Storybook coverage in `ConversationTimeline.stories.tsx` was expanded to
cover every tier (single/multiple images, inline + file-id text, JSON,
download-only files, fetch-failure retry, mixed attachments + file
references) with play-function assertions.

<img width="811" height="150" alt="image"
src="https://github.com/user-attachments/assets/27c71081-3502-4e80-92a7-d8adf1ff9323"
/>



## Cleanup

Per Mathias' post-merge suggestion on #24280, this PR also relocates
`coderd/chatfiles` → `coderd/x/chatfiles` so the durable-attachment
helpers live beside the rest of the `chatd` experimental surface.

Closes CODAGT-91
2026-04-22 20:11:53 +10:00
Thomas Kosiewski df7e838c21 feat(coderd): wire debug logging into chat lifecycle (#23917) 2026-04-20 12:27:16 +02:00
Cian Johnston 4b585465b8 feat: label chatd metrics by model, add stream-state diagnostics (#24475)
Adds production-observability metrics to coderd/x/chatd/ for
model-level correlation and a chatStreams memory-leak investigation.

- Label per-request chatd metrics (steps_total, message_count,
  prompt_size_bytes, tool_result_size_bytes, ttft_seconds,
  compaction_total) with `model` and enrich the per-turn logger
  with provider/model.
- Add `coderd_chatd_stream_retries_total{provider, model, kind}`
  counter incremented in chatloop before OnRetry.
- Register a prometheus.Collector exposing `streams_active`,
  `stream_buffer_size_max`, `stream_buffer_events`,
  `stream_subscribers` from p.chatStreams.
- Add `coderd_chatd_stream_buffer_dropped_total` counter,
  incremented per publishToStream drop independently of the
  existing log-rate-limited bufferDropCount.
- Snapshot logger/model before the title-generation goroutine to
  avoid a data race with the logger/model rebind below it.

> 🤖
2026-04-17 16:16:30 +01:00
Michael Suchacz 1cf0354f72 feat: add plan mode with restricted tool boundary (#24236)
> This PR was authored by Mux on behalf of Mike.

## Summary
- add persistent plan mode for chats and the chat-specific plan file
flow
- add structured planning tools such as `ask_user_question` and
`propose_plan`
- keep `write_file` and `edit_files` constrained to the chat-specific
plan file during plan turns
- allow shell exploration in plan mode, including subagents, via
`execute` and `process_output`
- block implementation-oriented, provider-native, MCP, dynamic, and
computer-use tools during plan turns
- update the chat UI, tests, and docs for the new planning flow
2026-04-16 11:12:01 +02:00