Commit Graph
80 Commits
Author SHA1 Message Date
Michael Suchacz c28ae0dc9b feat: trigger compaction independently for the chat and compaction models (#28568)
> [!NOTE]
> Xum (an AI agent) authored this pull request on Mike's behalf.

## Summary

Automatic compaction now fires at whichever of two independent triggers
is reached first, and the UI shows when the organization override's
trigger applies.

- **Chat trigger**: the chat owner's personal threshold (or the chat
model's) against the chat model's window. A threshold of 100 turns off
only this trigger.
- **Organization trigger**: a fixed 80% of the organization compaction
override model's window. The override model's own compression threshold
is ignored. The other 20% is a target margin for the summary prompt and
the summary, not a guarantee, because the trigger reads the previous
step's prompt size.

On main, the chat threshold was applied to the smaller of the two
windows, and nothing in the UI showed it. Example: with a 20K chat model
at 70% and an 8K override, main compacts at 5,600 tokens; this PR
compacts at 6,400 (80% of 8K, shown as 32%).

## Changes

Backend (`coderd/x/chatd`):

- The trigger with the lower token point binds (ties go to the chat
trigger). It drives the automatic compaction gate and the
still-over-limit check, and every summary, automatic or manual, records
it.
- An override that is unset, deleted, disabled, on a disabled provider,
or without credentials for the chat owner is treated as absent: the chat
model summarizes and only the chat trigger applies.
- The summary's output cap is bounded by the summarizer's own window,
not the trigger's.
- The still-over-limit error names what can clear it: the compaction
threshold (chat trigger), an override model with a larger window
(organization trigger), or both when usage is over both points.
- The still-over-limit check compacts once more when the latest summary
was recorded under a different threshold or trigger window. The
`chat_summarized` result gains `trigger_context_limit_tokens`, and
`context_limit_tokens` now records the chat model's window (main
recorded the smaller window). On main, lowering a threshold after a
compaction ended every later turn with the error.
- The still-over-limit check skips a 0% threshold ("always compact"),
which on main ended the run with the error after every compaction.
- `ARCHITECTURE.md` documents the two triggers, the summary cap, and the
still-over-limit rules.

Frontend (`site`), through a shared `compactionTriggers` helper that
mirrors the backend rules (no new API):

- Coder Agents settings: the compaction override row warns when the
organization trigger undercuts chat models' default triggers ("the
organization override compacts chats at 80% of {model}'s {N}-token
window").
- Personal thresholds page: an Effective column shows the trigger that
applies, with a popover when the organization override binds. "Compacts
at" follows the same trigger, and a point the chat model's window cannot
reach shows as Off. The header's 100% sentence, and the row warning when
a member types 100, say an organization override may still compact chats
when one is set or its organization's overrides failed to load (the
header also while they load).
- Chat context gauge: the compact-at percent follows the binding trigger
and is annotated when the organization override binds. If the overrides
request fails, it shows the chat-only value with "(organization override
not loaded)", or "Compaction off (organization override not loaded)"
when the chat trigger is off.
- An override whose provider the user cannot use is ignored on the gauge
and the personal page, matching chatd.
- The shared `formatContextLimit` prints counts below 1,000 exactly (an
800-token compaction point showed "~1K") and 999,500 to 999,999 as
"1.0M" instead of "1000K". It also labels the model pickers; their other
values are unchanged.
- `HelpPopoverIconTrigger` lets callers set their own `aria-label`; only
the Effective column's popover does.

Known v1 limits: the Coder Agents settings alert is organization-scoped,
so it can warn about an override that some members fall back from for
lack of provider credentials. The gauge checks the viewer's provider
access, so on a shared chat its label can differ from the owner's
runtime trigger (Codex P2, declined). Compaction itself is unaffected.

## Screenshots

From remote UAT round 13 on 093076f402: an isolated test deployment
with a local fake LLM. The models are test fixtures named for their
context windows: Chat model 20K (own threshold 70%), Chat model 5K
(40%), and Chat model 8K (50%). Chat model 8K is also the organization
compaction override, so the organization trigger is 80% of 8,000 = 6,400
tokens, which is 32% of Chat model 20K's window. Chat model 8K's own 50%
applies only to chats that use it as the chat model; changing it to 10%
or 100% left every value below unchanged.

**Coder Agents settings:** the organization trigger undercuts Chat model
20K's 70% default.

![Override warning: 80% of Chat model 8K's window undercuts Chat model
20K](https://github.com/user-attachments/assets/188c92e6-faaa-4e9a-a563-567cbf98af85)

**Personal thresholds:** the organization trigger binds Chat model 20K
at 32%. Chat model 5K (40%) and Chat model 8K (50%) keep their own chat
triggers.

![Effective column: 32% from the organization trigger on Chat model
20K](https://github.com/user-attachments/assets/9949be4e-9417-4cfc-a683-f1367a1dd77a)
![Organization override popover: 80% of Chat model 8K's window, about
32% of this model's
window](https://github.com/user-attachments/assets/4bc3e002-0be1-46fd-91f0-aa27503034af)

**Personal thresholds, organization settings request failed** (the first
red line is the injected error). Rows fall back to chat-only values.

![Organization settings load failure above usable chat-only
rows](https://github.com/user-attachments/assets/dcf48546-aa47-4f82-a6eb-85f106b4bf45)

**Chat context gauge:** organization trigger binds (Chat model 20K),
chat trigger binds (Chat model 5K), overrides request failed (injected).

![Gauge: Compacts at 32% (organization
override)](https://github.com/user-attachments/assets/cd5402d9-9849-4254-a7a7-6623c4006a2a)
![Gauge: Compacts at
40%](https://github.com/user-attachments/assets/f372c9e9-72ef-41c8-950c-e77d74965f65)
![Gauge: Compacts at 70% (organization override not
loaded)](https://github.com/user-attachments/assets/ab829746-2ab5-4743-848f-dce4e806de3a)

**Still-over-limit errors from real runs:** organization trigger, chat
trigger, both.

![Organization-only over-limit
error](https://github.com/user-attachments/assets/f210807a-a2cc-47a0-807c-f1404373bb28)
![Chat-only over-limit
error](https://github.com/user-attachments/assets/eecec38e-97de-49b6-8670-b32090103bbb)
![Combined over-limit
error](https://github.com/user-attachments/assets/459a6d08-7215-492d-93f5-96a4f767a09e)

## Testing

- Go: `TestBindingCompactionTriggerSource`,
`TestCompactionStillOverLimitError`, `TestCompactionStatusFromHistory`,
and `TestActiveServer_CompactionModelOverride` cover the 80% point
(fails if the override's own threshold is read again), user threshold
100 leaving the organization trigger on, the summary cap for automatic
and manual compaction, the over-limit messages, and recompaction after a
trigger change. The site binding tests mirror the Go table row for row.
- Site: `compactionTriggers` unit tests and Storybook stories for the
three surfaces, including the overrides fetch failure.
- On c6f8750479: `go vet`, the `chatd` and `chatloop` tests, and the
AgentsPage, AISettingsPage, and HelpPopover unit tests (2,274 passed, 2
skipped) pass, and CI is green. On 987cefe399, tsc, the five affected
story files (109 passed), and `make pre-commit` also passed. 093076f402
only changes `ARCHITECTURE.md`; the pre-commit hook's markdown format
and lint checks pass on it.
- On 3cd5b08f1f: tsc, the new `formatContextLimit` test (fails on the
old helper for 800 and 999,999), the `compactionTriggers` and `aiModels`
unit tests (224 passed), and the affected story files pass, and the
pre-commit hook passed on all three commits.

## Review record

- Earlier heads: 12 Codex passes and 3 human reviews, all threads
answered. Rebased onto main several times, most recently ff3fa22f68
(kept #30274, #29892, and #29890 in conflicted files).
- coder-agents-review rounds 1 to 18: findings fixed; approved since
round 15. The round 1 `ARCHITECTURE.md` finding was left for the
maintainer, who asked for the update in the last commit.
- coder-frontend-review (eight roles): all pass. Splitting the PR (RO6)
was declined because it is reviewed as one unit; the
`organizationTriggerFor` story helper stays because its three fields
must match the model.
- Design change after UAT round 11: the organization trigger is a fixed
80% of the override window instead of the override model's own threshold
(171d5f568e, 21d2ce7fce, 987cefe399).
- coder-agents-review round 19: copy, naming, and an unreachable
zero-window gate fixed (6e8efb9054, c6f8750479). A summary-time fit
check with chat-model fallback (CRF-156) was declined: the previous-step
lag applies to every trigger, including main's. Round 20 approved
c6f8750479 with no findings.
- Codex: one P2 on 987cefe399 declined (see known limits); no major
issues on c6f8750479.
- Remote UAT on dogfood (fake LLM, isolated test deployment): rounds 8
to 13 PASS. Round 12 on c6f8750479 first failed one criterion that was
written wrong (the admin alert correctly lists an override model that is
also a chat model at 100%); the reviewer endorsed PASS on the same
evidence after the correction. Round 13 on 093076f402 retook the
screenshots with neutral model names and confirmed that the override
model's own threshold (10%, 50%, 100%) does not move the organization
trigger.
- UAT round 13 notes: the header after a failed or pending overrides
load and the sub-1K labels are fixed in 4e172ab5ff and d5888ce15dc;
coder-agents-review round 21 found the same gap in the row warning
(CRF-167, fixed in 3cd5b08f1f). Left as on main: a summary's
`usage_percent` is measured against the binding trigger's window, so it
can exceed 100 (nothing reads it), and the 100% help text shows only for
an unsaved edit.
- Codex: no major issues on d5888ce15d and 3cd5b08f1f.
coder-agents-review round 22 approved 3cd5b08f1f with no open findings.
- coder-frontend-review on those commits: all roles pass after the
fixes. Declined: moving the formatter fix to its own PR (RO5; the owner
asked for it here), adding "tokens" to sub-1K labels (TN6; "~800" reads
like "~16K"), and splitting the component (RP6).
2026-10-11 12:10:59 +02:00
Michael Suchacz bb9f31d94b fix: apply Anthropic prompt caching through OpenRouter and OpenAI-compatible gateways (#29563)
Anthropic models configured through an `openrouter` provider (and, by
the same code path, `vercel` and generic `openai-compat` providers)
never got prompt caching. Two independent gaps: chatd only emitted
`cache_control` breakpoints when the fantasy client was `anthropic`, and
even when a client sent them, the AI Gateway chat-completions
interceptor rebuilt the upstream body from typed
`openai.ChatCompletionNewParams` and silently dropped the unknown key.
Dogfood UAT on main @ 77479408b4 measured 0 cached tokens on 27/27 turns
for `anthropic/claude-haiku-4.5` via OpenRouter while the direct
Anthropic provider cached 7k to 11k tokens per turn.

## Fix

**AI Gateway** (`aibridge/intercept/chatcompletions`):
`ChatCompletionNewParamsWrapper` records every message-level and
content-part-level `cache_control` from the raw request and re-applies
it at the same JSON path when the upstream body is rebuilt, before the
Google-specific patching. The blocking path now sends the raw body
whenever preserved fields exist; the streaming path already sent the
rebuilt body. Nothing is added unless the client sent it, and at most
four markers are recorded per request (Anthropic's breakpoint limit) so
the per-marker body rewrites stay bounded.

**chatd** (`coderd/x/chatd`): the caching gate becomes a small strategy
resolver keyed on the fantasy client and the model ID. `anthropic` keeps
the existing message-level breakpoints; `openrouter` and `vercel` reuse
them for Anthropic-family IDs since both clients translate them;
`openai-compat` gets content-part-level `ContentExtraFields` markers
(last text part of the last system message plus the final two
text-bearing user or assistant messages, tool messages skipped, at most
3 markers). `chatprovider.IsAnthropicFamilyModelID` recognizes
`anthropic/...`, Bedrock-style `anthropic.`/`.anthropic.` IDs, and bare
`claude` IDs. The `claude` substring match is deliberately broad; a
non-Anthropic model whose ID contains it would receive markers that
OpenAI-compatible upstreams ignore.

Docs: a short "Prompt caching" note in `docs/ai-coder/agents/models.md`.
`coderd/x/chatd/ARCHITECTURE.md` has no caching text to update.

## Dogfood UAT

Remote Coder Agents UAT on dogfood (chat:
https://dogfood.cdr.dev/agents/9f81cc40-add0-4b6f-b76c-f348b04762e4),
OpenRouter provider with real keys:

- Baseline (main @ `77479408b4`): 0 cached tokens on every OpenRouter
Haiku turn; direct Anthropic control cached normally; a hand-added
`cache_control` on the same key cached fine.
- Fix (`1d685f7e3e`, the first three commits):
`anthropic/claude-haiku-4.5` via OpenRouter now sends 2 markers on turn
1 and 3 afterwards (system last part plus the last two text-bearing
messages, never on tool messages). OpenRouter's `/generation` ground
truth reported `native_tokens_cached` 7137 to 7255 on turns 2+, equal to
Coder's `cache_read_tokens` and the AI Gateway's
`cache_read_input_tokens` on all 43 measured rows. Turn-2 prompt cost
dropped about 8x versus baseline. Agentic S3 cache reads grew from 7156
to 13993 as a 2500-line file entered the prompt. `openai/gpt-5-mini`,
`google/gemini-2.5-flash`, `deepseek/deepseek-chat-v3.1`, and
`moonshotai/kimi-k2.5` received 0 markers with unchanged request shapes;
the direct Anthropic control was unchanged. 76 upstream attempts, 0
non-200. The remote agent reported FAIL only because one acceptance
criterion I wrote asked the chat usage indicator to display cached
tokens, which the UI has never done for any provider; the runner
endorsed PASS for the fix itsel... [truncated]
- Smoke re-tests on `20ad259999b` (marker-code simplification) and on
`e31f1806748` (docs reflow plus the four-marker cap; the later head
`92b8f8ad0b8` only renames `google_openai_compat.go` to
`request_body.go` with no content change): S1 cache reads 0 / 7139 /
7161 / 7184 / 7207 equal to OpenRouter ground truth on every row,
markers 2 then 3 and never on tool messages, tool-heavy turn all 200, a
12 KB file read grew cache reads to 13548 with every request body
inspected for marker placement, `openai/gpt-5-mini` still 0 markers.

Known follow-ups surfaced by the UAT and out of scope here:
`provider_options.openrouter.*` never reach the wire on the gateway
path; OpenRouter cache writes are folded into `input_tokens` in chat
message usage (`cache_creation_tokens` stays 0) although the gateway
records them; the AI Gateway sessions UI has no cache columns and labels
every OpenAI-compatible provider "OpenAI"; attachment-only messages (no
text) fail upstream with HTTP 400 on OpenRouter (pre-existing,
reproduced without markers).

> Xum acted on behalf of @ibetitsmike for this pull request.
2026-10-04 22:55:37 +02:00
J. Scott Miller fc0af7fbd8 feat: open a chat_turn span per prompt (#29771)
**TL;DR:** Wraps each prompt in a `chat_turn` root span with every
lifecycle stage parented under it. The span records one outcome per turn
(`completed`, `interrupted`, `error`, `abandoned`), and the new
`coderd_chatd_turn_outcomes_total` counter counts those outcomes. Turns
stay correct when tasks are canceled, replaced, or retried.

Each prompt runs inside a `chat_turn` root span owned by the runner
(`runnerTurnSpan`). The generation step, prepare, MCP connect, stream,
TTFT, provider attempt, commit, thinking, tool call, compaction, and
retry backoff stages are parented under it, activating the stages from
#29770. `chat_turn` spans are standalone roots: the turn runs on a
worker, often on another replica, and no trace context is persisted.

**Lifecycle**

- **Start.** Each generation loop iteration calls `Ensure`, which opens
the turn if none is open and returns a `turnToken`. The span is
backdated to the trigger: the latest of the last user prompt, a dynamic
tool's result message after it, and a pending compaction request. An
`acquisition` stage covers trigger to pickup.
- **Stale trigger / takeover.** A trigger at or before the previous
anchor opens the turn at now with no `acquisition` and counts
`stale_anchor`. A runner that took the chat over from an owner with a
stale heartbeat (`TakenOver`, from acquisition) opens its first turn at
now with no `acquisition`, so the dead owner's run is not reported as
pickup delay.
- **Newer prompt.** `Ensure` on the current turn with a newer trigger
closes it and opens a new one. The closed turn is `abandoned` unless the
task that ran it records an outcome before releasing it.
- **Finish.** After `FinishTurn` or `EnterRequiresAction` commits, the
step calls `Complete`; `Settle` closes the span after the step's own
stage ends, so the finishing step is inside the turn. A chat waiting in
`requires_action` is outside any turn; the generation resumed by its
tool results opens a new one.
- **Emission.** Closing a turn fixes its end time; its span is emitted
once every task that joined it (`Ensure` takes the task ID) has called
`Release`, which `runTask` does after the task returns. A finishing task
whose `FinishTurn` promoted a queued prompt therefore still counts its
turn as `completed` even when the promoted task's `Ensure` runs before
its `Complete`. `End` at runner exit emits whatever is left.
- **Failure.** A step that commits `FinishError`, or fails closed,
invalidates the turn as `error` after the commit. Errors the task runner
retries, including task timeouts, keep the same turn open; an expected,
non-retryable exit records no outcome and leaves the turn open until the
interrupt task, a newer prompt, or runner exit closes it. A canceled
generation task records no outcome and cannot obtain a turn token, so a
replaced task never touches its successor's turn.
- **Interrupt.** The interrupt task marks the open turn `interrupted`
before `FinishInterruption` commits and closes it after. Only a stop (by
the user or a parent agent) is `interrupted`; it ends with unset span
status, since error status is reserved for `error` turns. runner
shutdown, loss of ownership, and message edits close the turn as
`abandoned` with unset status.
- **Promotion.** Every promoting transition (`FinishTurn`,
`FinishInterruption`, a send to an errored chat, `PromoteQueued`)
returns the promoted message's queued time in its result, and one
`recordQueueWait` helper records a standalone `queue_wait` to the
promotion. The promoted turn opens at the promotion time.

`chat_turn` is observed on `coderd_chatd_stage_duration_seconds` only
for completed turns, so the histogram is completed-turn latency; every
turn still emits its span.

`coderd_chatd_turn_outcomes_total{outcome, chat_kind}` counts every
closed turn exactly once (`completed`, `interrupted`, `error`,
`abandoned`), matching the span's `turn_outcome`, under the
`chat-stage-metrics` experiment. Turn time accounting is in draft
#29772.

**Stages and identity**

- `tool_call` is a live stage around each local tool call in `chatloop`;
the tool runs on its context, and it ends errored only when the tool
fails to execute. Calls that never run (inactive, unknown, or rejected
by policy) record no stage.
- `mcp_connect` carries connected and failed server counts and ends
errored only when servers were configured and none connected.
- The compaction stage carries the model that runs the summary.
- `withStageIdentity` puts scope and chat kind on the turn context.
Detached background work runs in `background` scope as its own trace
roots.
- The `advisor` tool's nested model call appears only as its enclosing
`tool_call`.

ARCHITECTURE.md gains a "Lifecycle tracing" section.

<details>
<summary>Review decisions</summary>

- Turn identity stays timestamp-based with targeted fixes (clamped
acquisition, rotation on a newer trigger, invalidate after commit,
fail-closed invalidation, settle closing invalidated turns) plus a
takeover flag; keying turns on message IDs would not detect takeover and
cannot express interrupted/abandoned outcomes.
- Outcome classification: the interrupt task owns `interrupted`, and
cancellation or a fence mismatch alone never records an outcome, so
shutdown, ownership loss, and edits are `abandoned`. Only `error` turns
set error span status. Remaining ordering races between a step and an
interrupt are documented as known limitations in ARCHITECTURE.md.
- Turns are emitted after the tasks that ran them return, not when they
close, so an outcome a replaced task records while unwinding reaches its
own turn; the successor task is not delayed. Chosen over an
after-commit, pre-publish hook on `ChatMachine.Update`, which would fix
only the `FinishTurn` ordering.
- `requires_action` ends the turn rather than pausing it, so `chat_turn`
measures chatd time; a prompt with client tools counts as several turns.
- Promoted turns anchor at promotion time and `queue_wait` is always
standalone, so `chat_turn` never includes time spent queued behind
another turn.
- The former `capacity_wait` stage (#29805) was dropped: it was a
per-worker lower bound, excluded from accounting, and needed its own SQL
and bookkeeping. #29805's remaining helpers are folded into this PR.

</details>

Supersedes #28972, #28973, and #29805.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
2026-09-30 11:53:58 -05:00
J. Scott Miller e988ddb11d feat: trace chatd provider stages and count only output parts as first token (#29770)
**TL;DR:** Traces the model provider call: a `stream` stage, a nested
`time_to_first_token` stage, and one `provider_attempt` per HTTP round
trip, labelled with provider type and model. Behavior change:
`coderd_chatd_ttft_seconds` now counts only a real output part as the
first token, so error, warning, and finish parts no longer count.

Instruments the provider call.

**Stream and time to first token.** `GenerateAssistant` records a
`stream` stage from stream open to close and a nested
`time_to_first_token` stage. The TTFT window closes once, on the first
streamed output part: any part other than warnings, finish, or error,
including start markers such as `text_start`. A closed window observes
the stage and the pre-existing `coderd_chatd_ttft_seconds` with the same
elapsed time. A failed open, an error part, or a release without an
output part ends the TTFT span with the error (a silence timeout is
classified as such) and observes nothing. The `stream` span ends once,
with the error `GenerateAssistant` returns, so content-filter and
no-output attempts end errored. Failed `stream` windows are observed
like successful ones. The attempt is released before the stream span
ends, so the TTFT window always falls inside its stream. The compaction
summary stream passes a nil tracer, so it records neither.

Without #29771, production emits only `provider_attempt`; #29771 passes
the tracer to `GenerateAssistant`.

**Provider attempts.** The provider HTTP transport is wrapped in
`stageSpanRoundTripper`, which emits one `provider_attempt` stage per
round trip, so retries each get a span. The span closes on response
headers; a status >= 400 marks it errored without changing what the
client sees.

`resolveModelCall` resolves the `StageModel` (wire provider, configured
`provider_type`, model, effective reasoning effort) and threads it to
the transport, so span attributes and labels match the call.
`provider_type` is read from `ai_providers.type`, so types outside the
fantasy set (copilot) keep their configured name. `chatd.New` builds the
tracer with the server clock.

**Behavior change to `coderd_chatd_ttft_seconds`:** only an output part
counts as a first token. Error parts no longer close the window as a
success, and warnings and finish parts are skipped; the help text says
so.

`StageSpan.End` returns the span's elapsed window, which `ttft_seconds`
observes. Callers pass `Stages` and `StageModel` through
`GenerateAssistantOptions`; a nil tracer discards (TTFT still records
`ttft_seconds`). `Server` gains the `stages` tracer and a
`TracerProvider` option, wired from `coderd`.

<details>
<summary>Review decisions</summary>

- Start markers close the TTFT window: they are the first sign the model
is producing output, and waiting for a delta would fold hidden reasoning
time into TTFT.
- Failed `stream` and `provider_attempt` windows are observed: that time
was spent. TTFT is the exception because a first token that never
arrived has no duration.
- `http_host` was dropped from `provider_attempt`; every request goes
through the local AI Gateway transport, so it was constant.
- The advisor tool's nested model call is intentionally not instrumented
as `stream`/TTFT.

</details>

Supersedes #28970 and #28971.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
2026-09-30 11:53:58 -05:00
J. Scott Miller 7a9442014e feat: add chatd lifecycle stage tracer (#29769)
**TL;DR:** Adds a stage tracer that turns each chat lifecycle stage into
both an OpenTelemetry span and a Prometheus histogram observation, so
traces and metrics always agree. The metrics are only registered with
the `chat-stage-metrics` experiment. Nothing calls the tracer until the
next PRs.

Adds `chatloop.StageTracer`, which emits each chat lifecycle stage as an
OpenTelemetry span and a `coderd_chatd_stage_duration_seconds{stage,
scope, chat_kind}` observation from one `End` call, so trace and metric
durations cannot disagree. The provider-bound stages (`stream`,
`provider_attempt`) are also observed on
`coderd_chatd_model_stage_duration_seconds{stage, provider_type,
chat_kind, model}`, the only family carrying `provider_type` and
`model`, and only for turn-scoped stages. Time to first token per model
stays on the existing `coderd_chatd_ttft_seconds`. Nothing calls the
tracer yet; the next PRs instrument the provider call and the runner.

- The families are registered only when the `chat-stage-metrics`
experiment is enabled (safe and known, so
`--experiments=chat-stage-metrics` or `--experiments=*` turns it on).
Spans are emitted regardless. `NewMetrics` leaves the families off;
`chatd.New` opts in from the experiment, and a test asserts the
experiment decides whether they reach the server's registry.
- `Stage`, `Scope`, `ChatKind`, and `StageAnomaly` are typed string
constants. `scope` separates latency attributable to a prompt (`turn`)
from detached `background` work; `chat_kind` is `root` or `subagent`,
empty without a known chat.
- `StageModel` carries the wire `provider`, `provider_type`, `model`,
and `reasoning_effort`. All are span attributes; only `provider_type`
and `model` are metric labels, on the model family. No family carries an
organization label.
- Observed stages: `chat_turn`, `queue_wait`, `acquisition`,
`mcp_connect`, `stream`, `time_to_first_token`, `provider_attempt`,
`tool_call`, `commit`, `retry_backoff`. `generation_step`, `prepare`,
`thinking`, and `compaction` are span-only. A test keeps the stage sets
and help text consistent.
- API: `Start` / `StartRootAt` for live spans (span timestamps come from
the tracer clock), `Record` for stages reconstructed from timestamps
(the scope comes from the context). `End` is idempotent;
`EndWithoutObservation` ends the span without a sample. A nil tracer,
provider, or metrics discards.
- `coderd_chatd_stage_anomalies_total{reason}` counts observations
dropped, adjusted, or inconsistent (`negative_elapsed`,
`inverted_window`, `missing_timestamp`, `future_start`, `stale_anchor`).
Span-only stages never feed it.
- Buckets are 12 round edges (`0.1 0.25 0.5 1 2.5 5 10 30 60 300 1800
3600`).
- Moves the body of the `for` loop in `StartGeneration` into
`runGenerationStep` so the step has a function boundary for a deferred
stage end. Control flow and error wrapping are unchanged.

<details>
<summary>Series cost (approximate, whole stack)</summary>

A histogram label set is 12 edges + `+Inf` + `_sum` + `_count`: 15
series. Per replica, each `chat_kind` costs about 150 stage series and 4
turn outcome counters; each `(chat_kind, model)` costs 30 (2 stages x
15). Both chat kinds on `M` models: about `308 + 60*M` per replica, plus
background-scope stage series.

Dogfood `main`, 3 replicas: ~3.4k `coderd_*` series per replica (~10.1k
total).

| Models used per replica | Added series (3 replicas) | vs. baseline |
|---:|---:|---:|
| 1 | ~1.1k | +11% |
| 5 | ~1.8k | +18% |
| 10 | ~2.7k | +27% |
| 35 | ~7.2k | +72% |

Most deployments run four or fewer models, so a typical replica adds
roughly 370 to 550 series. The experimental turn accounting in #29772
would add 88 more per replica (11 categories x 4 outcomes x 2 chat
kinds). Series appear lazily and do not grow with organizations.

</details>

<details>
<summary>Review decisions</summary>

- Experiment gate only; no deployment flag or metric levels. If the
experiment graduates, the families become always on like the other chatd
families; a flag is added then only if dogfood shows a cardinality
problem.
- Organization is on neither metrics nor spans, and `effort` is
span-only; `provider_type` (configured type), not the wire `provider`,
is the model-family label.
- `time_to_first_token` is not on the model family; per-model TTFT is
`ttft_seconds`, which observes the same measured window.
- The model family records only turn-scoped stages.
- Nil guards are kept so a nil or zero tracer is always safe to call.
- HTTP attribute keys stay snake_case like the rest of the chatd span
attributes.

</details>

Supersedes #28968 and #28969.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
2026-09-30 11:53:57 -05:00
Mathias Fredriksson fa5bac6e77 feat: send tool call IDs and cancel unresolved calls on interrupt (#29954)
Retries and replica takeovers ran tool calls again, and an interrupt
recorded the same generic result whether a call had finished, was
still running, or never started.

chatd now derives a tool call ID from the chat, the committed
assistant message, and the provider tool call ID, and sends it with
execute starts, edit_files, and write_file. Every replica and retry
computes the same ID, so the agent returns the first result instead
of running the tool again, and a retried foreground execute waits
only for the rest of its timeout.

On interrupt, chatd cancels the unresolved calls of those tools on
the agent before committing, within 30 seconds, and records what
happened: "tool call was canceled before it ran; no changes were
made", the output of a command canceled while running, or the real
result. The UI shows a canceled command with the existing killed
marker. If the cancel fails, including an old agent's 404, the result
stays the generic interrupted text.

Calls whose provider tool call ID is empty or repeats in the message
get no ID and behave as before.

Fixes CODAGT-757
2026-09-30 14:31:52 +03:00
Michael Suchacz 5a997c471a fix(coderd/x/chatd): tell the model when output hit the token limit mid tool call (#30120)
When an Anthropic response hits the output token limit while the model
is writing a tool call's input, chatd gets `FinishReasonLength` with a
cut-off call. It handled both shapes of that call badly:

- If the provider closed the `tool_use` block, the call carried
truncated JSON and got the generic malformed-input result, which invites
the model to resend the same oversized input and hit the limit again.
- If the block never closed, fantasy emitted no tool call, chatloop
discarded the dangling tool input, and the turn ended with only
reasoning and no explanation.

On a length finish, `GenerateAssistant` now gives each cut-off local
call an error result: "This tool call was not executed because the
response reached the output token limit (N tokens) while writing its
input. Split the content into smaller tool calls." The token count is
included when the call has a configured output limit. A tool input that
started streaming but never became a call first gets a stand-in call
with empty input. The call persists with nil args and replays as `{}`,
so the provider receives a valid `tool_use` and `tool_result` pair, and
generation continues with another step.

With lifecycle hooks enabled, `pre_tool_use` admission skips calls the
step already resolved, so the hook is not dispatched for them and no
second result is added. The duplicate tool ID check still runs on the
full batch, because a resolved call still occupies its ID.

Independent of #30119, which stops the AI Gateway from failing these
streams before chatd sees them.

## Testing

-
`TestGenerateAssistant_SilentNoOutputFinish/LengthFinishResolvesTruncatedToolCalls`
(truncated tool call, dangling tool input): no error, reasoning kept,
and each call paired with the output-limit result.
- `TestPreToolUseHookOutputLimitTruncatedToolCallResolvedOnce` with
hooks enabled, for truncated and unfinished input: no hook dispatch, one
stored error result per call, a second model request, and a replayed
request that pairs `tool_use` input `{}` with its `tool_result`.
`DuplicateIDFailsClosed` checks that a reused ID still fails the chat
with `duplicate tool use ID`.
- Red-green: with `chatloop.go` and `generation.go` restored to the base
code, all four truncation cases fail. `DuplicateIDFailsClosed` passes on
the base code by design and fails on `d02bf4aa89b`, which dropped
resolved calls before the duplicate check.
- `go test ./coderd/x/chatd/... -count=1` passes, and lint on the
touched packages reports no issues. Local pre-push checks were skipped
per standing preference; PR CI covers them.
- Remote UAT: not run yet.

## Review record

- Parent review (Xum) of `d02bf4aa89b`: skipping resolved calls inside
`chathooks.PendingToolCalls` hid them from the duplicate tool ID check
in `admitStepToolCalls`. Fixed in `745f722165b`.
- Codex: not requested yet.

> Xum, an AI coding agent, prepared this PR on behalf of @ibetitsmike.
2026-09-30 12:20:03 +02:00
Michael Suchacz fd5b37576a feat(coderd): record provider response IDs on chat messages (#29763)
Chat messages could only be linked to AI Gateway records per chat, so
nothing recorded which model response produced a given message.

fantasy surfaces the response ID the model endpoint returns on the
stream's finish part: the Anthropic Messages ID as the part ID, and the
OpenAI Responses ID in provider metadata. chatd now stores that ID in
the existing `chat_messages.provider_response_id` column for assistant
steps and the compaction summary row. No migration and no aibridge
changes.

What the stored value is depends on the route:

| Route | Stored value | Joins to |
|---|---|---|
| Anthropic Messages API through the AI Gateway (Anthropic and Bedrock
Claude) | The gateway's interception ID. The gateway replaces the
message ID because injected MCP tool loops can fold several upstream
messages into one response. | `aibridge_interceptions.id` |
| OpenAI Responses API through the AI Gateway | The upstream `resp_` ID,
passed through unchanged | `provider_response_id` on
`aibridge_token_usages`, `aibridge_user_prompts`, and
`aibridge_tool_usages` |
| Direct to the provider | The provider's own ID (`msg_`, `resp_`) |
Nothing in the gateway tables |

To resolve messages to interceptions, then read usage by
`interception_id`:

```sql
SELECT cm.id AS chat_message_id, i.id AS interception_id
FROM chat_messages cm
JOIN aibridge_interceptions i ON i.id::text = cm.provider_response_id
UNION
SELECT cm.id, tu.interception_id
FROM chat_messages cm
JOIN aibridge_token_usages tu ON tu.provider_response_id = cm.provider_response_id;
```

Not covered, and left NULL:
- Partial rows saved on interrupt, because fantasy only reports the ID
in the finish part.
- Chat Completions providers (OpenAI-compatible, OpenRouter, Vercel),
because fantasy does not surface the chunk ID there.
- User, tool, and system rows, and all existing messages.

<details>
<summary>Delivery record</summary>

- `96a896c` replaced the earlier interception-link design (`59fa58f`,
`7f93c18`, `9d04ccd`) in place: the aibridge hook, the
`aibridge_interception_id` column and migration, the message part buffer
tracking, and the `ARCHITECTURE.md` section are all removed. The net
diff against the base touches only `coderd/database` and
`coderd/x/chatd`.
- Local validation on `96a896c`: pre-commit hook green (gen, fmt, lint,
build); `go test ./coderd/x/chatd/... ./coderd/database/` green.
Red-green: removing the finish-part capture fails both new end-to-end
tests (`TestActiveServer_PersistsAnthropicMessageID`,
`TestOpenAIResponsesPersistsProviderResponseID`). CI on `96a896c`:
green.
- Remote UAT round 2 on `96a896c`
([chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)):
**FAIL** against this description's earlier claim that every stored ID
joins the gateway usage rows directly. Anthropic rows routed through the
gateway stored the interception ID, so 0 of 18 joined usage rows
directly, while every checked Anthropic row equaled
`aibridge_interceptions.id`. Both OpenAI Responses rows joined directly.
Interrupted partial rows, Chat Completions rows, and user and tool rows
were NULL, with no empty strings or duplicate IDs. Multi-step turns,
manual compaction, edit and regenerate, a provider switch, and a retried
429 behaved as expected. One unattributed anomaly: a single duplicate
user row after interrupt and continue, which did not reproduce and is
outside this diff.
- `2e889ba` corrects the doc comment on
`PersistedStep.ProviderResponseID` and this description to match;
behavior is unchanged. Pre-commit hook and CI green.
- Remote UAT round 3 on `2e889ba` against the corrected contract ([same
chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)):
**PASS**. The query above resolved all 40 stamped rows to exactly one
interception from the same chat, with none unresolved. Reading usage by
interception recovered every usage row for 38 of them; the last two
resolved but their usage rows were not counted. Fresh Anthropic and
Responses calls, a seven-step tool turn, manual compaction, an Anthropic
to Responses to Anthropic switch in one chat, and a UI reload behaved as
described, and the NULL cases held. Three interrupt and continue
attempts did not reproduce the round-2 duplicate, so it stays
unattributed. Not tested: gateway-injected MCP tool loops (unreachable
from chatd, whose delegated requests skip the gateway's MCP proxy),
direct Anthropic `msg_` IDs, Bedrock, and threshold-triggered
compaction. Both rounds had a verified isolation proof and a clean
dogfood audit.
- Codex rounds 1 to 3 reviewed the old design. Codex round 4 on
`2e889ba`
([request](https://github.com/coder/coder/pull/29763#issuecomment-5796193371),
13:58Z): no major issues
([result](https://github.com/coder/coder/pull/29763#issuecomment-5796267231),
reviewed commit `2e889ba05c`). No unresolved review threads. Review
passes so far: 4 of 6.
- Final head `2e889ba` is the exact commit remote UAT round 3 and Codex
round 4 covered; required checks green.

</details>

> Xum updated this PR on behalf of @ibetitsmike.
2026-09-23 19:01:24 +02:00
Michael Suchacz e35cbe2a7f fix(coderd/x/chatd): stream compaction summaries (#28886)
Chats could get permanently stuck once their context crossed the
compaction threshold on adaptive-thinking Claude models: every
compaction attempt failed with `compaction produced an empty summary`,
and both automatic and manual compaction take the same path, so the chat
could never recover.

## Problem

`compactionSummaryCall` deliberately set `MaxOutputTokens = nil` because
the summary request was non-streaming and the Anthropic SDK rejects
non-streaming requests whose `max_tokens` implies a completion longer
than ten minutes. But a nil cap makes fantasy's Anthropic provider
silently default to 4096 output tokens, and adaptive-thinking Claude
models can spend that entire budget on a thinking block, leaving zero
summary text.

## Fix

- Stream the compaction summary request. Streaming lifts the Anthropic
SDK's non-streaming duration restriction, which was the only reason the
cap was nil'd.
- Keep the resolved output cap (32k default) on the summary call instead
of nil'ing it.
- When the summary comes back empty with truncation evidence (length
finish reason, or reasoning with no text), fail with `compaction summary
was truncated at the output token cap` instead of the generic
empty-summary error, so the next occurrence is attributable at a glance.

Unit tests cover text-block joining and the truncation classification;
the compaction e2e fixtures now serve streamed summaries and assert the
summary request streams with `max_tokens: 32000`. Dogfood UAT on a
deployment built from this branch verified manual compaction with Claude
Opus 5 (adaptive thinking, twice, with reload persistence) and an OpenAI
regression chat.

> 🤖 Opened by [Xum](https://mux.coder.com) on behalf of @ibetitsmike.
2026-09-21 21:28:45 +02:00
Michael SuchaczandPaul Frybarger 19e2b0415a fix(coderd/x/chatd): preserve tool definitions during compaction (#29436)
Supersedes #28883 by @pfrybar. His commit is carried with authorship
intact (rebased onto main as 32d261f94f); the second commit adds the
overflow fallback found while testing it.

## Problem

Compaction rebuilt the summary request without the conversation's tool
definitions, so the summary prompt no longer shared the turn's prefix
and providers could not reuse the cached context (measurements in
#28883).

Sending the tools with the summary makes that request at least as large
as the generation it summarizes. At a model's real context window
(gpt-3.5-turbo-0125, 16385 tokens) the summary request was rejected with
`Your input exceeds the context window of this model.` on the same
overflow compaction exists to recover from. The chat stayed in `error`,
and both manual `/compact` and the automatic retrigger failed the same
way, so the chat was stuck.

## Fix

`generateCompactionSummary` sends the prepared tool definitions with the
summary request. When the provider rejects that request for size, it
regenerates the summary once without tools, which is the request shape
compaction used before and still fits whenever the history alone does.
The context-window check reuses fantasy's classification and also
matches bad-request wordings fantasy does not parse (OpenAI Responses
API, Anthropic, `context_length_exceeded`).

`BuildToolDefinitions` moves into its own file in `chatloop` so the turn
and the summary share one construction path.

On Anthropic the summary prompt now carries the same `cache_control`
breakpoints as the turn; without them Anthropic never reads the cache,
so the shared tool and history prefix would have been pure cost there. A
compaction override that resolves to a different model shares no prompt
cache and may not support the chat model's provider-native tools, so its
summary request carries no tool definitions; an override resolving to
the chat model itself keeps them. HTTP 413 counts as a size rejection
for the tool-less retry.

## Testing

- Unit:
`TestGenerateCompactionSummaryRetriesWithoutToolsWhenContextTooLarge`,
`TestGenerateCompactionSummaryDoesNotRetryOtherFailures`,
`TestGenerateCompactionSummaryAppliesAnthropicPromptCaching`,
`TestIsContextTooLargeError`, `TestBuildToolDefinitions*`.
- End to end: `TestActiveServer_ManualCompaction/retries the summary
without tools when the prompt exceeds the context window` asserts two
summary requests (with tools, then none), the chat returns to `waiting`
with `last_error` cleared, and the summary is persisted.
- Red-green: with the retry disabled the unit test fails on the returned
provider error and the e2e chat never leaves `error`; with the phrase
matching disabled the Responses, Anthropic, and error-code classifier
cases fail.

## Review record

- Local standards and spec review at 99f7e80a0a9 (pre-rebase content of
8799db2f0e): switched `errors.As` to `errors.AsType`, removed two
comments restating test names, trimmed the phrase list to
provider-attested wordings. Rebase onto main resolved one test-file
conflict (both sides deleted different tests) and inlined the
`chattest.AnthropicErrorResponse` helper that main removed in #29008.
- Codex round 1 on 8799db2f0e: one P1, parent tool definitions sent to
a compaction override on a different provider. Mechanism verified
partially (fantasy skips foreign provider-defined tools with a warning;
`web_search` translates across providers and schemas can be rejected).
Fixed in a7fa7a61e4 by dropping tool definitions when the override is a
separate provider instance, with a red-green case in
`TestActiveServer_CompactionModelOverride`. Thread resolved.
- Codex round 2 on a7fa7a61e4: P1 override tools should drop for any
different model (agreed, guard replaced by an unconditional drop); P2
HTTP 413 should take the tool-less retry (agreed); P2 Anthropic summary
request had no cache breakpoints so the tools bought no cache hit
(confirmed in code, matches earlier dogfood `cache_read=0`). All three
fixed in e7bd72390c with red-green unit and e2e assertions. Threads
resolved.
- Codex round 3 on e7bd72390c: P2 keep tools when the override resolves
to the chat model (agreed, fixed in 6d8d8d6f71 with a red-green e2e
case); P1 missing ARCHITECTURE.md TODO items for the changed compaction
flow (agreed, added in 6d8d8d6f71 per the chatd rule); P2 summary tool
set may differ from the triggering request after a `find_tools`
activation (mechanism confirmed, declined as a cache-efficiency edge
case with the overflow path covered by the retry). Threads resolved.
- Codex round 4 on 6d8d8d6f71: "Didn't find any major issues", reviewed
commit `6d8d8d6f71`, zero unresolved threads. CI green on the same head
(Storybook rerun once after Pixel returned HTTP 500 while uploading
snapshots; this branch has no `site/` changes).

> Xum, an AI coding agent, prepared this PR on behalf of @ibetitsmike.

---------

Co-authored-by: Paul Frybarger <pfrybar@frymail.net>
2026-09-18 23:19:00 +02:00
Ethan 77479408b4 feat: make the Coder Agents stream silence timeout configurable (#29394)
The `chatloop` stream silence guard cancels and retries a model attempt
after `10m` of silence, and it was hard-coded. It's armed before the
request opens, so it also bounds time to first token, and slow local
models (e.g. `llama-server` on a large prompt) can spend longer than
that on prompt processing alone, meaning every attempt just gets
cancelled and retried.

This PR adds `--chat-stream-silence-timeout` /
`CODER_CHAT_STREAM_SILENCE_TIMEOUT` (default `10m`) and passes it to
both the main loop and the advisor. Setting it to `0` disables the guard
entirely, and anything above `24h` is rejected, which also keeps the
watchdog arithmetic below from overflowing.

Whilst doing this I ran into the runner's `15m` task watchdog, which
kept counting during an open stream. That silently cancelled any healthy
stream longer than `15m`, and would've stopped a silence timeout above
`15m` from ever firing. To fix, the silence guard now kicks the watchdog
out to `max(15m, silence + 5m)` whenever it arms or sees a part. Silent
streams still fail through the guard, healthy streams can run for as
long as they like, and hang detection is unchanged everywhere else. With
the guard disabled there's nothing to kick against, so arming it just
stops the watchdog for the rest of the attempt. That does mean the DB
write after the stream isn't bounded either when `0` is set; I think
that's fine for an opt-in setting, and a user can still stop the chat
from the UI if it wedges there.

I did consider removing the guard entirely, but the provider HTTP
clients have no timeout, so a provider that holds the connection open
would only ever be caught as a generic task hang, which is what #25782
was fixing.
2026-09-18 15:24:15 +10:00
Michael Suchacz 26e4429362 test: fold near-identical test bodies into tables across eight packages (#29463)
Folds groups of near-identical test bodies (same parent, same parallel
flag, differing only in inputs and expected values) into table-driven
tests in eight packages: `coderd/templatebuilder`, `scripts/develop`,
`agent/agentfiles`, `coderd/x/chatd/chatloop`,
`coderd/x/chatd/chatdebug`, `enterprise/aibridgeproxyd`, `cli`, and
`coderd/database/db2sdk`. One commit per package.

Every case keeps its previous subtest name, `t.Parallel()` placement is
unchanged, and the set of `=== RUN` names is identical except where
standalone `TestX_Y` funcs became `TestX/Y` subtests (chatloop: 9 funcs
into `TestExecuteLocalTools_BatchRuntime`; chatdebug: 15 funcs into two
tables). Members whose bodies differ in more than data stayed as they
were (listed per package in the commit messages). In `agent/agentfiles`
the four `TestFuzzyReplace_*` funcs were already tables, so the shared
request plumbing was hoisted into one helper instead of merging the
funcs and losing their per-rule docs. Net -1035 lines across 9 test
files; runtime is unchanged (all sub-100ms unit tests).

| Package | Diff | Funcs removed | Subtests | RUN lines |
|---|---|---|---|---|
| coderd/templatebuilder | +237 / -291 | 0 | 31 -> 31 | 33 -> 33 |
| scripts/develop | +194 / -246 | 0 | 27 -> 27 | 28 -> 28 |
| agent/agentfiles | +117 / -260 | 0 | unchanged | 58 -> 58 |
| coderd/x/chatd/chatloop | +224 / -306 | 9 | 3 -> 12 | 14 -> 15 |
| coderd/x/chatd/chatdebug | +251 / -537 | 13 | 0 -> 15 | 42 -> 44 |
| enterprise/aibridgeproxyd | +287 / -419 | 0 | 31 -> 31 | 32 -> 32 |
| cli | +143 / -326 | 0 | 15 -> 15 | 16 -> 16 |
| coderd/database/db2sdk | +130 / -233 | 0 | 9 -> 9 | 10 -> 10 |

## Validation

- Per package: `go test -count=1 -tags=testsmallbatch -parallel=8
./<pkg>/ -run <regex> -v` before and after, with the `=== RUN` name sets
diffed
- `go test -race -count=2 -shuffle=on` on each converted test: pass
- `golangci-lint run ./<pkg>/...` with a fresh cache: 0 issues in all
eight packages
- Repository pre-commit hook (including lint/go) passed on every commit

Part of the test-suite speedup series (see #29457); independent of the
other PRs.

## Remote validation

Remote run on a dogfood workspace (separate compute), chat
https://dogfood.cdr.dev/agents/5bbe73d0-866a-4841-9cc3-c5e5b4d97e44,
tested head `d7c55d6b0e8` against base `9d973372774`: all eight packages
pass on the head and under `-race -count=3 -shuffle=on`; package times
are sub-second on both sides (unchanged). `=== RUN` name sets: chatloop
15 to 16 lines with all nine `TestExecuteLocalTools_<X>` leaves
surviving as `TestExecuteLocalTools_BatchRuntime/<X>`; chatdebug 42 to
44; `aibridgeproxyd` `TestNew` 32 to 32, byte-identical sorted list. One
correction to the text above: seven chatdebug leaves are renamed, not
preserved verbatim (`TestRecordingTransport_*UnknownLength*` became
`TestRecordingTransport_UnknownLengthClose/<name without
"UnknownLength">`), so `-run` filters keyed on those old names need
updating. Endorsed verdict: PASS.


> Xum (an AI agent) authored this PR on behalf of @ibetitsmike.
2026-09-17 19:26:18 +02:00
Michael Suchacz 8397680687 fix: keep text and image together in MCP tool results (#29332)
## Problem

MCP tool results that mix text and an image (Playwright's default
`browser_take_screenshot` returns `[text, image]`) were reduced to text
only in both MCP conversion paths, so screenshots never reached the
model. The generic tool renderer also showed media results as a JSON
dump of base64.

## Fix

- `mcpclient` and `chattool`: keep all text plus the first usable binary
item. Error flags survive empty and MIME-less results. Only one binary
payload is represented; additional images are still dropped.
- `chatloop`: at execution time, cap tool media at the 10 MiB attachment
limit and correct or reject declared image types by byte signature
(sniffing, not decoding).
- `chatd` prompt build and `chatprovider`: at generation and compaction,
replace media the destination provider cannot accept with the text plus
an omission note (Anthropic/Bedrock: JPEG, PNG, GIF, WebP under the
inline cap; OpenAI/Azure and Google: their image allowlists, applying
the configured vendor's rules behind `openai-compat`). Persisted history
is unchanged, so the UI keeps the payload and provider switches keep
working.
- `site`: results flagged `is_media` render through `ToolResultMedia`
(image with the existing lightbox, or a short notice for non-image
media, plus the text). Unflagged media-shaped JSON stays in the JSON
viewer.
- `go.mod`: pin [coder/fantasy
`9a3598480a71`](https://github.com/coder/fantasy/commit/9a3598480a71e45901bfcfbf566ffa42c6bbdd63),
main's `a511353f83eb` plus the batch-order fix `460d4fdd113b` (Chat
Completions and compatible transports emit every tool result before
synthetic user images). The fix lives on
`mike/mcp-tool-media-batch-order`; no upstream PR yet.

## Validation

Round 3 dogfood UAT on this head, `522bd76a58a`: **PASS**
([chat](https://dogfood.cdr.dev/agents/5e617636-c6f2-46fc-96e5-97ae35ed822e),
real Playwright MCP, Claude Haiku 4.5 and GPT-4o mini through the AI
gateway). Verified in the UI and on the wire: image plus text render
together, the lightbox opens and closes, both survive a reload; audio
shows a notice instead of base64; media-shaped JSON stays in the JSON
viewer; BMP on OpenAI Responses and Chat Completions reaches the model
as text plus an omission note; SVG and PNG histories switch from
Anthropic to OpenAI; cross-provider compaction through the organization
override works; a result over 10 MiB persists only text plus a note, and
a 5.5 MiB PNG is substituted for Anthropic yet still renders in the UI.
Google is not routable on the dogfood gateway (404), so it was not
exercised. The remote agent's own FAIL was a probe artifact (the code
viewer renders in a shadow root) and was overturned on its screenshots
by the independent runner. Rounds 1 and 2 (`65db84a`, `5001de7`) covered
the same API surface earlier.

Known limits: signature sniffing does not catch a truncated PNG with a
valid header (providers return 400 for it), and only the first binary
item of a result is kept. Pre-existing and out of scope: chatd opens a
new MCP session per generation action, so Playwright's default
per-session browser loses page state between steps (UAT used
`--cdp-endpoint`).

Reviews:

| Review | Head | Outcome |
|---|---|---|
| Codex 1 to 5 | `472914f` to `bcf4c72` | 16 findings (empty binary
promoted over text, no provider caps, SVG on Anthropic, provider-switch
gap, Bedrock labels, unbounded media, unvalidated bytes, renderer
gating, lightbox test), all fixed |
| Codex 6, 7 | `8a9a9e0`, `5001de7` | no major issues; one Responses
audio concern [refuted with a live
probe](https://github.com/coder/coder/pull/29332#discussion_r4018514197)
|
|
[Codex](https://github.com/coder/coder/pull/29332#pullrequestreview-5226106293)
| `83e1a10` | two P2s refuted with serializer captures
([MIME](https://github.com/coder/coder/pull/29332#discussion_r4029188332),
[advisor](https://github.com/coder/coder/pull/29332#discussion_r4029160781));
[reassessment](https://github.com/coder/coder/pull/29332#issuecomment-5702295886)
clean |
| Codex | `522bd76` | [no major
issues](https://github.com/coder/coder/pull/29332#issuecomment-5713663081)
on the rewritten tests |
| @DanielleMaywood | `472914f`, `5001de7` | renderer restructure and
explicit audio mapping done; lazy slice cloning removed; two threads
left open for her re-check |

Local pre-push hooks were skipped per Mike's instruction; CI is the
gate.

> Xum acted on Mike's (@ibetitsmike) behalf to create this PR.
2026-09-17 15:09:19 +02:00
Michael Suchacz 0362e5386c fix(coderd/x/chatd/chatloop): tell the model to gather context before retrying an exclusive tool (#29281)
When the model batches `advisor` (an exclusive tool) with other tool
calls, every call in the batch is rejected and the `advisor` result only
says to retry with `advisor` alone. That nudges the model toward
re-calling `advisor` immediately, even when the sibling calls were there
to gather context for it.

Extend the message so the model is told to run the other tools first
when it still needs their output, then retry with only the exclusive
tool call.

Generated by Coder Agents on behalf of @ibetitsmike.
2026-09-15 17:02:48 +02:00
Michael Suchacz 46c904d181 fix: show estimated context usage after compaction (#28984)
After compaction, the context gauge showed the unused-chat placeholder
until the next response. It now shows a clearly labeled estimate based
on the compacted summary, keeps that estimate after reload, and uses the
selected chat model’s context limit. The next response replaces it with
measured usage.

The estimate stays separate from provider-reported token accounting.
Remote dogfood UAT passed the core workflow on
cb18cbdac596231acf821cd612c5a8c5d1db1c7b. It has not been repeated on
the current head, 8cfc3da83c.

> Xum acted on behalf of @ibetitsmike.

<details>
<summary>Review follow-up and validation</summary>

- DanielleMaywood's review of df01d8b536: simplified both nested label
ternaries and removed the unrelated mock handlers in 8cfc3da83c.
Replies: [accessible
label](https://github.com/coder/coder/pull/28984#discussion_r4003806647),
[panel
label](https://github.com/coder/coder/pull/28984#discussion_r4003807264),
[mock
scope](https://github.com/coder/coder/pull/28984#discussion_r4003807845).
All three threads resolved.
- Local visual review, task ecb1c01d70: six states of the label refactor
inspected with no regressions found. Frontend self-review: FE1 through
FE10 passed.
- Passed frontend format, check, and lint; 50 targeted unit tests; 19
stories; 64 label parity comparisons; and the full pre-commit and
pre-push hooks, including Go tests, frontend tests, and the production
frontend build.
- The pre-push run used a process-local LD_PRELOAD shim to refuse
loopback port 3000, preventing unmocked test requests from reaching the
unrelated host service. No hooks were disabled and the host service was
not changed.
- [Documentation analysis is
blocked](https://github.com/coder/coder/actions/runs/34827361213/job/103922480177):
its Coder workspace is dormant, so the review did not run. The workspace
needs reactivation before retrying that check.

</details>
2026-09-14 20:50:12 +02:00
Michael Suchacz 680b2f59cd refactor: remove obsolete chatd code and tests (#29228)
Chatd still carries an unreachable interruption flush path, unused state
and options, an uncalled workspace MCP cache invalidation callback, and
a stream-buffer-drop counter that production never increments. Remove
these paths and their dedicated assertions, and update the generated
metrics inventory and documentation.

Consolidate duplicate publication and rollback tests and move fixture
writers and test-only inspection helpers into test files. Retain the
interruption persistence, state-machine matrix, legacy message format,
and provider compatibility coverage.
2026-09-14 10:51:22 +02:00
Garrett Delfosse c12355d6e5 fix(coderd/x/chatd): carry unanswered user messages through compaction verbatim (#28645)
Fixes
[CODAGT-737](https://linear.app/codercom/issue/CODAGT-737/compaction-can-treat-the-last-user-message-as-an-assistant-message)

## Problem

Auto-compaction can trigger on a turn whose last history message is a
fresh, unanswered **user** message. Two things go wrong:

1. The summary instruction is appended as another user message.
Providers that merge consecutive same-role messages (verified against
fantasy's Anthropic conversion) fuse it with the user's message, and the
summarizer misattributes the user's instruction as assistant intent.
2. After compaction the original user row falls outside the prompt
window, so the instruction survives only through summary fidelity.

## Fix

Carry unanswered trailing user messages through compaction verbatim
instead of summarizing them. All logic lives in `chatd`; `chatloop` is
unchanged apart from prompt wording.

- **Prepare** (`generation_preparer.go`): `pendingUserSegmentStart`
locates the trailing user rows on the persisted rows, before
provider-switch sanitization (which can drop the assistant row
separating them from an already-answered message). Head and tail are
converted separately so the summarizer input excludes the tail exactly.
Falls back to summarizing everything when no usable segment exists.
- **Commit** (`buildCompactionMessages`): replays the tail after the
boundary trio as uncompressed, model-only user rows. The existing prompt
window query includes them; no SQL, UI, or turn-guard changes. Replay
rows carry no usage, so compaction cannot immediately re-fire.
- **Prompt** (`chatloop/compaction.go`): the summary prefix no longer
claims the assistant "was actively working", which is false in this
case. The summarizer prompt is unchanged.
- `ARCHITECTURE.md` compaction section updated.

## Testing

`go test ./coderd/x/chatd/...` green. New:
`TestBuildCompactionMessages_ReplaysPendingUserMessages`,
`TestPendingUserSegmentStart`,
`TestPendingUserSegmentConversionEquivalence`,
`TestPendingUserSegmentDerivedBeforeProviderSwitchSanitization`,
`TestDecisionGeneratesAfterCompactionWithReplayedPendingUser`.

<details>
<summary>Root-cause repro</summary>

Replaying the reported scenario through fantasy's Anthropic provider
against a capture server:

```
=== fantasy prompt (roles as chatd builds them) ===
[2] role=user      "You'll hit compaction now, we'll need to go through some fin..."
[3] role=user      "You are performing a context compaction. Summarize the conve..."

=== wire request to Anthropic ===
[2] role=user, 2 content block(s)
    block[0]: "You'll hit compaction now, we'll need to go through some final checks..."
    block[1]: "You are performing a context compaction. Summarize the conversation so..."
```

</details>

---

🤖 This PR was generated by Coder Agents on behalf of @f0ssel.
2026-09-08 09:31:42 -04:00
Michael Suchacz ac824cd235 chore(coderd): remove dead code from chatd (#29008)
Removes production code in `coderd/x/chatd` that nothing reaches, the
tests that only existed to exercise it, and option/config fields that
callers filled in but chatd never read. No runtime behaviour changes;
the only edits outside `coderd/x/chatd` drop the now-nonexistent fields
at their three call sites (`coderd/coderd.go`, `coderd/exp_chats.go`,
`coderd/coderdtest`).

Net: 42 files, +147 / -1065.

Verification method for every item: `golang.org/x/tools/cmd/deadcode`
(with and without `-test`) rooted at `cmd/coder`, `enterprise/cmd/coder`
and every package that imports chatd; a `go/packages` pass counting
`TypesInfo.Uses` per declared type, const, var, field and method across
the same roots, split into test and non-test use sites, plus a
read-vs-write split for struct fields; `rg` over the whole repo
including `site/src` for json tag names; then `go build ./...`, `go vet
./coderd/x/chatd/...`, `golangci-lint run ./coderd/x/chatd/... ./coderd/
./coderd/coderdtest/`, and `go test ./coderd/x/chatd/...` (all 20
packages) locally.

<details>
<summary>Evidence per deletion</summary>

| Removed | What it was | Why unused |
|---|---|---|
| `subagent.go` `resolveConfiguredModelOverride`, `parseModelOverride`,
`parsedModelOverride` (~135 lines) +
`TestResolveConfiguredModelOverride_*`, `TestParseModelOverride` |
raw-string model override resolver | Only callers were its two tests;
production resolves overrides through `resolveOrganizationModelOverride`
(`title_override.go`, `compaction_override.go`, `subagent.go`). |
| `chatd.go` `removeSkillIndexMessages`, `isSkillIndexMessage` +
`TestSkillIndexRefreshReplacesStaleAliases` | skill-index refresh
helpers | Zero production references; the test simulated a refresh path
no production code performs. |
| `chatd.go` `chatMessage`, `newChatMessage`, `withCreatedBy`,
`appendMessageFields`, `BuildSingleChatMessageInsertParams` (~90 lines)
| insert-params builder | Exported, but every caller was a `chatd_test`
file (6 sites). Replaced by a 25-line `singleChatMessageInsertParams`
helper in `chatd_helpers_test.go`. |
| `chatd.go` `Server.closeInflightAdmission` | one-line wrapper around
`inflightClosed.Store(true)` | Single call site (`Close`); inlined. |
| `chatd.go` `Server.workerID`, `instructionLookupTimeout`,
`pendingChatAcquireInterval`, `maxChatsPerAcquire`,
`chatHeartbeatInterval` | Server struct fields | Assigned in `New` and
never read anywhere (the values reach the worker through
`chatWorkerOptions`, not these fields). |
| `Config.InstructionLookupTimeout`, `homeInstructionLookupTimeout`,
`coderd.Options.ChatdInstructionLookupTimeout`,
`coderdtest.Options.ChatdInstructionLookupTimeout` | timeout plumbing |
The value only landed in the dead `Server.instructionLookupTimeout`; no
CLI flag sets it and nothing reads it. |
| `PromoteQueuedOptions.CreatedBy` | option field | Set by
`coderd/exp_chats.go` and three tests, never read in chatd (promotion
keeps the queued row's `created_by`). |
| `SubmitToolResultsOptions.DynamicTools` | option field | Set by
`coderd/exp_chats.go` (which re-read `chat.DynamicTools` just to pass
it) and three tests, never read in chatd. |
| `attachments.go` `buildAssistantPartsForPersist` (whole file) |
assistant part builder | Stale copy left by #26270; production uses
`buildAssistantParts` in `message_conversion.go`. Its six tests were
retargeted to the live function (now
`message_conversion_internal_test.go`) and a dead-copy assertion in
`subagent_internal_test.go` was dropped. |
| `chatloop.RunOptions` (94 lines) | old loop options struct | Only two
of 23 fields were ever populated, for one internal call to
`prepareMessagesForRequest`, which now takes `model, logger` directly
and no longer returns the ignored canonical slice.
`GenerateAssistantOptions` / `ExecuteLocalToolsOptions` are the live
types. |
| `chatloop.ErrDynamicToolCall`, `ErrStopAfterTool`, `PendingToolCall`,
`PersistedStep.PendingDynamicToolCalls` | dynamic-tool exit protocol |
Zero references outside their declarations (one stale comment in
`chatdebug/service.go` updated). |
| `chatloop.CompactionOptions.Persist`, `.OnError` | callbacks |
`Persist` had zero references; `OnError` was set only by two tests that
asserted it was never invoked. |
| `chatloop` `stepResult.shouldContinue` + its computation | flag |
Written once, never read. |
| `chatloop.StateWaiting` | metric label const | Zero references. |
| `generation.go` `generationOutcome.InsertedMessages`,
`runnerActionMessage` | outcome field | Built on every commit, never
read. |
| `chatstate` `ExecutionState.IsArchived`, `AllOwnershipStates`,
`OwnershipState`/`StateU`/`StateO`, `TransitionAcquire` | state-machine
leftovers | Zero references (the ownership consts became dead once
`AllOwnershipStates` went). |
| `chatstate` `ExecutionState.QueueNonEmpty` | predicate | Only
referenced by the test asserting its own table; the sibling `IsRunnable`
check in that test is kept. |
| `chatdebug.Attempt.RetryClassification`, `.RetryDelayMs` | json fields
| Never assigned anywhere, `omitempty`, and
`retry_classification`/`retry_delay_ms` do not appear in `site/src` or
`typesGenerated.ts`. |
| `chatdebug.RunContext.Kind`, `StepContext.HistoryTipMessageID`,
`RecorderOptions.Provider`/`.Model`, `stepHandle.opts`/`.usage`/`.err` |
context/handle fields | Written (from `chatd.go`,
`chatloop/compaction.go`, `modelcall.go`, `recorder.go`) and never read.
The DB run row keeps its own `kind` column. |
| `chatprovider.MergeProviderAPIKeys` +
`TestMergeProviderAPIKeys_PreservesProviderRegions` | key merge |
Superseded by `ResolveUserProviderKeys`, whose region test already
covers the same behaviour. |
| `chatprovider` `providerCreationError` | one-line `xerrors.Errorf`
wrapper | Single call site; inlined with identical message. |
| `chatretry.IsRetryable` | classifier wrapper | Zero references. |
| `computeruse.IsTool` + `TestComputerUseTool` | one-line forwarder to
`fantasyopenai.IsComputerUseTool` | No production caller; two test
assertions now call the fantasy function directly, and the dropped test
only asserted the library's own constructor. |
| `chatprompt.ToolResultToPart` | one-line forwarder to
`codersdk.ChatMessageToolResult` | Single call site; inlined. |
| `subagent_catalog.go` `formatSubagentDefinitions` | forwarder with
`nil` overrides | Single call site; inlined. |
| `worker.go` `chatWorker.chatWorkerID` | getter | Only tests called it;
they read `opts.WorkerID` directly now. |
| `chattest.AnthropicErrorResponse`, `OpenAIServerErrorResponse`,
`ChatMessageWithParts` (`messages.go` removed) | test fixtures | Zero
references. |
| `chattest` `AnthropicRequest.Options`, `OpenAIRequest.Options` |
`json:",inline"` map fields | `encoding/json` has no inline tag (the
TODO on each said so); never populated, never read. |

</details>

<details>
<summary>Considered and kept</summary>

- `chatstate.CreateChat` (test-only thin wrapper over `insertChat`,
production uses `CreateChatWithID`): documented package API with 25 test
call sites; not worth the churn.
- `chatprompt.MarshalContent` / `MarshalToolResult`: test-only, but they
deliberately produce legacy-format DB rows so the read path's legacy
handling stays covered.
- `chatstate.AllowedExecutionTransitionsFrom` / `AllowedInputStates` /
`AllowedExecutionTransitionOutputs`, `chatdebug.StepFromContext`,
`WithClock`, `WithStaleThreshold`, `WaitUntilIdleForTest`,
`PublishBuffer.BufferedChannels`, `chattool.NewWorkspaceMCPTool`: test
seams that tests actively use to reach otherwise unexported state.
- `AgentCapacityLimiter` interface (one production implementation):
tests inject fakes.
- `applyHookResultMessages` / `appendHookResultMessages`: a named pair
over one placement enum, both used.
- Nil-receiver guards on `*Metrics`, `*Result`, `*streamSyncPoller`,
`*runnerDebugTurn`: nil is a supported "disabled" value on those paths.
- Tests flagged by heuristics as assertion-light
(`TestNopMetrics_DoesNotPanic`, `TestStepHandleFinish_NilHandle`,
`TestRetry_OnRetryNilDoesNotPanic`, the `chatstate`
synthetic-cancellation tests that assert through helpers): they cover
real nil-safety or delegate assertions to helpers; kept.

</details>

<details>
<summary>Out-of-scope findings (not changed here)</summary>

- `chatdebug.Service.FinalizeStale` has had no production caller since
#26270 removed the background sweep loop; only its tests call it, and
comments in `chatd_debug.go` and `database/querier.go` still describe
the loop. Deleting it would erase the implementation a fix needs, so it
is left for a decision.
- `coderd_chatd_chats` (gauge) and
`coderd_chatd_stream_buffer_dropped_total` (counter) are registered and
documented in `docs/admin/integrations/prometheus.md` but never set or
incremented in production; `Metrics.RecordStreamBufferDropped` is
test-only. Removing them changes the exported metric set and the docs,
so they are left alone.

</details>

## Declined review findings

None yet.

> Xum (AI agent) opened this PR on behalf of @ibetitsmike.
2026-09-05 11:39:10 +02:00
Jaayden Halko b428c8feb6 feat(coderd): bill completed local tool batches in agent runtime (#28360)
**Stack**: #28359 → **#28360 (this PR)** → #28361 → #28362

## Summary

Completed local tool batches now count toward Coder Agent runtime
(`chat_messages.runtime_ms`, the source of truth for
`hb_agent_runtime_v1`), excluding sub-agent orchestration tools to avoid
double counting, and without multiplying runtime for parallel tool
calls.

Second of a four-PR stack splitting #28211. Stacked on #28359. Refs
CODAGT-928
(https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).

## Problem

Agent runtime previously measured only model invocation wall clock
(stream open to fully consumed). Time spent executing local tools
between steps, including file operations, terminal commands, workspace
provisioning, and MCP tools, was deliberately excluded, which
undercounts the product definition of "actively processing a task".

Naive inclusion has two hazards: a batch of parallel tool calls would
bill N windows for one wall-clock wait, and `wait_agent` would re-bill
child agents that already bill their own model and tool time.

## Fix

Each completed local tool batch bills one window: the union of the
billed tools' execution intervals, persisted as `runtime_ms` on a
dedicated usage record appended after the batch's tool-result rows. The
record is a tool-role message with `visibility='model'`, so it never
reaches the API, SSE, or clients, and prompt replay drops it because its
single internal `tool-batch-usage` part converts to no provider content.
Its content carries an audit payload (`billed_ms`, `billed_calls`) so
the billed window is inspectable after the fact; real tool-result rows
never carry batch-level runtime. Concurrent calls all start at batch
start, so 5 parallel 10s reads bill 10s, not 50s; serial calls
(`SerialToolCalls`) count only from their own launch, so unbilled waits
before them do not count.

Sub-agent orchestration tools (`spawn_agent`, `wait_agent`,
`message_agent`, `interrupt_agent`, `list_agents`,
`list_subagent_models`, plus the deprecated `close_agent` alias) never
extend the window: every chat, including children, applies the same
rules to its own runtime, so a parent's `wait_agent` window would double
count. A lone `wait_agent` bills 0 and appends no usage record;
`execute` 10s in parallel with `wait_agent` 60s bills 10s. A test pins
the unbilled set to the registered sub-agent tool catalog so they cannot
drift.

`GetTotalChatMessageRuntimeMsInRange` already sums `runtime_ms`
role-agnostically, so the usage record is picked up with no schema,
query, or cron changes.

Client-executed dynamic tools, external agents, parked/idle time, and
retry backoff remain unbilled. Interrupted batches still bill only the
model window; the rest of the stack adds partial-window billing via the
same usage record.

`ExecuteLocalToolsOptions` also gains an optional `ToolBillingRecorder`
(`RecordStart`/`RecordComplete`) observing per-occurrence dispatch-order
execution stamps. This PR wires no recorder; the next PR in the stack
connects it to the message part buffer for interrupt billing.

**NOTE**: Reported agent runtime (`hb_agent_runtime_v1`) increases from
deploy forward. There is no backfill and no feature flag.
2026-08-25 16:40:41 +07:00
Jaayden Halko d44aae37de refactor(coderd/x/chatd): return PersistedStep directly from ExecuteLocalTools (#28359)
**Stack**: **#28359 (this PR)** → #28360 → #28361 → #28362

`chatloop.ToolExecutionOutcome` only wrapped a `PersistedStep`, so
callers had to reach through `outcome.Step` for every field.
`ExecuteLocalTools` now returns the `PersistedStep` directly and the
wrapper type is deleted. No behavior change.

First of a four-PR stack that makes local tool execution count toward
Coder Agent runtime (splitting #28211). This refactor keeps the rename
noise out of the feature PRs.

Refs CODAGT-928
(https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).
2026-08-25 16:31:06 +07:00
Michael Suchacz a48aedd924 fix(coderd/x): surface Gemini malformed-function-call stream deaths as retryable errors (#28470)
## Problem

When Gemini's OpenAI-compatible endpoint rejects a model-generated
function call server-side, it ends the SSE stream cleanly with the
nonstandard finish reason `function_call_filter:
MALFORMED_FUNCTION_CALL` after streaming only thought summaries; the
rejected call never reaches the wire. chatd treated this as a normal
completion: fantasy maps the unrecognized finish reason to `unknown`,
the reasoning block never closes (the only non-thought delta is the
`</thought>` marker, which the transport seam strips to an empty string,
and the openaicompat hook only ends reasoning on a non-empty content
delta), so the step accumulates no content and the generation loop
finishes the turn as complete. The user sees the model think for ~40
seconds and then nothing: no assistant message, no `last_error`, chat
status `waiting`. Observed twice in a row in production on
gemini-3.7-flash, with "Resume" reproducing it identically.

## Fix

Two independent layers:

- `coderd/x/googleopenai`: the stream rewrite now converts any chunk
whose `finish_reason` starts with `function_call_filter` into an
OpenAI-style SSE `{"error": ...}` event embedding the raw reason.
openai-go turns error-bearing events into stream errors, so the failure
rides the existing stream-error path instead of ending the stream
cleanly.
- `coderd/x/chatd/chaterror`: classifies that injected error as
retryable (kind `generic`, provider `google`) with a clear user-facing
message, so the existing generation retry machinery re-runs the step and
persists a `last_error` if retries exhaust.
- `coderd/x/chatd/chatloop`: provider-agnostic guard: a step that
produced no user-visible content and no tool calls under a finish reason
of `unknown`, `error`, `other`, or `tool-calls` (a tool-calls finish
that delivered zero calls) now returns a retryable error instead of
silently completing the turn. `stop` and `length` finishes keep their
existing semantics.

Tests cover the seam rewrite (live-capture SSE shape plus standard
finish reason passthrough), the new classification, and the chatloop
guard (error cases plus preserved stop, length, text, and tool-call
behavior). Each layer was red-green verified independently.

Remote dogfood UAT ran against this exact commit: normal reasoning and
tool-call chats on a real model complete cleanly with no spurious guard
errors and no retry loops. The Google-side failure itself is not
deterministically triggerable against live Gemini and is owned by the
unit tests.

> Xum acted on Mike's (@ibetitsmike) behalf.

<!-- xum-attribution: model=claude-opus, thinking=enabled -->
2026-08-24 17:09:00 +02:00
Michael Suchacz 9168dbf4d3 refactor(coderd/x/chatd): unify LLM model-call construction behind a resolver (#28145)
Centralizes every LLM client build in chatd behind one resolver and
makes all model calls behave the same, instead of preserving per-flow
quirks.

## Why

`chatd` built LLM clients in five places (standard turn, compaction
override, quickgen, title override, advisor), each repeating config
parsing, route resolution, client construction, and debug wrapping with
slightly different behavior. An earlier iteration of this PR centralized
construction but preserved every historical quirk behind
purpose-specific spec constructors and policy flags. Review concluded
the quirks themselves were the problem.

## What changed

- `Server.resolveModelCall` (`modelcall.go`) is the single pipeline from
a `modelCallSpec` to a ready client plus call metadata. Callers build
spec literals inline; the only spec fields are data: model source (chat
config, explicit config row, or fixed provider/model pair for computer
use), requested reasoning effort, chatd-scoped routing, and build
options.
- Uniform behavior for every call:
- Provider options are always derived. Whole-chat summaries, turn status
labels, and chat-model compaction summaries previously sent none; they
now follow the config like everything else.
- `MaxOutputTokens` always defaults to 32k when the config leaves it
unset. The compaction summary call nulls the cap after copying the
template because it is non-streaming and the Anthropic SDK rejects
non-streaming requests whose cap implies a completion over ten minutes.
- Debug recording always wraps the client when chat debug is enabled,
labeled with the resolved provider/model. The debug-rebuild paths
(`manualTitleDebugSpec`, `quickgenDebugSpec`) are deleted; debug helpers
now only create and finalize run records, since run association flows
through the generate context, not the model.
- Clients are always constructed with the configured model string and
unmutated route, so AI Gateway validation sees the name exactly as
configured. The resolved identity feeds metadata, logs, and debug labels
only.
- `resolvedModelCall.newCall` / `newObjectCall` are the only production
constructors of `fantasy.Call` / `fantasy.ObjectCall`; object calls now
carry the config's sampling parameters.
- Net: -240 lines.

## Validation

- Full `./coderd/x/chatd/...` suite green (PostgreSQL-backed).
- Red-green toggles verified for the three key unifications (option
derivation, token default, summary cap removal).
- ARCHITECTURE.md: prose the refactor made stale is deleted; no new
resolver documentation is added.

> Mux acted on Mike's behalf for this PR.
2026-08-20 20:54:52 +00:00
Danielle Maywood 5480299cef fix: serialize empty MCP tool properties as {} not null for OpenAI (#28380)
## Summary

Fixed a bug where MCP tools with an empty `properties` schema had their
tool definitions sent to OpenAI with `"properties": null`, causing
OpenAI to reject the request.

## Problem

When an MCP server reports a tool with an empty input schema, e.g.:

```json
{
  "type": "object",
  "properties": {},
  "required": []
}
```

Coder was forwarding the tool definition to OpenAI as:

```json
{
  "type": "object",
  "properties": null
}
```

OpenAI's function-calling API rejects `properties: null`, making these
tools unusable with OpenAI models.

## Root cause

The empty `properties` object collapsed to Go `nil` at two layers:

1. `agent/x/agentmcp/manager.go` `toolInputSchemaMap` stripped the
present-but-empty `"properties": {}` (guarded by `len(properties) > 0`),
so the wire copy lost it entirely.
2. `coderd/x/chatd/chatloop/chatloop.go` `buildToolDefinitions` embedded
`info.Parameters` directly into the input schema. When nil,
`encoding/json` marshals it as `null`.

## Fix

- `buildToolDefinitions` now substitutes an empty object when
`Parameters` is nil, mirroring the existing `required` guard (the sink,
protects all tool sources).
- `toolInputSchemaMap` now preserves an empty `properties` object
instead of dropping it (the upstream producer).

## Tests

Added regression tests (failing before, passing after) covering both
layers and asserting the schema serializes to
`{"type":"object","properties":{}}`:

- `TestBuildToolDefinitionsNilPropertiesBecomesEmptyObject`
- `TestToolInputSchemaMapPreservesEmptyProperties`
-
`TestWorkspaceMCPToolInfosFromResources/EmptyPropertiesYieldsEmptySchema`

---

*This PR was generated by Coder Agents.*
2026-08-20 15:56:58 +02:00
Michael Suchacz 7724ee281a feat: defer MCP tool schemas behind a find_tools search (#28225)
## Summary

When the `mcp-tool-search` experiment is enabled, chatd stops inlining
connected MCP tool schemas into every generation. It instead exposes a
built-in `find_tools` tool whose description carries a compact catalog
of the deferred tools, and only ships full JSON schemas for tools the
model has activated by searching or by calling them directly.

Closes [CODAGT-760](https://linear.app/coder/issue/CODAGT-760).

## Problem

Tool-heavy agent configurations (GitHub, Linear, Notion, and dev-tooling
MCP servers) inline over 100k tokens of tool schema definitions into
every generation. Initial uncached requests reached ~216k tokens with
time-to-first-token close to nine minutes, while the model typically
invokes only a handful of tools per turn.

## How it works

- `decideMCPToolSearch` defers external and workspace `.mcp.json` MCP
tools whenever the experiment is enabled. Native, dynamic, provider,
skill, and transport tools are never deferred.
- `find_tools` embeds a server-grouped catalog in its tool description
(degrading to names-only, then counts-only, then a constant-size summary
past a context-scaled size cap) and scores keyword matches across tool
names, descriptions, parameter schemas, and server metadata. Queries can
scope to one server with a `server:` prefix, and exact `names` arguments
always activate.
- Activation state is ephemeral: it is re-derived each generation from
surviving chat history (`find_tools` results and direct calls to
deferred tools), so activations naturally lapse when compaction
summarizes them away. Aggregate activated schema weight is capped at 10%
of the context window, shedding the least recently activated schemas
first; `find_tools` shares that budget across parallel calls in one
step. No new persistence.
- Deferred tools stay registered for execution, so the model can call a
cataloged tool directly without searching first; the schema is activated
for subsequent steps.
- Fail-open: the experiment being disabled, an empty candidate set, or
an MCP tool named `find_tools` all disable deferral, leaving today's
behavior byte-identical on the wire.
- Prometheus counters/histograms track `find_tools` calls, matches,
activations, and deferred token weight.
- The conversation timeline renders `find_tools` calls with a collapsed
search summary and expandable match list, falling back to the generic
renderer on malformed payloads.

## Validation

- Unit tests for the catalog, matcher, experiment-gated decision, and
activation derivation; end-to-end chatd generation tests covering
search-then-call, direct-call activation, experiment-off wire parity,
compaction lapse, and subagent tool gating.
- Storybook interaction tests for the timeline rendering and
malformed-payload fallback.
- Remote dogfood UAT on dev.coder.com passed: deferral with a real MCP
server and Anthropic model, direct calls without prior search,
activation persistence across turns, experiment-off parity, and clean
UI/console.

> Disclosure: Mux (AI agent) authored this PR on Mike's behalf.
2026-08-18 19:12:47 +02:00
4b7494be72 feat: harden chat generation runtime instrumentation for billing (#27451)
Closes CODAGT-835

## Summary

`chat_messages.runtime_ms` becomes the billing source of truth for Coder
Agents runtime (summed hourly by #27312), but it was built for
debugging: the June refactor (#26270) silently stopped recording
tool-step runtime, compaction was never measured, and interrupted turns
lost their partial runtime entirely. This PR defines the billable
metric, closes the paths that dropped it, and documents the definition
where the data lives.

## The billable definition

**`runtime_ms` is the wall-clock duration of the model invocation that
produced the persisted message content**, measured from just before the
provider stream opens until it is fully consumed.

What counts:

- Assistant generation steps, in top-level and sub-agent chats
(sub-agents are ordinary chats on the same generation path).
- Compaction summarization calls, persisted on the compaction assistant
message (**new**).
- Interrupted attempts: the message-part episode's lifetime is persisted
on the partial assistant message committed by `FinishInterruption`, so
partial generation time survives interruption (**new**; measured via a
new `Buffer.EpisodeDuration`, which works even though the generation
goroutine and the interrupt task are different tasks).

What deliberately does not count (each is documented in code and docs):

- **Local tool execution.** Tool wall time includes idle waits, most
importantly `wait_agent` polling a sub-agent chat that already bills its
own model invocations; billing the batch would double count, and
excluding one tool from a concurrent batch's wall time is ill-defined.
Pre-refactor instrumentation did include tool time; this makes the
exclusion an explicit product definition instead of a silent regression.
- **Failed model calls whose output is discarded** (retried attempts,
terminal errors, content-filter refusals). They persist no content, so
they bill nothing; billing errs toward undercounting. Notably a
stream-silence timeout can burn 10 idle minutes before a retry, which
should not be billable "active generation". If product later wants
failed attempts billed, that needs a place to persist runtime on error
turns (`FinishError` inserts no rows today) and is a deliberate
follow-up, not instrumentation drift.
- **Ancillary calls that produce no chat messages** (title generation,
advisor, turn summaries) and all idle/parked time (`requires_action`,
queueing).

The definition is documented as `COMMENT ON COLUMN
chat_messages.runtime_ms` (migration 000551, surfacing as a Go doc
comment on `ChatMessage.RuntimeMs`), on
`chatloop.PersistedStep.Runtime`, in the chatd architecture doc, and in
the Spend Management docs page.

## Index for the hourly scan

None needed: `GetTotalChatMessageRuntimeMsInRange` (#27312) filters an
hour-wide `created_at` range, which the existing
`idx_chat_messages_created_at` b-tree already serves; the residual
`runtime_ms IS NOT NULL` filter applies to one hour of rows. A partial
index would add permanent write amplification for a query that runs once
an hour.

> [!NOTE]
> Migration 000551 is also claimed by #27312; whichever merges second
renumbers via `fix_migration_numbers.sh`.

## Tests

- End-to-end: the existing full-server generation test now asserts
`RuntimeMs.Valid` on the committed assistant row (it previously read
`.Int64` without checking `.Valid`, so it passed on NULL).
- Interrupted turn: full task-level test (real DB, mock clock) asserting
the partial assistant message persists the attempt's runtime.
- Errored stream: asserts a failed invocation yields no step and no
runtime.
- Tool-using turn: asserts runtime lands on the assistant row only and
tool rows stay NULL.
- Compaction: asserts the summarization call duration is recorded and
lands on the compaction assistant message only.
- `messagepartbuffer.EpisodeDuration` unit coverage.

Blocks: CODAGT-843 (B3), CODAGT-838 (D8).

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: Hugo Dutka <hugo@coder.com>
2026-08-06 16:09:38 +07:00
Michael Suchacz c17bed25e0 feat: wire chat lifecycle hooks into chatd (#27429)
Wires chat lifecycle hooks into chatd, gated by the
`agent-lifecycle-hooks` experiment. Part of the lifecycle hooks stack
(#27401, #27428, #27430). See `docs/admin/setup/chat-lifecycle-hooks.md`
for the consumer-facing contract.

## Summary

When a hook URL is configured, chatd dispatches `session_start`,
`user_prompt_submit`, `pre_tool_use`, `post_tool_use`, `pre_compact`,
`post_compact`, and `stop` events to the consumer and applies its
responses.

## Design

- **Stateless**: Coder stores no hook dispatch or decision state.
Delivery is at least once; consumers deduplicate on stable payload
identifiers (chat ID, event type, tool-use ID) and answer duplicates
with the same decision.
- **Admission-time prompt effects**: `user_prompt_submit` dispatches
exactly once per submission (create, send, queue, edit, subagent spawn)
and folds its effects into the stored prompt as typed message parts:
original-or-overridden user parts, then model-only `hook-context`, then
a user-visible `hook-notice`. Hook context is stripped from every
client-facing conversion; hook notices are excluded from model prompts.
The server rejects hook parts in client-submitted content.
- **Tool gating**: `pre_tool_use` allow can override tool input; deny
becomes a synthetic denied tool result, with any returned model context
persisted as a model-only transcript row so it never reaches clients.
The denial text identifies an external policy (the deployment's
lifecycle hook) as the source and marks the decision as persistent, so
the model explains the denial instead of retrying it or misreporting it
as an infrastructure failure.
- **Fail closed**: a dispatch failure rejects the triggering request or
moves the chat to the error state in the same transaction as the
affected step, so a runnable state is never published with unapproved
content.
- **Admission before persistence**: `pre_tool_use` is dispatched for the
calls the model produced, before the assistant message is stored. See
"Staged tool admission" below.
- **Fresh dispatch per tool call**: every non-provider-executed tool
call is decided by its own `pre_tool_use` dispatch; Coder never reuses
an earlier decision on the consumer's behalf. Retries re-dispatch the
same logical event.

## Structure

All hook dispatch flows through one seam: entry points build a
`chathooks.Chat` (chat identity) and a `chathooks.Message` (event
details) and call `Trigger.Trigger`, the only component that talks to
the dispatcher. The integration lives in the `coderd/x/chatd/chathooks`
subpackage, split by responsibility:

- `trigger.go`: the trigger seam; builds the wire envelope per event,
normalizes deny into a typed error, and holds the package's single
enabled-check.
- `effects.go`: pure conversion of hook results into transcript rows and
prompt parts.
- `errors.go`: failure classification (dispatch error messages, denial
mapping, tool-result dispatch-failure scanning).
- `tooluse.go`: the tool-call gate (`pre_tool_use` preflight,
`post_tool_use` payloads, applying admitted input to the step).

Server-bound glue stays in `coderd/x/chatd/hook_server.go`: the
chat-parking dispatch error handlers, the step-commit row insertion
wrappers, and the dynamic post-tool-use state loader, which depends on
chatd validation types.

This PR adopts the `codersdk/x/agenthooks` and
`coderd/x/agenthooks/dispatch` import paths introduced at the tip of
#27401; intermediate commits still reference the pre-move paths and are
not individually buildable.

## Staged tool admission

`pre_tool_use` originally ran at tool execution time, which is after the
assistant message carrying the tool call was already committed. An
`input_override` therefore had to rewrite stored message content in
place. @hugodutka pointed out that chatd treats message content as
immutable, and that the rewrite was a shortcut rather than a
requirement.

It was also a correctness problem in its own right: the rewrite only
updated the database, so the transcript could show one input while a
different one had executed.

The hook now runs before the step is persisted:

```text
provider stream ends (tool calls complete, in memory)
  -> pre_tool_use dispatch per call
  -> ONE transaction: assistant row with admitted inputs, synthetic denials, hook rows
  -> execute
```

The step is inserted once, carrying the input the tool runs with.
`UpdateChatMessageContentByID` and `Tx.UpdateMessageContent` are deleted
from #27428, so message content stays immutable.

Two consequences, both intentional:

- **Clients converge rather than wait.** Tool-call parts still stream
live, so a rewritten call briefly shows the model's proposed input
before the committed message replaces it. The chat store already clears
stream state when an assistant message arrives, so the stored input wins
with no frontend change and no added latency before tool cards appear.
- **A call already in history was already admitted.** Execution consumes
the stored input instead of dispatching a second decision, which keeps
one dispatch and one set of hook effects per call. A consumer policy
change between admission and execution applies to later calls, not to
calls already admitted.

The per-chat debug endpoint still records the provider's original tool
input. Its purpose is to report provider behavior, and it requires an
explicit per-chat debug flag; the invariant here covers the transcript.

## Configuration

Adds `chat-hook-url`, `chat-hook-secret`, `chat-hook-timeout`, and
`chat-hook-enabled` deployment options with startup validation. The
flags are hidden from `coder server --help` while the feature is
experimental; the setup guide documents them.

## Tool input validation

Built-in tool arguments reach a consumer as raw JSON with key spelling
preserved, but the tools decode those bytes with Go, which matches
struct fields case-insensitively and keeps the last match. A policy
reading `path` could therefore authorize one value while the tool
executed another, and a lone case variant such as `{"PATH":"/secret"}`
was invisible to a policy checking for `path`.

Coder now rejects a built-in tool call whose input repeats a key or
spells a schema property with different capitalization, before the
`pre_tool_use` dispatch, so a consumer is never asked to authorize bytes
whose meaning depends on the reader. Rejected calls produce an error
result the model can retry; unambiguous calls in the same batch still
run. A consumer-authored `input_override` is rechecked after the
dispatch and fails the turn closed, because the model cannot correct it.
Dynamic and MCP inputs are excluded because the client and the workspace
agent execute those calls rather than coderd.

Two paths needed more than a schema check. Execution resolves a
deprecated tool name to its canonical tool, so validation resolves
aliases first. The `edit_files` decoder also reads `search` and
`replace`, which its schema does not advertise, so those aliases are now
matched exactly and their case variants ignored.

A hook denial now returns a structured 403 carrying `kind:
"hook_denied"`, mirroring the dispatch-failure response that already
carries its own kind. Without it a client cannot tell a policy decision
apart from a generic failure, and the chat UI titled a denial "Request
failed". Adding a kind needs no migration: `ChatErrorKind` is persisted
only inside the JSONB `chats.last_error` column, whose decoder accepts
unknown kinds.

The hook docs also correct the tool-input convergence window. A batch
dispatches sequentially before the assistant row commits, so the
original input stays visible for a span that scales with the number of
tool calls in the step rather than a single hook timeout.

> This PR was written by Mux, an AI coding agent, on Mike's behalf.
2026-07-29 11:39:12 +00:00
Michael Suchacz e9951b07d4 fix(coderd/x/chatd/chatloop): surface reasoning-only content-filter refusals as terminal errors (#27476)
## Problem

Anthropic can end a stream with `stop_reason: "refusal"` after reasoning
content has already streamed. The content-filter guard in
`chatloop.GenerateAssistant` only fired when the step content was
completely empty, so a reasoning-only refusal bypassed it: the turn
finished as `status=waiting` with `last_error=null`, and the user saw
the chat silently stop mid-turn with no explanation. This looked like a
Coder fault when the provider had rejected the response. Observed twice
in dogfood on 2026-07-23 (chat `c72f99fc`, debug steps show
`finish_reason=content-filter` with reasoning-only content).

## Change

Treat a content-filter finish as terminal whenever the step produced no
user-visible output. A new `hasUserVisibleContent` helper counts any
non-reasoning part (text, tool call, tool result) as user-visible;
reasoning-only or empty steps now return the existing
`contentFilterError`, which flows through the established pipeline:
classified `ChatErrorKindContentFilter` (non-retryable, refusal
category/detail when provided), persisted `chats.last_error`, streamed
error event, and the "Response blocked" callout in the chat UI.

Behavior for steps with visible text or tool calls is unchanged, and the
frontend needs no changes.

## Testing

- New regression subtest `ReasoningOnlyContentSurfacesTerminalError`
(reasoning stream then content-filter finish) beside the existing
empty-content and partial-content subtests, which are unchanged.
- `go test ./coderd/x/chatd/...` and lint pass.
- Dogfood UAT against a local dev instance with a mock Anthropic
upstream passed all three scenarios: reasoning-only refusal shows the
"Response blocked" callout with `last_error.kind=content_filter` and no
retry affordance; text-then-refusal still completes normally; empty
refusal still errors.

> This PR was created by Mux acting on Mike's behalf.
2026-07-24 19:40:53 +02:00
Michael Suchacz 3227cac217 feat: add manual chat compaction via /compact (#27081)
Adds a user-triggered `/compact` action for Coder Agents chats: typing
`/compact` in the composer (or picking it from the `/` trigger menu)
summarizes the conversation so far to free up context window space.

## How it works

- New `POST /api/experimental/chats/{chat}/compact` endpoint
(owner-only, RBAC `ActionUpdate`, excluded from the public API reference
via `x-apidocgen skip`). It marks the chat with a durable one-shot
`chats.compaction_requested_at` signal and moves it `waiting -> running`
via a new `RequestCompaction` state transition; no message row is
inserted. AI Gateway attribution needs no per-request key: generation
preparation resolves the owner's synthetic API key (#27170) like any
other turn.
- `RequestCompaction` hands off chat ownership (clears
`worker_id`/`runner_id`) so a worker acquisition hint is published;
since the transition changes no history, the previous runner could
otherwise miss the request under reordered pubsub delivery.
- The background chat worker picks the chat up like any other turn. A
pending manual request takes precedence over turn completion in the
generation decision, and forces compaction even below the automatic
threshold (and when compaction is disabled via threshold=100). The
commit step consumes the request marker in the same transaction; any
transition that ends the turn clears stale markers.
- The summary triplet reuses the automatic-compaction path, now tagged
with a `source` (`automatic` | `manual`) that is plumbed through
streamed progress parts, persisted tool JSON, and the UI label
("Summarized (manual)").
- Validation order: busy chats reject with 409 (state-machine conflict),
empty/already-compacted chats with 409 "nothing to compact", archived
chats with 400; the owner usage-limit check runs last so no-op requests
surface the specific conflict instead of a limit error.
- Web UI: the `/` trigger menu now has a built-in "Commands" group
listing `/compact`; submit intercepts exactly `/compact` and calls the
endpoint instead of sending a message. A personal or workspace skill
named `compact` takes precedence over the built-in command; while skill
collisions are still resolving, an exact `/compact` submission is
blocked with a retryable hint instead of leaking as message text.
History and queued-message edits are never intercepted. After
compaction, the context usage indicator resets to its unknown state
until the next assistant response reports fresh usage, instead of
showing the stale pre-compaction number.
- codersdk: `ExperimentalClient.CompactChat`.

Worker-path execution (rather than compacting synchronously in the
handler) reuses the existing lock fencing, live "Summarizing..."
streaming, retry accounting, restart resilience, and debug-run
observability. Rationale documented in `coderd/x/chatd/ARCHITECTURE.md`.

## Testing

- State machine: transition-matrix coverage for `RequestCompaction`,
marker lifecycle tests (carried by lease renewals/queue appends, cleared
by terminal transitions, consumed by commit), ownership handoff +
acquisition hint assertions.
- Worker: decision-ordering and forced-compaction unit tests;
active-server end-to-end test (manual compact below threshold produces a
`source=manual` summary, returns to `waiting`, no assistant follow-up;
busy chat rejected).
- API: success, archived, non-owner, RBAC-denied, empty-chat, no-daemon
cases; usage-limit ordering (at-limit owners still get
state/nothing-to-compact conflicts for no-op requests, with marker
rollback).
- Frontend: Storybook play tests for the Commands menu group, submit
intercept, skill-name collision, queued-edit passthrough, and
manual/automatic tool rendering; unit tests for command availability
resolution and the post-compaction context usage reset.

> This PR was created by Mux, an AI coding agent, working on Mike's
behalf.
2026-07-21 10:58:08 +02:00
Mathias Fredriksson f997afa220 feat(coderd/x/chatd/chatloop): retain user constraints in compaction summaries (#27230)
Compaction summaries drop or soften user-stated constraints,
corrections, and prohibitions, so post-compaction assistants repeat
behavior the user already corrected. Add a summary prompt bullet that
instructs the summarizer to quote them, treat them as standing until
revoked, and attribute rules to their true source instead of
defaulting to the user.

Validated offline on unseen human chats: holdout P1 delta +0.175
(arbitrated), 13/18 cases improve. This improves per-compaction
retention only; it does not address deep-chain correction loss.
2026-07-16 09:18:43 +00:00
Ethan cc11c8a536 feat: surface model content-filter refusals as a blocked chat error (#27118)
Blocked turns from a provider's content filter (Anthropic's `refusal`
stop reason with empty content) previously ended silently on the
"Thinking" spinner. They now end as a terminal `content_filter` error
that renders as a "Response blocked" message with the provider's
category and explanation.

<img width="888" height="335" alt="image"
src="https://github.com/user-attachments/assets/cef85a59-4091-4e62-9d45-1eb06748db48"
/>


Closes CODAGT-611

Follow-ups will involve implementing fallbacks, but this alone is pretty
important
2026-07-15 16:36:31 +10:00
Michael Suchacz a567f6a89f feat: allow admins to override the chat compaction model (#27151) 2026-07-14 16:57:04 +02:00
Michael Suchacz 0ae4554de9 fix(coderd/x/chatd/chatloop): remove compaction timeout (#27007) 2026-07-07 12:22:59 +02:00
Cian Johnston b21e0717d5 feat: remove chat chain mode (#26980)
Removes OpenAI Responses "chain mode" from chatd. Closes CODAGT-445.

- Deletes `chatopenai/responses.go` (chain detection, activation, prompt filtering, response ID extraction) and its tests.
- Deletes the `ChainBroken` classification in `chaterror` and the chatloop retry bookkeeping that disabled chain mode mid-generation.
- Drops the `chain_broken` label from the `coderd_chatd_stream_retries_total` metric.
- Stops reading and writing `chat_messages.provider_response_id`
- Deletes the dead `ClearChatMessageProviderResponseIDsByChatID` query. Dropping the column is a follow-up migration.
- Deletes three chatloop hooks no caller sets (`ReloadMessages`, `DisableChainMode`, `PrepareMessages`), the dead `const AgentChatContextSentinelPath`, and stale chain-mode comments.

🤖 Generated by Coder Agents on behalf of @johnstcn.
2026-07-06 11:57:12 +01:00
Kyle Carberry d6daf273aa fix: tighten single tool result byte budget (#26763)
## Problem

#26637 caps each locally-executed tool result (built-in,
global/deployment MCP, workspace MCP) at a per-result byte budget
derived from the model's context window. The budget was `ContextLimit/2
* 4 bytes` — i.e. **half the window at an optimistic 4 bytes/token**.

On large-context models that is far too generous. With a
`1,000,000`-token `ContextLimit` the per-result cap is **~2 MB**. A user
hit exactly this with a chatd (deployment-pinned) MCP tool: the result
was truncated to **1,998,709 characters** and still overflowed the
prompt. 2 MB of dense text (JSON/logs/code) is ~650k–1M tokens — most or
all of the window for a *single* result — so the cap fired but didn't
actually prevent the overflow.

## Fix

Tighten the two budget constants in `tooltruncate.go`:

| constant | before | after |
| --- | --- | --- |
| `toolResultContextDivisor` | `2` (½ window) | `3` (⅓ window) |
| `bytesPerTokenEstimate` | `4` | `3` (conservative) |

The budget becomes `ContextLimit/3 * 3 ≈ ContextLimit` bytes:

| ContextLimit | before | after |
| --- | --- | --- |
| 1,000,000 | ~2 MB | ~1 MB |
| 200,000 | ~400 KB | ~200 KB |
| unknown (≤0) | 64 KB | 64 KB (unchanged) |

The 16 KB floor and 64 KB unknown-window default are unchanged. A
conservative bytes-per-token estimate is intentional: dense payloads run
well under 4 B/tok, so a lower estimate yields a smaller byte budget
that is less likely to underestimate the true token cost.

No behavioral code paths change — only the two constants and their doc
comments. The existing `tooltruncate_internal_test.go` cases derive
their expectations from the constants (`LargeWindow`) or exercise the
floor/default (`BelowFloor`, `Unknown`), so they remain green.

<details>
<summary>Investigation notes</summary>

Global/deployment MCP tools (`mcpclient.ConnectAll`) are appended to
`prepared.Tools` and execute locally via `ExecuteLocalTools →
executeTools → executeSingleTool`, so the #26637 cap *does* apply to
them for text results (`convertCallResult` joins text content into
`resp.Content`). The cap was simply too large:
`toolResultByteBudget(ContextLimit)` = `ContextLimit/2*4` ≈ 2 MB for a
1M-token window. Reverse-engineering the reported `1,998,709` truncated
characters confirms a `ContextLimit` of ~1,000,000 tokens.

Known gaps left for follow-ups (out of scope here):
- **Per-step aggregate is unbounded.** MCP tools advertise `Parallel:
true` and `executeSingleTool` caps each result independently, so N
parallel calls in one step can sum to N × the per-result cap.
- **Binary/media `Data` bypasses the cap.** Only the text payload is
bounded; `image`/`media`/blob embedded-resource results are
base64-encoded untouched in `executeSingleTool`.
- **Compaction is reactive.** It is gated on the prior step's reported
usage (`latestPromptUsage`), so it can't pre-empt a single large result
appended on the current step.
</details>

---

Generated by Coder Agents on behalf of @kylecarbs.
2026-06-26 13:33:55 -06:00
Mathias Fredriksson 59fcc9c0ad feat: improve sub-agent orchestration tools (#26673)
Tool errors caused orchestrators to abandon spawned agents. Bare error
responses and the close_agent name framed delegation as one-shot: one
transient failure or timeout ended the work, and the orchestrator had no
way to recover or reuse agents.

Renames close_agent to interrupt_agent with a hidden backward-compatible
alias. wait_agent and message_agent return structured payloads instead
of bare errors, so the orchestrator can retry after a timeout, recover
from an error status, or redirect an idle agent. Adds list_agents so
orchestrators can rediscover spawned agents. Adds root-only
orchestration guidance for error recovery.
2026-06-26 13:41:43 +03:00
Kyle Carberry 32217259b7 feat: cap tool output to fit the model context window (#26637)
## Problem

Local tool results were persisted and replayed to the model verbatim,
with no size cap. A single oversized result, most often a multi-megabyte
response from an MCP tool, overflows the prompt on the next request.
Every retry rebuilds the same history and fails the same way, leaving
the chat wedged in `error`. Auto-compaction is reactive (token usage is
only known after a response), so it can't catch a single result that
blows the very next request.

## Fix

Cap every locally-executed tool result at its single choke point,
`executeSingleTool` in `chatloop`, so the cap covers built-in tools,
**global (deployment-pinned) MCP**, **workspace MCP**, and provider
runners uniformly. Because this runs before the result is published to
the live stream and before it is committed, the SSE preview, the
persisted message, and the model replay all see the same bounded output.

The budget is token-aware: a single tool result may use at most half the
model's context window (`~4 bytes/token`), with a `16KB` floor and a
`64KB` default when the window is unknown. Truncation keeps the head and
tail of the output and replaces the middle with a marker telling the
model how much was removed and to narrow its query; it is UTF-8 safe and
never exceeds the budget. Binary media `Data` is passed through
untouched (only the text payload is bounded).

A `coderd_chatd_tool_result_truncated_total{provider,model,tool_name}`
counter and a warning log record each truncation.

## Out of scope

- Provider-executed results (e.g. web search) arrive via the stream, not
`executeSingleTool`.
- Dynamic/external tool results submitted through the `/tool-results`
API are validated as JSON elsewhere.
- Cumulative growth across many results is still handled by context
compaction; this change only bounds any single result.

<details>
<summary>Implementation notes</summary>

- New `coderd/x/chatd/chatloop/tooltruncate.go`:
`toolResultByteBudget(contextLimitTokens)` and
`truncateToolResultText(text, maxBytes)` (pure, unit-tested).
- `chatloop.go`: added `ContextLimit` to `ExecuteLocalToolsOptions`;
threaded a computed byte budget through `executeTools` into
`executeSingleTool`, where `resp.Content` is capped for the text,
media-text, and error branches.
- `generation.go`: passes `ContextLimit: prepared.ContextLimitFallback`
(the model's configured context limit).
- `metrics.go`: new `ToolResultTruncatedTotal` counter +
`RecordToolResultTruncated`.
- Tunable knobs live as constants in `tooltruncate.go`
(`toolResultContextDivisor = 2`, `bytesPerTokenEstimate`,
`minToolResultBytes`, `defaultToolResultBytes`).

Verified: `go build ./coderd/x/chatd/...`, `go test
./coderd/x/chatd/chatloop/...`, and the `chatd` test binary compiles.

</details>

---

Resolves CODAGT-678

Generated by Coder Agents on behalf of @kylecarbs.
2026-06-24 09:16:38 -06:00
Hugo Dutka 4f74a7adee fix: enable goleak in chatd tests (#26335)
Enable goleak in chatd tests and fix some leaks. Addresses
https://github.com/coder/coder/pull/26109#discussion_r3380039964
2026-06-16 12:35:40 +00:00
Cian Johnston 21a2652343 fix(coderd/x/chatd): show correct provider and clean detail for Bedrock errors (#26338)
## Problem
Bedrock errors (routed through aibridge) showed the wrong provider
("Anthropic ...")
and a doubly-wrapped detail string instead of the clean message.

## Fix (chatd only)
- **Provider label:** thread the configured provider to error
classification via
`GenerateAssistantOptions.ErrorProvider`. Transport provider still
drives prompt
  prep, sanitization, and metric labels (unchanged).
- **Detail:** unwrap the SDK transport wrapper (`METHOD "URL": NNN
{body}`) to surface
the inner message; handles top-level `{"message":...}` and nested
`{"error":{"message":...}}`.

## Notes
- Surfacing a top-level `message` now applies to all providers
(intentional; nested wins when both present).
- The advisor path keeps the transport label; accepted as-is (it returns
`err.Error()`, not the classification).
- Fixes display in chatd only; the wrapper originates in aibridge (out
of scope here).

🤖 Generated by Coder Agents.
2026-06-16 11:14:37 +01:00
Hugo Dutka 4debd23cbb fix: chatd refactor (#26270)
Implements the chatd stabilization RFC.

Combines:
- https://github.com/coder/coder/pull/25908
- https://github.com/coder/coder/pull/25923
- https://github.com/coder/coder/pull/26109
- https://github.com/coder/coder/pull/26110
- https://github.com/coder/coder/pull/26111
- https://github.com/coder/coder/pull/26112
2026-06-12 13:33:12 +02:00
Ethan 3ab1323bc9 fix!: rename chat stream silence timeout error (#25973)
Renames the Agents chat stream-silence error from `startup_timeout` to
`stream_silence_timeout` now that the timeout applies to any gap between
provider stream parts, not just first-token startup.

Updates the SDK enum, generated API docs/types, chat error copy, and
Agents UI stories/status labels so the user-facing wording describes a
stalled provider response instead of startup delay.

> **Breaking change:** This is a very minor breaking change for the
Coder Agents API: the public chat error kind enum no longer includes
`startup_timeout`, so clients matching that specific value should handle
`stream_silence_timeout` instead.
2026-06-04 18:36:02 +10:00
Ethan becc858fa8 fix(coderd/x/chatd): retry provider stream cancellations (#26010)
Closes CODAGT-541.

## Problem

An Agents chat stream could die with a terminal `context cancelled`
error and surface to the user as a permanent chat failure, even when no
context in our process had actually been canceled. The cancellation was
a provider-returned error value (HTTP/2 RST_STREAM mid-body surfacing as
`context.Canceled` from Go's net/http2), not a real caller cancel.

The chain that produced the bug:

- fantasy passed the provider's `context.Canceled` through unchanged.
- `chaterror.Classify` short-circuited any `errors.Is(err,
context.Canceled)` (or `"context canceled"` text) as terminal generic,
before checking HTTP status codes or other retry signals.
- `chatretry.Retry` did not retry.
- The frontend rendered `type:"error"` and the chat was dead.

The same short-circuit also masked retryable 5xx responses whose
underlying transport error happened to wrap `context.Canceled`.

## Approach

`context.Canceled` has no inherent intent. The same error value can mean
a user pressing Stop, a server shutdown, the silence guard firing, or a
provider-side stream reset. The only layer that can disambiguate is the
one holding both the returned error and the caller context. That is
`chatretry`.

This PR centralizes the policy there and keeps `chaterror` context-free.

## Changes

`coderd/x/chatd/chaterror/classify.go`

- Add `ErrProviderTransportReset` sentinel to explicitly mark
provider-side stream cancellations.
- Remove the broad `context.Canceled` / `"context canceled"`
short-circuit so status codes and other retry signals can win.
- Classify `ErrProviderTransportReset` (with no status code) as a
retryable timeout.
- Keep a fallback that classifies bare `context.Canceled` as
terminal-generic when no other signal is present, so legitimate caller
cancels still terminate cleanly.

`coderd/x/chatd/chatretry/chatretry.go`

- Add `contextError(ctx)` that returns `context.Cause(ctx)` when set,
falling back to `ctx.Err()`, so caller-owned cancel causes
(`ErrInterrupted`, `errStreamSilenceTimeout`, server shutdown sentinels)
propagate cleanly out of the retry loop.
- Add `classifyProviderAttemptError(err)` that wraps a bare
`context.Canceled` in `ErrProviderTransportReset` and reclassifies.
Errors that already classify as retryable or carry a status code are
left alone.
- Restructure `Retry` so the policy is explicit and readable: check
caller cancellation before attempting, run the attempt, check caller
cancellation again before normalizing the provider error, then classify
and retry.

## End-to-end behavior

- Provider returns `context.Canceled` while caller context is healthy:
classified as a retryable timeout, retried, the user sees a brief
`type:"retry"` event and the chat continues.
- User presses Stop: `contextError(ctx)` returns `ErrInterrupted`. Retry
stops. `chatloop` flushes partial content and persists.
- Stream-silence guard fires: `attemptCtx` is canceled with
`errStreamSilenceTimeout`, `guardedStream` produces a classified
retryable error, retry proceeds normally on the still-alive parent.
- Server shutdown: parent context's cause propagates out, retry stops.
2026-06-04 12:52:37 +10:00
Mathias Fredriksson 8b7e040105 fix(coderd/x/chatd/chatloop): discourage doctrine in compaction summaries (#25850)
Two additions to the compaction summary prompt:

1. Error specificity: the "errors encountered" bullet now instructs the
   model to keep error notes specific (name the file, the error, the
   fix) and not generalize from a specific failure to a blanket
   tool-avoidance rule. This addresses the doctrine crystallization
   pattern where a single tool failure gets promoted to a standing
   "avoid tool X" rule that persists across compactions and model swaps.

2. Reproducibility: a new closing sentence instructs the model to
   reference reproducible content by path, command, or URL rather than
   inlining it. Content without a stable reproducer is still preserved
   inline with a brief summary. This targets summary bloat from
   inlined code blocks (worst case: 34k chars, 76 code blocks
   reproducing repo content verbatim).

Refs CODAGT-331
2026-06-01 12:42:09 +03:00
Cian Johnston 6df1536256 fix: add missing_key error kind for missing chat api_key_id (#25783)
Refs CODAGT-486

- `codersdk/chats.go`: New `ChatErrorKindMissingKey` constant and
`AllChatErrorKinds` entry
- `coderd/x/chatd/chaterror/message.go`: `terminalMessage` and
`retryMessage` cases
- `coderd/x/chatd/model_routing_aibridge.go`: Pre-classify error with
`WithClassification`
- `coderd/x/chatd/model_routing_internal_test.go`: Classification
assertion on production path (CRF-2)
- `chatStatusHelpers.ts`: Frontend title "Chat interrupted"
- `LiveStreamTail.stories.tsx`: Storybook story with `detail` assertion
- `docs/ai-coder/ai-gateway/clients/coder-agents.md`: Troubleshooting
entry
- Tests: classification round-trip, terminal message, metrics kind
enumeration

> Generated with [Coder Agents](https://coder.com/agents) on behalf of
@johnstcn
2026-05-28 15:50:52 +01:00
Ethan 7e2f7198dd fix(coderd/x/chatd/chatloop): use stream silence timeout (#25782)
Replaces the 60 second first-token timeout in the chat loop with a 10
minute stream-silence timeout.

Previously, the guard bounded only the gap before the first stream part.
Once any part arrived the attempt could hang indefinitely if the
provider stopped streaming without closing the connection, and even
normal long-running responses could be killed after 60 seconds if the
provider was slow to emit the first token.

The guard now arms when a model attempt opens its stream, resets on
every received stream part, and fires after 10 minutes of complete
silence. The existing retry path still handles the timeout, and the
public `startup_timeout` error kind is preserved to avoid API and
frontend churn.

10 minutes matches the default request timeout used by the Anthropic and
OpenAI Python SDKs.


Closes CODAGT-493
2026-05-28 21:02:40 +10:00
Ethan c650aabbef chore: standardize on *_internal_test.go for white-box tests (#25601)
My agent added `//nolint:testpackage` to a test file on one of my PRs.
Again. This PR cleans it up across the entire repo and updates the
in-repo conventions so future agents stop doing it.

The repo already has a precedent for white-box tests that need to touch
unexported symbols: `*_internal_test.go` (145+ existing files). The
`testpackage` linter's default `skip-regexp` exempts that filename
suffix, so the `//nolint:testpackage` directive is unnecessary in every
case where someone reached for it. This PR renames 51 such files to
`*_internal_test.go` via `git mv` so blame and history follow, and
strips the dead directive from 2 files that were already correctly named
(`coderd/oauth2provider/authorize_internal_test.go`,
`coderd/x/chatd/advisor_internal_test.go`).

`.claude/docs/TESTING.md` now documents the rule explicitly under *Test
Package Naming*, which is imported into the root `AGENTS.md` via
`@.claude/docs/TESTING.md`. The rule: prefer `package foo_test`; if you
need internal access, rename the file to `*_internal_test.go` rather
than adding a nolint directive.
2026-05-22 20:24:38 +10:00
Ethan 1e8c8d7dba fix(coderd/x/chatd): drop orphan provider tool calls on replay (#25491)
Anthropic replay can fail when stored history contains a
provider-executed tool call like `web_search` without the matching
provider-executed result. That orphaned call is incomplete
provider-internal state, so replaying it can make an otherwise usable
chat unreplayable even though there is no search result to preserve.

This fixes replay by dropping orphan provider-executed tool calls from
the model-visible prompt, preserving signed reasoning and the rest of
the assistant content, then revalidating before the request. We do not
synthesize tool results or drop reasoning. The database can retain the
historical artifact for inspection, while Anthropic only sees replayable
content.

This matches permissively licensed prior art. Vercel AI SDK
(Apache-2.0), used by mux, keeps incomplete tool state in UI/history but
omits it from model requests with `convertToModelMessages(..., {
ignoreIncompleteToolCalls: true })`. LangChain, LiteLLM, and OpenAI
Agents (MIT for the relevant open-source code) also preserve Anthropic
signed reasoning as opaque replay data. Coder applies that model-visible
replay boundary explicitly because our persisted history is already in
provider-message form.

This matches mux, is cleaner than the older idea around not persisting
the search query tool, and the model handles the repaired prompt fine.

Closes CODAGT-448

## Before

<img width="963" height="491" alt="image"
src="https://github.com/user-attachments/assets/a7788ebf-2728-4420-90cf-5e4f6905bdf7"
/>

## After

<img width="842" height="513" alt="image"
src="https://github.com/user-attachments/assets/ae39c262-7586-4e2d-b7db-1b639a7e8e15"
/>
2026-05-20 01:28:02 +10:00
Kyle CarberryandCoder Agent 385146000b feat: record created_at/completed_at on reasoning ChatMessageParts (#24789)
Records reasoning start and end times on persisted reasoning
`ChatMessagePart`s so reasoning duration can be computed for stored
chats. Backend-only: no SSE changes and no frontend rendering ship in
this PR.

The `created_at` field on `ChatMessagePart` is extended to also be
present on `reasoning` parts (it previously appeared only on `tool-call`
and `tool-result`), and a new `completed_at` field is added for
`reasoning` parts.

### How timestamps are recorded

- `StreamPartTypeReasoningStart`: stamp `startedAt = dbtime.Now()` on
the active reasoning state.
- `StreamPartTypeReasoningEnd`: stamp `completedAt = dbtime.Now()` and
append both into parallel `[]time.Time` slices on `stepResult`.
- Persistence reads the slices in occurrence order (reasoning has no
provider-side ID) and applies them to the matching `ChatMessagePart` via
`buildAssistantPartsForPersist`. The first reasoning block's stamps go
onto the first reasoning part, and so on.
- `flushActiveState` flushes partial reasoning interrupted before
`StreamPartTypeReasoningEnd` with `startedAt` from the active state and
`completedAt = dbtime.Now()` at the interruption.

### Why two fields, not one?

Tool calls and results are point events. The frontend computes their
duration by subtracting the call's `created_at` from the result's
`created_at`. Reasoning is one assistant part that brackets a span, so
we record both endpoints on the part itself.

### Why not stamp in `PartFromContent`?

Same rationale as #24101: `PartFromContent` is called during both SSE
publishing and persistence. Stamping there would yield incorrect
persistence-time timestamps for reasoning blocks that finished much
earlier in the step. Instead we capture in the chatloop and apply during
persistence.

<details><summary>Implementation plan</summary>

- `codersdk/chats.go`: extend `CreatedAt`'s `variants` to include
`reasoning?`; add `CompletedAt *time.Time` with `variants:"reasoning?"`.
- `coderd/x/chatd/chatloop/chatloop.go`: extend `reasoningState` with
`startedAt`; extend `stepResult` and `PersistedStep` with parallel
`[]time.Time` reasoning slices; stamp on
`ReasoningStart`/`ReasoningEnd`; thread the slices through all
`PersistStep` call sites including the interrupt-safe path; record
partial reasoning in `flushActiveState`.
- `coderd/x/chatd/attachments.go`: walk reasoning parts in occurrence
order and apply `step.ReasoningStartedAt[i]` to `part.CreatedAt` and
`step.ReasoningCompletedAt[i]` to `part.CompletedAt`.

### Tests

- `codersdk/chats_test.go` round-trips `created_at` + `completed_at` on
reasoning parts and verifies omission when absent and partial
interrupted parts.
- `coderd/x/chatd/chatprompt/chatprompt_test.go` asserts
`PartFromContent(ReasoningContent{})` does NOT stamp timestamps.
- `coderd/x/chatd/chatloop/chatloop_test.go`
`TestRun_ReasoningTimestamps` drives a stream with two reasoning blocks
and verifies parallel slices, monotonicity, ordering, non-zero values,
and content-block ordering.
`TestRun_InterruptedReasoningFlushesTimestamps` cancels mid-reasoning
and verifies `flushActiveState` records a non-zero pair.
- `coderd/x/chatd/attachments_test.go` covers
`buildAssistantPartsForPersist` for normal interleaved reasoning,
partial (zero `completed_at`), and missing slices.

</details>

> Generated by Coder Agents.

Co-authored-by: Coder Agent <agent@coder.com>
2026-05-18 12:30:30 -04:00
Ethan e75bd3aca4 fix: preserve Anthropic replay fidelity (#25377)
Anthropic is strict about replaying the latest assistant turn once it
contains signed or redacted reasoning. We were still mutating that turn
in a few Coder-owned places: dropping empty reasoning blocks on replay,
rewriting provider-tool history during sanitization, and in the worst
case sending a prompt we already knew Anthropic would reject.

This patch keeps the latest signed assistant immutable through Coder's
replay and sanitization paths, preserves empty signed or redacted
reasoning anywhere Coder owns the ledger, and fails before the provider
call if the prompt is still unsafe.

It also bumps the existing `coder/fantasy` `coder_2_33` fork that `main`
already uses to the commit containing coder/fantasy#35. These fixes have
also been upstreamed to charmbracelet/fantasy.

Closes CODAGT-409.
2026-05-18 15:20:33 +10:00
Kyle Carberry 376fc80451 fix(coderd/x/chatd): discover workspace MCP tools mid-turn after create_workspace (#25169)
## Problem

In `coderd/x/chatd/chatd.go` `runChat`, workspace MCP discovery is gated
on `chat.WorkspaceID.Valid` at the start of each turn. New chats that
bind their workspace mid-turn (via `create_workspace` or
`start_workspace`) get an empty workspace tool list on the first step,
and the model falls back to `execute` (bash) because no workspace MCP
tools are advertised.

**Repro:** new chat → "create a workspace and use MCP tools". No
`/api/v0/mcp/tools` request hits the agent on turn 1; turn 2 in the same
chat works fine.

## Fix

- Add a `PrepareTools` callback to `chatloop.RunOptions`, analogous to
`PrepareMessages`. It is invoked once before each LLM step with the
current tool list. When it returns non-nil, the chatloop replaces
`opts.Tools`, rebuilds the per-step tool definitions, and appends new
tool names to `opts.ActiveTools` so newly injected tools are callable
immediately.
- Wire `PrepareTools` in `runChat` to trigger workspace MCP discovery
the first time the chat snapshot reports a valid `WorkspaceID`. The
previous top-of-turn discovery path is unchanged for chats that start
with a workspace.
- Extract the discovery logic into `Server.discoverWorkspaceMCPTools` so
the top-of-turn and mid-turn paths share identical behavior (cache,
agent resolution, `ListMCPTools` timeout, invalidation).

Mid-turn discovery stays disabled in plan-mode turns and Explore
subagents, matching the existing top-of-turn gate. The
`workspaceMCPDiscovered` flag prevents redundant dials after the first
successful discovery.

## Tests

- `coderd/x/chatd/chatloop/chatloop_test.go`: two new
`TestRun_PrepareTools*` cases covering injection on the next step and
active-set merging when `ActiveTools` is non-empty.
- `coderd/x/chatd/chatd_test.go`:
`TestRunChat_WorkspaceMCPDiscoveryAfterMidTurnCreateWorkspace` drives
`runChat` through a `create_workspace` tool call against a real Postgres
+ mocked agent conn and asserts the second streamed LLM request
advertises the workspace MCP tool. Verified that the test fails (and
pinpoints the missing tool) when the `PrepareTools` wiring is disabled.

## Validation

```
go test ./coderd/x/chatd/chatloop/... -count=1
go test ./coderd/x/chatd/... -count=1
make lint/emdash
```

<details>
<summary>Decision log</summary>

- Chose a per-step `PrepareTools` callback over mutating `opts.Tools` in
place because `chatloop.Run` builds the `fantasy.Tool` definitions once
at start; a hook is required to let the LLM see new tools on the next
step.
- Returned `[]fantasy.AgentTool` (not also active-tool-names) and let
the chatloop derive name merges via `mergeNewToolNames`. This avoids
leaking plan-mode gating decisions into the callback contract.
- Kept the existing top-of-turn discovery path so chats that already
have a workspace at turn start pay no extra latency.
- Skipped reusing `ReloadMessages` (history reload) since this is purely
a tool-availability concern; coupling it to a history reload would
defeat the chatloop cache prefix optimizations.

</details>

---

_This pull request was generated by Coder Agents._
2026-05-12 00:30:56 -04:00
Cian Johnston e8508b2d90 fix: recover chatd from poisoned chain anchor on retry (#25097)
When OpenAI's Responses API returns `Previous response with id ... not
found` for a chained turn, classify it as a `ChainBroken` retry, clear
`previous_response_id`, exit chain mode, reload full history, and let
`chatretry` retry. Self-heals chats whose anchor was poisoned before
#25074 stopped truncated streams from being persisted as a successful
turn with a stored response id.

The new state is exposed via the existing
`coderd_chatd_stream_retries_total` counter as a
`chain_broken="true"|"false"` label. Aggregating queries (`sum`, `rate`
over `provider`/`model`/`kind`) keep working without changes; raw-series
matchers without aggregation will now see two series per `(provider,
model, kind)` where they previously saw one. The metric is internal-only
so the blast radius should be small, but if you have dashboards that
index by exact label matchers without aggregation they will need an
extra `sum` or an explicit `chain_broken` selector.

> 🤖 This PR was created with the help of Coder Agents, and was reviewed by a human 🧑‍💻
2026-05-11 17:43:40 +01:00