mirror of
https://github.com/coder/coder.git
synced 2026-11-22 11:28:40 +08:00
90948802e2439d2641f392e6229dd058a1dcafe3
80
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c28ae0dc9b |
feat: trigger compaction independently for the chat and compaction models (#28568)
> [!NOTE]
> Xum (an AI agent) authored this pull request on Mike's behalf.
## Summary
Automatic compaction now fires at whichever of two independent triggers
is reached first, and the UI shows when the organization override's
trigger applies.
- **Chat trigger**: the chat owner's personal threshold (or the chat
model's) against the chat model's window. A threshold of 100 turns off
only this trigger.
- **Organization trigger**: a fixed 80% of the organization compaction
override model's window. The override model's own compression threshold
is ignored. The other 20% is a target margin for the summary prompt and
the summary, not a guarantee, because the trigger reads the previous
step's prompt size.
On main, the chat threshold was applied to the smaller of the two
windows, and nothing in the UI showed it. Example: with a 20K chat model
at 70% and an 8K override, main compacts at 5,600 tokens; this PR
compacts at 6,400 (80% of 8K, shown as 32%).
## Changes
Backend (`coderd/x/chatd`):
- The trigger with the lower token point binds (ties go to the chat
trigger). It drives the automatic compaction gate and the
still-over-limit check, and every summary, automatic or manual, records
it.
- An override that is unset, deleted, disabled, on a disabled provider,
or without credentials for the chat owner is treated as absent: the chat
model summarizes and only the chat trigger applies.
- The summary's output cap is bounded by the summarizer's own window,
not the trigger's.
- The still-over-limit error names what can clear it: the compaction
threshold (chat trigger), an override model with a larger window
(organization trigger), or both when usage is over both points.
- The still-over-limit check compacts once more when the latest summary
was recorded under a different threshold or trigger window. The
`chat_summarized` result gains `trigger_context_limit_tokens`, and
`context_limit_tokens` now records the chat model's window (main
recorded the smaller window). On main, lowering a threshold after a
compaction ended every later turn with the error.
- The still-over-limit check skips a 0% threshold ("always compact"),
which on main ended the run with the error after every compaction.
- `ARCHITECTURE.md` documents the two triggers, the summary cap, and the
still-over-limit rules.
Frontend (`site`), through a shared `compactionTriggers` helper that
mirrors the backend rules (no new API):
- Coder Agents settings: the compaction override row warns when the
organization trigger undercuts chat models' default triggers ("the
organization override compacts chats at 80% of {model}'s {N}-token
window").
- Personal thresholds page: an Effective column shows the trigger that
applies, with a popover when the organization override binds. "Compacts
at" follows the same trigger, and a point the chat model's window cannot
reach shows as Off. The header's 100% sentence, and the row warning when
a member types 100, say an organization override may still compact chats
when one is set or its organization's overrides failed to load (the
header also while they load).
- Chat context gauge: the compact-at percent follows the binding trigger
and is annotated when the organization override binds. If the overrides
request fails, it shows the chat-only value with "(organization override
not loaded)", or "Compaction off (organization override not loaded)"
when the chat trigger is off.
- An override whose provider the user cannot use is ignored on the gauge
and the personal page, matching chatd.
- The shared `formatContextLimit` prints counts below 1,000 exactly (an
800-token compaction point showed "~1K") and 999,500 to 999,999 as
"1.0M" instead of "1000K". It also labels the model pickers; their other
values are unchanged.
- `HelpPopoverIconTrigger` lets callers set their own `aria-label`; only
the Effective column's popover does.
Known v1 limits: the Coder Agents settings alert is organization-scoped,
so it can warn about an override that some members fall back from for
lack of provider credentials. The gauge checks the viewer's provider
access, so on a shared chat its label can differ from the owner's
runtime trigger (Codex P2, declined). Compaction itself is unaffected.
## Screenshots
From remote UAT round 13 on
|
||
|
|
bb9f31d94b |
fix: apply Anthropic prompt caching through OpenRouter and OpenAI-compatible gateways (#29563)
Anthropic models configured through an `openrouter` provider (and, by
the same code path, `vercel` and generic `openai-compat` providers)
never got prompt caching. Two independent gaps: chatd only emitted
`cache_control` breakpoints when the fantasy client was `anthropic`, and
even when a client sent them, the AI Gateway chat-completions
interceptor rebuilt the upstream body from typed
`openai.ChatCompletionNewParams` and silently dropped the unknown key.
Dogfood UAT on main @
|
||
|
|
fc0af7fbd8 |
feat: open a chat_turn span per prompt (#29771)
**TL;DR:** Wraps each prompt in a `chat_turn` root span with every lifecycle stage parented under it. The span records one outcome per turn (`completed`, `interrupted`, `error`, `abandoned`), and the new `coderd_chatd_turn_outcomes_total` counter counts those outcomes. Turns stay correct when tasks are canceled, replaced, or retried. Each prompt runs inside a `chat_turn` root span owned by the runner (`runnerTurnSpan`). The generation step, prepare, MCP connect, stream, TTFT, provider attempt, commit, thinking, tool call, compaction, and retry backoff stages are parented under it, activating the stages from #29770. `chat_turn` spans are standalone roots: the turn runs on a worker, often on another replica, and no trace context is persisted. **Lifecycle** - **Start.** Each generation loop iteration calls `Ensure`, which opens the turn if none is open and returns a `turnToken`. The span is backdated to the trigger: the latest of the last user prompt, a dynamic tool's result message after it, and a pending compaction request. An `acquisition` stage covers trigger to pickup. - **Stale trigger / takeover.** A trigger at or before the previous anchor opens the turn at now with no `acquisition` and counts `stale_anchor`. A runner that took the chat over from an owner with a stale heartbeat (`TakenOver`, from acquisition) opens its first turn at now with no `acquisition`, so the dead owner's run is not reported as pickup delay. - **Newer prompt.** `Ensure` on the current turn with a newer trigger closes it and opens a new one. The closed turn is `abandoned` unless the task that ran it records an outcome before releasing it. - **Finish.** After `FinishTurn` or `EnterRequiresAction` commits, the step calls `Complete`; `Settle` closes the span after the step's own stage ends, so the finishing step is inside the turn. A chat waiting in `requires_action` is outside any turn; the generation resumed by its tool results opens a new one. - **Emission.** Closing a turn fixes its end time; its span is emitted once every task that joined it (`Ensure` takes the task ID) has called `Release`, which `runTask` does after the task returns. A finishing task whose `FinishTurn` promoted a queued prompt therefore still counts its turn as `completed` even when the promoted task's `Ensure` runs before its `Complete`. `End` at runner exit emits whatever is left. - **Failure.** A step that commits `FinishError`, or fails closed, invalidates the turn as `error` after the commit. Errors the task runner retries, including task timeouts, keep the same turn open; an expected, non-retryable exit records no outcome and leaves the turn open until the interrupt task, a newer prompt, or runner exit closes it. A canceled generation task records no outcome and cannot obtain a turn token, so a replaced task never touches its successor's turn. - **Interrupt.** The interrupt task marks the open turn `interrupted` before `FinishInterruption` commits and closes it after. Only a stop (by the user or a parent agent) is `interrupted`; it ends with unset span status, since error status is reserved for `error` turns. runner shutdown, loss of ownership, and message edits close the turn as `abandoned` with unset status. - **Promotion.** Every promoting transition (`FinishTurn`, `FinishInterruption`, a send to an errored chat, `PromoteQueued`) returns the promoted message's queued time in its result, and one `recordQueueWait` helper records a standalone `queue_wait` to the promotion. The promoted turn opens at the promotion time. `chat_turn` is observed on `coderd_chatd_stage_duration_seconds` only for completed turns, so the histogram is completed-turn latency; every turn still emits its span. `coderd_chatd_turn_outcomes_total{outcome, chat_kind}` counts every closed turn exactly once (`completed`, `interrupted`, `error`, `abandoned`), matching the span's `turn_outcome`, under the `chat-stage-metrics` experiment. Turn time accounting is in draft #29772. **Stages and identity** - `tool_call` is a live stage around each local tool call in `chatloop`; the tool runs on its context, and it ends errored only when the tool fails to execute. Calls that never run (inactive, unknown, or rejected by policy) record no stage. - `mcp_connect` carries connected and failed server counts and ends errored only when servers were configured and none connected. - The compaction stage carries the model that runs the summary. - `withStageIdentity` puts scope and chat kind on the turn context. Detached background work runs in `background` scope as its own trace roots. - The `advisor` tool's nested model call appears only as its enclosing `tool_call`. ARCHITECTURE.md gains a "Lifecycle tracing" section. <details> <summary>Review decisions</summary> - Turn identity stays timestamp-based with targeted fixes (clamped acquisition, rotation on a newer trigger, invalidate after commit, fail-closed invalidation, settle closing invalidated turns) plus a takeover flag; keying turns on message IDs would not detect takeover and cannot express interrupted/abandoned outcomes. - Outcome classification: the interrupt task owns `interrupted`, and cancellation or a fence mismatch alone never records an outcome, so shutdown, ownership loss, and edits are `abandoned`. Only `error` turns set error span status. Remaining ordering races between a step and an interrupt are documented as known limitations in ARCHITECTURE.md. - Turns are emitted after the tasks that ran them return, not when they close, so an outcome a replaced task records while unwinding reaches its own turn; the successor task is not delayed. Chosen over an after-commit, pre-publish hook on `ChatMachine.Update`, which would fix only the `FinishTurn` ordering. - `requires_action` ends the turn rather than pausing it, so `chat_turn` measures chatd time; a prompt with client tools counts as several turns. - Promoted turns anchor at promotion time and `queue_wait` is always standalone, so `chat_turn` never includes time spent queued behind another turn. - The former `capacity_wait` stage (#29805) was dropped: it was a per-worker lower bound, excluded from accounting, and needed its own SQL and bookkeeping. #29805's remaining helpers are folded into this PR. </details> Supersedes #28972, #28973, and #29805. 🤖 This PR was generated by Coder Agents on behalf of @jscottmiller. |
||
|
|
e988ddb11d |
feat: trace chatd provider stages and count only output parts as first token (#29770)
**TL;DR:** Traces the model provider call: a `stream` stage, a nested `time_to_first_token` stage, and one `provider_attempt` per HTTP round trip, labelled with provider type and model. Behavior change: `coderd_chatd_ttft_seconds` now counts only a real output part as the first token, so error, warning, and finish parts no longer count. Instruments the provider call. **Stream and time to first token.** `GenerateAssistant` records a `stream` stage from stream open to close and a nested `time_to_first_token` stage. The TTFT window closes once, on the first streamed output part: any part other than warnings, finish, or error, including start markers such as `text_start`. A closed window observes the stage and the pre-existing `coderd_chatd_ttft_seconds` with the same elapsed time. A failed open, an error part, or a release without an output part ends the TTFT span with the error (a silence timeout is classified as such) and observes nothing. The `stream` span ends once, with the error `GenerateAssistant` returns, so content-filter and no-output attempts end errored. Failed `stream` windows are observed like successful ones. The attempt is released before the stream span ends, so the TTFT window always falls inside its stream. The compaction summary stream passes a nil tracer, so it records neither. Without #29771, production emits only `provider_attempt`; #29771 passes the tracer to `GenerateAssistant`. **Provider attempts.** The provider HTTP transport is wrapped in `stageSpanRoundTripper`, which emits one `provider_attempt` stage per round trip, so retries each get a span. The span closes on response headers; a status >= 400 marks it errored without changing what the client sees. `resolveModelCall` resolves the `StageModel` (wire provider, configured `provider_type`, model, effective reasoning effort) and threads it to the transport, so span attributes and labels match the call. `provider_type` is read from `ai_providers.type`, so types outside the fantasy set (copilot) keep their configured name. `chatd.New` builds the tracer with the server clock. **Behavior change to `coderd_chatd_ttft_seconds`:** only an output part counts as a first token. Error parts no longer close the window as a success, and warnings and finish parts are skipped; the help text says so. `StageSpan.End` returns the span's elapsed window, which `ttft_seconds` observes. Callers pass `Stages` and `StageModel` through `GenerateAssistantOptions`; a nil tracer discards (TTFT still records `ttft_seconds`). `Server` gains the `stages` tracer and a `TracerProvider` option, wired from `coderd`. <details> <summary>Review decisions</summary> - Start markers close the TTFT window: they are the first sign the model is producing output, and waiting for a delta would fold hidden reasoning time into TTFT. - Failed `stream` and `provider_attempt` windows are observed: that time was spent. TTFT is the exception because a first token that never arrived has no duration. - `http_host` was dropped from `provider_attempt`; every request goes through the local AI Gateway transport, so it was constant. - The advisor tool's nested model call is intentionally not instrumented as `stream`/TTFT. </details> Supersedes #28970 and #28971. 🤖 This PR was generated by Coder Agents on behalf of @jscottmiller. |
||
|
|
7a9442014e |
feat: add chatd lifecycle stage tracer (#29769)
**TL;DR:** Adds a stage tracer that turns each chat lifecycle stage into
both an OpenTelemetry span and a Prometheus histogram observation, so
traces and metrics always agree. The metrics are only registered with
the `chat-stage-metrics` experiment. Nothing calls the tracer until the
next PRs.
Adds `chatloop.StageTracer`, which emits each chat lifecycle stage as an
OpenTelemetry span and a `coderd_chatd_stage_duration_seconds{stage,
scope, chat_kind}` observation from one `End` call, so trace and metric
durations cannot disagree. The provider-bound stages (`stream`,
`provider_attempt`) are also observed on
`coderd_chatd_model_stage_duration_seconds{stage, provider_type,
chat_kind, model}`, the only family carrying `provider_type` and
`model`, and only for turn-scoped stages. Time to first token per model
stays on the existing `coderd_chatd_ttft_seconds`. Nothing calls the
tracer yet; the next PRs instrument the provider call and the runner.
- The families are registered only when the `chat-stage-metrics`
experiment is enabled (safe and known, so
`--experiments=chat-stage-metrics` or `--experiments=*` turns it on).
Spans are emitted regardless. `NewMetrics` leaves the families off;
`chatd.New` opts in from the experiment, and a test asserts the
experiment decides whether they reach the server's registry.
- `Stage`, `Scope`, `ChatKind`, and `StageAnomaly` are typed string
constants. `scope` separates latency attributable to a prompt (`turn`)
from detached `background` work; `chat_kind` is `root` or `subagent`,
empty without a known chat.
- `StageModel` carries the wire `provider`, `provider_type`, `model`,
and `reasoning_effort`. All are span attributes; only `provider_type`
and `model` are metric labels, on the model family. No family carries an
organization label.
- Observed stages: `chat_turn`, `queue_wait`, `acquisition`,
`mcp_connect`, `stream`, `time_to_first_token`, `provider_attempt`,
`tool_call`, `commit`, `retry_backoff`. `generation_step`, `prepare`,
`thinking`, and `compaction` are span-only. A test keeps the stage sets
and help text consistent.
- API: `Start` / `StartRootAt` for live spans (span timestamps come from
the tracer clock), `Record` for stages reconstructed from timestamps
(the scope comes from the context). `End` is idempotent;
`EndWithoutObservation` ends the span without a sample. A nil tracer,
provider, or metrics discards.
- `coderd_chatd_stage_anomalies_total{reason}` counts observations
dropped, adjusted, or inconsistent (`negative_elapsed`,
`inverted_window`, `missing_timestamp`, `future_start`, `stale_anchor`).
Span-only stages never feed it.
- Buckets are 12 round edges (`0.1 0.25 0.5 1 2.5 5 10 30 60 300 1800
3600`).
- Moves the body of the `for` loop in `StartGeneration` into
`runGenerationStep` so the step has a function boundary for a deferred
stage end. Control flow and error wrapping are unchanged.
<details>
<summary>Series cost (approximate, whole stack)</summary>
A histogram label set is 12 edges + `+Inf` + `_sum` + `_count`: 15
series. Per replica, each `chat_kind` costs about 150 stage series and 4
turn outcome counters; each `(chat_kind, model)` costs 30 (2 stages x
15). Both chat kinds on `M` models: about `308 + 60*M` per replica, plus
background-scope stage series.
Dogfood `main`, 3 replicas: ~3.4k `coderd_*` series per replica (~10.1k
total).
| Models used per replica | Added series (3 replicas) | vs. baseline |
|---:|---:|---:|
| 1 | ~1.1k | +11% |
| 5 | ~1.8k | +18% |
| 10 | ~2.7k | +27% |
| 35 | ~7.2k | +72% |
Most deployments run four or fewer models, so a typical replica adds
roughly 370 to 550 series. The experimental turn accounting in #29772
would add 88 more per replica (11 categories x 4 outcomes x 2 chat
kinds). Series appear lazily and do not grow with organizations.
</details>
<details>
<summary>Review decisions</summary>
- Experiment gate only; no deployment flag or metric levels. If the
experiment graduates, the families become always on like the other chatd
families; a flag is added then only if dogfood shows a cardinality
problem.
- Organization is on neither metrics nor spans, and `effort` is
span-only; `provider_type` (configured type), not the wire `provider`,
is the model-family label.
- `time_to_first_token` is not on the model family; per-model TTFT is
`ttft_seconds`, which observes the same measured window.
- The model family records only turn-scoped stages.
- Nil guards are kept so a nil or zero tracer is always safe to call.
- HTTP attribute keys stay snake_case like the rest of the chatd span
attributes.
</details>
Supersedes #28968 and #28969.
🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
|
||
|
|
fa5bac6e77 |
feat: send tool call IDs and cancel unresolved calls on interrupt (#29954)
Retries and replica takeovers ran tool calls again, and an interrupt recorded the same generic result whether a call had finished, was still running, or never started. chatd now derives a tool call ID from the chat, the committed assistant message, and the provider tool call ID, and sends it with execute starts, edit_files, and write_file. Every replica and retry computes the same ID, so the agent returns the first result instead of running the tool again, and a retried foreground execute waits only for the rest of its timeout. On interrupt, chatd cancels the unresolved calls of those tools on the agent before committing, within 30 seconds, and records what happened: "tool call was canceled before it ran; no changes were made", the output of a command canceled while running, or the real result. The UI shows a canceled command with the existing killed marker. If the cancel fails, including an old agent's 404, the result stays the generic interrupted text. Calls whose provider tool call ID is empty or repeats in the message get no ID and behave as before. Fixes CODAGT-757 |
||
|
|
5a997c471a |
fix(coderd/x/chatd): tell the model when output hit the token limit mid tool call (#30120)
When an Anthropic response hits the output token limit while the model
is writing a tool call's input, chatd gets `FinishReasonLength` with a
cut-off call. It handled both shapes of that call badly:
- If the provider closed the `tool_use` block, the call carried
truncated JSON and got the generic malformed-input result, which invites
the model to resend the same oversized input and hit the limit again.
- If the block never closed, fantasy emitted no tool call, chatloop
discarded the dangling tool input, and the turn ended with only
reasoning and no explanation.
On a length finish, `GenerateAssistant` now gives each cut-off local
call an error result: "This tool call was not executed because the
response reached the output token limit (N tokens) while writing its
input. Split the content into smaller tool calls." The token count is
included when the call has a configured output limit. A tool input that
started streaming but never became a call first gets a stand-in call
with empty input. The call persists with nil args and replays as `{}`,
so the provider receives a valid `tool_use` and `tool_result` pair, and
generation continues with another step.
With lifecycle hooks enabled, `pre_tool_use` admission skips calls the
step already resolved, so the hook is not dispatched for them and no
second result is added. The duplicate tool ID check still runs on the
full batch, because a resolved call still occupies its ID.
Independent of #30119, which stops the AI Gateway from failing these
streams before chatd sees them.
## Testing
-
`TestGenerateAssistant_SilentNoOutputFinish/LengthFinishResolvesTruncatedToolCalls`
(truncated tool call, dangling tool input): no error, reasoning kept,
and each call paired with the output-limit result.
- `TestPreToolUseHookOutputLimitTruncatedToolCallResolvedOnce` with
hooks enabled, for truncated and unfinished input: no hook dispatch, one
stored error result per call, a second model request, and a replayed
request that pairs `tool_use` input `{}` with its `tool_result`.
`DuplicateIDFailsClosed` checks that a reused ID still fails the chat
with `duplicate tool use ID`.
- Red-green: with `chatloop.go` and `generation.go` restored to the base
code, all four truncation cases fail. `DuplicateIDFailsClosed` passes on
the base code by design and fails on `d02bf4aa89b`, which dropped
resolved calls before the duplicate check.
- `go test ./coderd/x/chatd/... -count=1` passes, and lint on the
touched packages reports no issues. Local pre-push checks were skipped
per standing preference; PR CI covers them.
- Remote UAT: not run yet.
## Review record
- Parent review (Xum) of `d02bf4aa89b`: skipping resolved calls inside
`chathooks.PendingToolCalls` hid them from the duplicate tool ID check
in `admitStepToolCalls`. Fixed in `745f722165b`.
- Codex: not requested yet.
> Xum, an AI coding agent, prepared this PR on behalf of @ibetitsmike.
|
||
|
|
fd5b37576a |
feat(coderd): record provider response IDs on chat messages (#29763)
Chat messages could only be linked to AI Gateway records per chat, so nothing recorded which model response produced a given message. fantasy surfaces the response ID the model endpoint returns on the stream's finish part: the Anthropic Messages ID as the part ID, and the OpenAI Responses ID in provider metadata. chatd now stores that ID in the existing `chat_messages.provider_response_id` column for assistant steps and the compaction summary row. No migration and no aibridge changes. What the stored value is depends on the route: | Route | Stored value | Joins to | |---|---|---| | Anthropic Messages API through the AI Gateway (Anthropic and Bedrock Claude) | The gateway's interception ID. The gateway replaces the message ID because injected MCP tool loops can fold several upstream messages into one response. | `aibridge_interceptions.id` | | OpenAI Responses API through the AI Gateway | The upstream `resp_` ID, passed through unchanged | `provider_response_id` on `aibridge_token_usages`, `aibridge_user_prompts`, and `aibridge_tool_usages` | | Direct to the provider | The provider's own ID (`msg_`, `resp_`) | Nothing in the gateway tables | To resolve messages to interceptions, then read usage by `interception_id`: ```sql SELECT cm.id AS chat_message_id, i.id AS interception_id FROM chat_messages cm JOIN aibridge_interceptions i ON i.id::text = cm.provider_response_id UNION SELECT cm.id, tu.interception_id FROM chat_messages cm JOIN aibridge_token_usages tu ON tu.provider_response_id = cm.provider_response_id; ``` Not covered, and left NULL: - Partial rows saved on interrupt, because fantasy only reports the ID in the finish part. - Chat Completions providers (OpenAI-compatible, OpenRouter, Vercel), because fantasy does not surface the chunk ID there. - User, tool, and system rows, and all existing messages. <details> <summary>Delivery record</summary> - `96a896c` replaced the earlier interception-link design (`59fa58f`, `7f93c18`, `9d04ccd`) in place: the aibridge hook, the `aibridge_interception_id` column and migration, the message part buffer tracking, and the `ARCHITECTURE.md` section are all removed. The net diff against the base touches only `coderd/database` and `coderd/x/chatd`. - Local validation on `96a896c`: pre-commit hook green (gen, fmt, lint, build); `go test ./coderd/x/chatd/... ./coderd/database/` green. Red-green: removing the finish-part capture fails both new end-to-end tests (`TestActiveServer_PersistsAnthropicMessageID`, `TestOpenAIResponsesPersistsProviderResponseID`). CI on `96a896c`: green. - Remote UAT round 2 on `96a896c` ([chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)): **FAIL** against this description's earlier claim that every stored ID joins the gateway usage rows directly. Anthropic rows routed through the gateway stored the interception ID, so 0 of 18 joined usage rows directly, while every checked Anthropic row equaled `aibridge_interceptions.id`. Both OpenAI Responses rows joined directly. Interrupted partial rows, Chat Completions rows, and user and tool rows were NULL, with no empty strings or duplicate IDs. Multi-step turns, manual compaction, edit and regenerate, a provider switch, and a retried 429 behaved as expected. One unattributed anomaly: a single duplicate user row after interrupt and continue, which did not reproduce and is outside this diff. - `2e889ba` corrects the doc comment on `PersistedStep.ProviderResponseID` and this description to match; behavior is unchanged. Pre-commit hook and CI green. - Remote UAT round 3 on `2e889ba` against the corrected contract ([same chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)): **PASS**. The query above resolved all 40 stamped rows to exactly one interception from the same chat, with none unresolved. Reading usage by interception recovered every usage row for 38 of them; the last two resolved but their usage rows were not counted. Fresh Anthropic and Responses calls, a seven-step tool turn, manual compaction, an Anthropic to Responses to Anthropic switch in one chat, and a UI reload behaved as described, and the NULL cases held. Three interrupt and continue attempts did not reproduce the round-2 duplicate, so it stays unattributed. Not tested: gateway-injected MCP tool loops (unreachable from chatd, whose delegated requests skip the gateway's MCP proxy), direct Anthropic `msg_` IDs, Bedrock, and threshold-triggered compaction. Both rounds had a verified isolation proof and a clean dogfood audit. - Codex rounds 1 to 3 reviewed the old design. Codex round 4 on `2e889ba` ([request](https://github.com/coder/coder/pull/29763#issuecomment-5796193371), 13:58Z): no major issues ([result](https://github.com/coder/coder/pull/29763#issuecomment-5796267231), reviewed commit `2e889ba05c`). No unresolved review threads. Review passes so far: 4 of 6. - Final head `2e889ba` is the exact commit remote UAT round 3 and Codex round 4 covered; required checks green. </details> > Xum updated this PR on behalf of @ibetitsmike. |
||
|
|
e35cbe2a7f |
fix(coderd/x/chatd): stream compaction summaries (#28886)
Chats could get permanently stuck once their context crossed the compaction threshold on adaptive-thinking Claude models: every compaction attempt failed with `compaction produced an empty summary`, and both automatic and manual compaction take the same path, so the chat could never recover. ## Problem `compactionSummaryCall` deliberately set `MaxOutputTokens = nil` because the summary request was non-streaming and the Anthropic SDK rejects non-streaming requests whose `max_tokens` implies a completion longer than ten minutes. But a nil cap makes fantasy's Anthropic provider silently default to 4096 output tokens, and adaptive-thinking Claude models can spend that entire budget on a thinking block, leaving zero summary text. ## Fix - Stream the compaction summary request. Streaming lifts the Anthropic SDK's non-streaming duration restriction, which was the only reason the cap was nil'd. - Keep the resolved output cap (32k default) on the summary call instead of nil'ing it. - When the summary comes back empty with truncation evidence (length finish reason, or reasoning with no text), fail with `compaction summary was truncated at the output token cap` instead of the generic empty-summary error, so the next occurrence is attributable at a glance. Unit tests cover text-block joining and the truncation classification; the compaction e2e fixtures now serve streamed summaries and assert the summary request streams with `max_tokens: 32000`. Dogfood UAT on a deployment built from this branch verified manual compaction with Claude Opus 5 (adaptive thinking, twice, with reload persistence) and an OpenAI regression chat. > 🤖 Opened by [Xum](https://mux.coder.com) on behalf of @ibetitsmike. |
||
|
|
19e2b0415a |
fix(coderd/x/chatd): preserve tool definitions during compaction (#29436)
Supersedes #28883 by @pfrybar. His commit is carried with authorship intact (rebased onto main as |
||
|
|
77479408b4 |
feat: make the Coder Agents stream silence timeout configurable (#29394)
The `chatloop` stream silence guard cancels and retries a model attempt after `10m` of silence, and it was hard-coded. It's armed before the request opens, so it also bounds time to first token, and slow local models (e.g. `llama-server` on a large prompt) can spend longer than that on prompt processing alone, meaning every attempt just gets cancelled and retried. This PR adds `--chat-stream-silence-timeout` / `CODER_CHAT_STREAM_SILENCE_TIMEOUT` (default `10m`) and passes it to both the main loop and the advisor. Setting it to `0` disables the guard entirely, and anything above `24h` is rejected, which also keeps the watchdog arithmetic below from overflowing. Whilst doing this I ran into the runner's `15m` task watchdog, which kept counting during an open stream. That silently cancelled any healthy stream longer than `15m`, and would've stopped a silence timeout above `15m` from ever firing. To fix, the silence guard now kicks the watchdog out to `max(15m, silence + 5m)` whenever it arms or sees a part. Silent streams still fail through the guard, healthy streams can run for as long as they like, and hang detection is unchanged everywhere else. With the guard disabled there's nothing to kick against, so arming it just stops the watchdog for the rest of the attempt. That does mean the DB write after the stream isn't bounded either when `0` is set; I think that's fine for an opt-in setting, and a user can still stop the chat from the UI if it wedges there. I did consider removing the guard entirely, but the provider HTTP clients have no timeout, so a provider that holds the connection open would only ever be caught as a generic task hang, which is what #25782 was fixing. |
||
|
|
26e4429362 |
test: fold near-identical test bodies into tables across eight packages (#29463)
Folds groups of near-identical test bodies (same parent, same parallel flag, differing only in inputs and expected values) into table-driven tests in eight packages: `coderd/templatebuilder`, `scripts/develop`, `agent/agentfiles`, `coderd/x/chatd/chatloop`, `coderd/x/chatd/chatdebug`, `enterprise/aibridgeproxyd`, `cli`, and `coderd/database/db2sdk`. One commit per package. Every case keeps its previous subtest name, `t.Parallel()` placement is unchanged, and the set of `=== RUN` names is identical except where standalone `TestX_Y` funcs became `TestX/Y` subtests (chatloop: 9 funcs into `TestExecuteLocalTools_BatchRuntime`; chatdebug: 15 funcs into two tables). Members whose bodies differ in more than data stayed as they were (listed per package in the commit messages). In `agent/agentfiles` the four `TestFuzzyReplace_*` funcs were already tables, so the shared request plumbing was hoisted into one helper instead of merging the funcs and losing their per-rule docs. Net -1035 lines across 9 test files; runtime is unchanged (all sub-100ms unit tests). | Package | Diff | Funcs removed | Subtests | RUN lines | |---|---|---|---|---| | coderd/templatebuilder | +237 / -291 | 0 | 31 -> 31 | 33 -> 33 | | scripts/develop | +194 / -246 | 0 | 27 -> 27 | 28 -> 28 | | agent/agentfiles | +117 / -260 | 0 | unchanged | 58 -> 58 | | coderd/x/chatd/chatloop | +224 / -306 | 9 | 3 -> 12 | 14 -> 15 | | coderd/x/chatd/chatdebug | +251 / -537 | 13 | 0 -> 15 | 42 -> 44 | | enterprise/aibridgeproxyd | +287 / -419 | 0 | 31 -> 31 | 32 -> 32 | | cli | +143 / -326 | 0 | 15 -> 15 | 16 -> 16 | | coderd/database/db2sdk | +130 / -233 | 0 | 9 -> 9 | 10 -> 10 | ## Validation - Per package: `go test -count=1 -tags=testsmallbatch -parallel=8 ./<pkg>/ -run <regex> -v` before and after, with the `=== RUN` name sets diffed - `go test -race -count=2 -shuffle=on` on each converted test: pass - `golangci-lint run ./<pkg>/...` with a fresh cache: 0 issues in all eight packages - Repository pre-commit hook (including lint/go) passed on every commit Part of the test-suite speedup series (see #29457); independent of the other PRs. ## Remote validation Remote run on a dogfood workspace (separate compute), chat https://dogfood.cdr.dev/agents/5bbe73d0-866a-4841-9cc3-c5e5b4d97e44, tested head `d7c55d6b0e8` against base `9d973372774`: all eight packages pass on the head and under `-race -count=3 -shuffle=on`; package times are sub-second on both sides (unchanged). `=== RUN` name sets: chatloop 15 to 16 lines with all nine `TestExecuteLocalTools_<X>` leaves surviving as `TestExecuteLocalTools_BatchRuntime/<X>`; chatdebug 42 to 44; `aibridgeproxyd` `TestNew` 32 to 32, byte-identical sorted list. One correction to the text above: seven chatdebug leaves are renamed, not preserved verbatim (`TestRecordingTransport_*UnknownLength*` became `TestRecordingTransport_UnknownLengthClose/<name without "UnknownLength">`), so `-run` filters keyed on those old names need updating. Endorsed verdict: PASS. > Xum (an AI agent) authored this PR on behalf of @ibetitsmike. |
||
|
|
8397680687 |
fix: keep text and image together in MCP tool results (#29332)
## Problem MCP tool results that mix text and an image (Playwright's default `browser_take_screenshot` returns `[text, image]`) were reduced to text only in both MCP conversion paths, so screenshots never reached the model. The generic tool renderer also showed media results as a JSON dump of base64. ## Fix - `mcpclient` and `chattool`: keep all text plus the first usable binary item. Error flags survive empty and MIME-less results. Only one binary payload is represented; additional images are still dropped. - `chatloop`: at execution time, cap tool media at the 10 MiB attachment limit and correct or reject declared image types by byte signature (sniffing, not decoding). - `chatd` prompt build and `chatprovider`: at generation and compaction, replace media the destination provider cannot accept with the text plus an omission note (Anthropic/Bedrock: JPEG, PNG, GIF, WebP under the inline cap; OpenAI/Azure and Google: their image allowlists, applying the configured vendor's rules behind `openai-compat`). Persisted history is unchanged, so the UI keeps the payload and provider switches keep working. - `site`: results flagged `is_media` render through `ToolResultMedia` (image with the existing lightbox, or a short notice for non-image media, plus the text). Unflagged media-shaped JSON stays in the JSON viewer. - `go.mod`: pin [coder/fantasy `9a3598480a71`](https://github.com/coder/fantasy/commit/9a3598480a71e45901bfcfbf566ffa42c6bbdd63), main's `a511353f83eb` plus the batch-order fix `460d4fdd113b` (Chat Completions and compatible transports emit every tool result before synthetic user images). The fix lives on `mike/mcp-tool-media-batch-order`; no upstream PR yet. ## Validation Round 3 dogfood UAT on this head, `522bd76a58a`: **PASS** ([chat](https://dogfood.cdr.dev/agents/5e617636-c6f2-46fc-96e5-97ae35ed822e), real Playwright MCP, Claude Haiku 4.5 and GPT-4o mini through the AI gateway). Verified in the UI and on the wire: image plus text render together, the lightbox opens and closes, both survive a reload; audio shows a notice instead of base64; media-shaped JSON stays in the JSON viewer; BMP on OpenAI Responses and Chat Completions reaches the model as text plus an omission note; SVG and PNG histories switch from Anthropic to OpenAI; cross-provider compaction through the organization override works; a result over 10 MiB persists only text plus a note, and a 5.5 MiB PNG is substituted for Anthropic yet still renders in the UI. Google is not routable on the dogfood gateway (404), so it was not exercised. The remote agent's own FAIL was a probe artifact (the code viewer renders in a shadow root) and was overturned on its screenshots by the independent runner. Rounds 1 and 2 (`65db84a`, `5001de7`) covered the same API surface earlier. Known limits: signature sniffing does not catch a truncated PNG with a valid header (providers return 400 for it), and only the first binary item of a result is kept. Pre-existing and out of scope: chatd opens a new MCP session per generation action, so Playwright's default per-session browser loses page state between steps (UAT used `--cdp-endpoint`). Reviews: | Review | Head | Outcome | |---|---|---| | Codex 1 to 5 | `472914f` to `bcf4c72` | 16 findings (empty binary promoted over text, no provider caps, SVG on Anthropic, provider-switch gap, Bedrock labels, unbounded media, unvalidated bytes, renderer gating, lightbox test), all fixed | | Codex 6, 7 | `8a9a9e0`, `5001de7` | no major issues; one Responses audio concern [refuted with a live probe](https://github.com/coder/coder/pull/29332#discussion_r4018514197) | | [Codex](https://github.com/coder/coder/pull/29332#pullrequestreview-5226106293) | `83e1a10` | two P2s refuted with serializer captures ([MIME](https://github.com/coder/coder/pull/29332#discussion_r4029188332), [advisor](https://github.com/coder/coder/pull/29332#discussion_r4029160781)); [reassessment](https://github.com/coder/coder/pull/29332#issuecomment-5702295886) clean | | Codex | `522bd76` | [no major issues](https://github.com/coder/coder/pull/29332#issuecomment-5713663081) on the rewritten tests | | @DanielleMaywood | `472914f`, `5001de7` | renderer restructure and explicit audio mapping done; lazy slice cloning removed; two threads left open for her re-check | Local pre-push hooks were skipped per Mike's instruction; CI is the gate. > Xum acted on Mike's (@ibetitsmike) behalf to create this PR. |
||
|
|
0362e5386c |
fix(coderd/x/chatd/chatloop): tell the model to gather context before retrying an exclusive tool (#29281)
When the model batches `advisor` (an exclusive tool) with other tool calls, every call in the batch is rejected and the `advisor` result only says to retry with `advisor` alone. That nudges the model toward re-calling `advisor` immediately, even when the sibling calls were there to gather context for it. Extend the message so the model is told to run the other tools first when it still needs their output, then retry with only the exclusive tool call. Generated by Coder Agents on behalf of @ibetitsmike. |
||
|
|
46c904d181 |
fix: show estimated context usage after compaction (#28984)
After compaction, the context gauge showed the unused-chat placeholder until the next response. It now shows a clearly labeled estimate based on the compacted summary, keeps that estimate after reload, and uses the selected chat model’s context limit. The next response replaces it with measured usage. The estimate stays separate from provider-reported token accounting. Remote dogfood UAT passed the core workflow on cb18cbdac596231acf821cd612c5a8c5d1db1c7b. It has not been repeated on the current head, |
||
|
|
680b2f59cd |
refactor: remove obsolete chatd code and tests (#29228)
Chatd still carries an unreachable interruption flush path, unused state and options, an uncalled workspace MCP cache invalidation callback, and a stream-buffer-drop counter that production never increments. Remove these paths and their dedicated assertions, and update the generated metrics inventory and documentation. Consolidate duplicate publication and rollback tests and move fixture writers and test-only inspection helpers into test files. Retain the interruption persistence, state-machine matrix, legacy message format, and provider compatibility coverage. |
||
|
|
c12355d6e5 |
fix(coderd/x/chatd): carry unanswered user messages through compaction verbatim (#28645)
Fixes [CODAGT-737](https://linear.app/codercom/issue/CODAGT-737/compaction-can-treat-the-last-user-message-as-an-assistant-message) ## Problem Auto-compaction can trigger on a turn whose last history message is a fresh, unanswered **user** message. Two things go wrong: 1. The summary instruction is appended as another user message. Providers that merge consecutive same-role messages (verified against fantasy's Anthropic conversion) fuse it with the user's message, and the summarizer misattributes the user's instruction as assistant intent. 2. After compaction the original user row falls outside the prompt window, so the instruction survives only through summary fidelity. ## Fix Carry unanswered trailing user messages through compaction verbatim instead of summarizing them. All logic lives in `chatd`; `chatloop` is unchanged apart from prompt wording. - **Prepare** (`generation_preparer.go`): `pendingUserSegmentStart` locates the trailing user rows on the persisted rows, before provider-switch sanitization (which can drop the assistant row separating them from an already-answered message). Head and tail are converted separately so the summarizer input excludes the tail exactly. Falls back to summarizing everything when no usable segment exists. - **Commit** (`buildCompactionMessages`): replays the tail after the boundary trio as uncompressed, model-only user rows. The existing prompt window query includes them; no SQL, UI, or turn-guard changes. Replay rows carry no usage, so compaction cannot immediately re-fire. - **Prompt** (`chatloop/compaction.go`): the summary prefix no longer claims the assistant "was actively working", which is false in this case. The summarizer prompt is unchanged. - `ARCHITECTURE.md` compaction section updated. ## Testing `go test ./coderd/x/chatd/...` green. New: `TestBuildCompactionMessages_ReplaysPendingUserMessages`, `TestPendingUserSegmentStart`, `TestPendingUserSegmentConversionEquivalence`, `TestPendingUserSegmentDerivedBeforeProviderSwitchSanitization`, `TestDecisionGeneratesAfterCompactionWithReplayedPendingUser`. <details> <summary>Root-cause repro</summary> Replaying the reported scenario through fantasy's Anthropic provider against a capture server: ``` === fantasy prompt (roles as chatd builds them) === [2] role=user "You'll hit compaction now, we'll need to go through some fin..." [3] role=user "You are performing a context compaction. Summarize the conve..." === wire request to Anthropic === [2] role=user, 2 content block(s) block[0]: "You'll hit compaction now, we'll need to go through some final checks..." block[1]: "You are performing a context compaction. Summarize the conversation so..." ``` </details> --- 🤖 This PR was generated by Coder Agents on behalf of @f0ssel. |
||
|
|
ac824cd235 |
chore(coderd): remove dead code from chatd (#29008)
Removes production code in `coderd/x/chatd` that nothing reaches, the tests that only existed to exercise it, and option/config fields that callers filled in but chatd never read. No runtime behaviour changes; the only edits outside `coderd/x/chatd` drop the now-nonexistent fields at their three call sites (`coderd/coderd.go`, `coderd/exp_chats.go`, `coderd/coderdtest`). Net: 42 files, +147 / -1065. Verification method for every item: `golang.org/x/tools/cmd/deadcode` (with and without `-test`) rooted at `cmd/coder`, `enterprise/cmd/coder` and every package that imports chatd; a `go/packages` pass counting `TypesInfo.Uses` per declared type, const, var, field and method across the same roots, split into test and non-test use sites, plus a read-vs-write split for struct fields; `rg` over the whole repo including `site/src` for json tag names; then `go build ./...`, `go vet ./coderd/x/chatd/...`, `golangci-lint run ./coderd/x/chatd/... ./coderd/ ./coderd/coderdtest/`, and `go test ./coderd/x/chatd/...` (all 20 packages) locally. <details> <summary>Evidence per deletion</summary> | Removed | What it was | Why unused | |---|---|---| | `subagent.go` `resolveConfiguredModelOverride`, `parseModelOverride`, `parsedModelOverride` (~135 lines) + `TestResolveConfiguredModelOverride_*`, `TestParseModelOverride` | raw-string model override resolver | Only callers were its two tests; production resolves overrides through `resolveOrganizationModelOverride` (`title_override.go`, `compaction_override.go`, `subagent.go`). | | `chatd.go` `removeSkillIndexMessages`, `isSkillIndexMessage` + `TestSkillIndexRefreshReplacesStaleAliases` | skill-index refresh helpers | Zero production references; the test simulated a refresh path no production code performs. | | `chatd.go` `chatMessage`, `newChatMessage`, `withCreatedBy`, `appendMessageFields`, `BuildSingleChatMessageInsertParams` (~90 lines) | insert-params builder | Exported, but every caller was a `chatd_test` file (6 sites). Replaced by a 25-line `singleChatMessageInsertParams` helper in `chatd_helpers_test.go`. | | `chatd.go` `Server.closeInflightAdmission` | one-line wrapper around `inflightClosed.Store(true)` | Single call site (`Close`); inlined. | | `chatd.go` `Server.workerID`, `instructionLookupTimeout`, `pendingChatAcquireInterval`, `maxChatsPerAcquire`, `chatHeartbeatInterval` | Server struct fields | Assigned in `New` and never read anywhere (the values reach the worker through `chatWorkerOptions`, not these fields). | | `Config.InstructionLookupTimeout`, `homeInstructionLookupTimeout`, `coderd.Options.ChatdInstructionLookupTimeout`, `coderdtest.Options.ChatdInstructionLookupTimeout` | timeout plumbing | The value only landed in the dead `Server.instructionLookupTimeout`; no CLI flag sets it and nothing reads it. | | `PromoteQueuedOptions.CreatedBy` | option field | Set by `coderd/exp_chats.go` and three tests, never read in chatd (promotion keeps the queued row's `created_by`). | | `SubmitToolResultsOptions.DynamicTools` | option field | Set by `coderd/exp_chats.go` (which re-read `chat.DynamicTools` just to pass it) and three tests, never read in chatd. | | `attachments.go` `buildAssistantPartsForPersist` (whole file) | assistant part builder | Stale copy left by #26270; production uses `buildAssistantParts` in `message_conversion.go`. Its six tests were retargeted to the live function (now `message_conversion_internal_test.go`) and a dead-copy assertion in `subagent_internal_test.go` was dropped. | | `chatloop.RunOptions` (94 lines) | old loop options struct | Only two of 23 fields were ever populated, for one internal call to `prepareMessagesForRequest`, which now takes `model, logger` directly and no longer returns the ignored canonical slice. `GenerateAssistantOptions` / `ExecuteLocalToolsOptions` are the live types. | | `chatloop.ErrDynamicToolCall`, `ErrStopAfterTool`, `PendingToolCall`, `PersistedStep.PendingDynamicToolCalls` | dynamic-tool exit protocol | Zero references outside their declarations (one stale comment in `chatdebug/service.go` updated). | | `chatloop.CompactionOptions.Persist`, `.OnError` | callbacks | `Persist` had zero references; `OnError` was set only by two tests that asserted it was never invoked. | | `chatloop` `stepResult.shouldContinue` + its computation | flag | Written once, never read. | | `chatloop.StateWaiting` | metric label const | Zero references. | | `generation.go` `generationOutcome.InsertedMessages`, `runnerActionMessage` | outcome field | Built on every commit, never read. | | `chatstate` `ExecutionState.IsArchived`, `AllOwnershipStates`, `OwnershipState`/`StateU`/`StateO`, `TransitionAcquire` | state-machine leftovers | Zero references (the ownership consts became dead once `AllOwnershipStates` went). | | `chatstate` `ExecutionState.QueueNonEmpty` | predicate | Only referenced by the test asserting its own table; the sibling `IsRunnable` check in that test is kept. | | `chatdebug.Attempt.RetryClassification`, `.RetryDelayMs` | json fields | Never assigned anywhere, `omitempty`, and `retry_classification`/`retry_delay_ms` do not appear in `site/src` or `typesGenerated.ts`. | | `chatdebug.RunContext.Kind`, `StepContext.HistoryTipMessageID`, `RecorderOptions.Provider`/`.Model`, `stepHandle.opts`/`.usage`/`.err` | context/handle fields | Written (from `chatd.go`, `chatloop/compaction.go`, `modelcall.go`, `recorder.go`) and never read. The DB run row keeps its own `kind` column. | | `chatprovider.MergeProviderAPIKeys` + `TestMergeProviderAPIKeys_PreservesProviderRegions` | key merge | Superseded by `ResolveUserProviderKeys`, whose region test already covers the same behaviour. | | `chatprovider` `providerCreationError` | one-line `xerrors.Errorf` wrapper | Single call site; inlined with identical message. | | `chatretry.IsRetryable` | classifier wrapper | Zero references. | | `computeruse.IsTool` + `TestComputerUseTool` | one-line forwarder to `fantasyopenai.IsComputerUseTool` | No production caller; two test assertions now call the fantasy function directly, and the dropped test only asserted the library's own constructor. | | `chatprompt.ToolResultToPart` | one-line forwarder to `codersdk.ChatMessageToolResult` | Single call site; inlined. | | `subagent_catalog.go` `formatSubagentDefinitions` | forwarder with `nil` overrides | Single call site; inlined. | | `worker.go` `chatWorker.chatWorkerID` | getter | Only tests called it; they read `opts.WorkerID` directly now. | | `chattest.AnthropicErrorResponse`, `OpenAIServerErrorResponse`, `ChatMessageWithParts` (`messages.go` removed) | test fixtures | Zero references. | | `chattest` `AnthropicRequest.Options`, `OpenAIRequest.Options` | `json:",inline"` map fields | `encoding/json` has no inline tag (the TODO on each said so); never populated, never read. | </details> <details> <summary>Considered and kept</summary> - `chatstate.CreateChat` (test-only thin wrapper over `insertChat`, production uses `CreateChatWithID`): documented package API with 25 test call sites; not worth the churn. - `chatprompt.MarshalContent` / `MarshalToolResult`: test-only, but they deliberately produce legacy-format DB rows so the read path's legacy handling stays covered. - `chatstate.AllowedExecutionTransitionsFrom` / `AllowedInputStates` / `AllowedExecutionTransitionOutputs`, `chatdebug.StepFromContext`, `WithClock`, `WithStaleThreshold`, `WaitUntilIdleForTest`, `PublishBuffer.BufferedChannels`, `chattool.NewWorkspaceMCPTool`: test seams that tests actively use to reach otherwise unexported state. - `AgentCapacityLimiter` interface (one production implementation): tests inject fakes. - `applyHookResultMessages` / `appendHookResultMessages`: a named pair over one placement enum, both used. - Nil-receiver guards on `*Metrics`, `*Result`, `*streamSyncPoller`, `*runnerDebugTurn`: nil is a supported "disabled" value on those paths. - Tests flagged by heuristics as assertion-light (`TestNopMetrics_DoesNotPanic`, `TestStepHandleFinish_NilHandle`, `TestRetry_OnRetryNilDoesNotPanic`, the `chatstate` synthetic-cancellation tests that assert through helpers): they cover real nil-safety or delegate assertions to helpers; kept. </details> <details> <summary>Out-of-scope findings (not changed here)</summary> - `chatdebug.Service.FinalizeStale` has had no production caller since #26270 removed the background sweep loop; only its tests call it, and comments in `chatd_debug.go` and `database/querier.go` still describe the loop. Deleting it would erase the implementation a fix needs, so it is left for a decision. - `coderd_chatd_chats` (gauge) and `coderd_chatd_stream_buffer_dropped_total` (counter) are registered and documented in `docs/admin/integrations/prometheus.md` but never set or incremented in production; `Metrics.RecordStreamBufferDropped` is test-only. Removing them changes the exported metric set and the docs, so they are left alone. </details> ## Declined review findings None yet. > Xum (AI agent) opened this PR on behalf of @ibetitsmike. |
||
|
|
b428c8feb6 |
feat(coderd): bill completed local tool batches in agent runtime (#28360)
**Stack**: #28359 → **#28360 (this PR)** → #28361 → #28362 ## Summary Completed local tool batches now count toward Coder Agent runtime (`chat_messages.runtime_ms`, the source of truth for `hb_agent_runtime_v1`), excluding sub-agent orchestration tools to avoid double counting, and without multiplying runtime for parallel tool calls. Second of a four-PR stack splitting #28211. Stacked on #28359. Refs CODAGT-928 (https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime). ## Problem Agent runtime previously measured only model invocation wall clock (stream open to fully consumed). Time spent executing local tools between steps, including file operations, terminal commands, workspace provisioning, and MCP tools, was deliberately excluded, which undercounts the product definition of "actively processing a task". Naive inclusion has two hazards: a batch of parallel tool calls would bill N windows for one wall-clock wait, and `wait_agent` would re-bill child agents that already bill their own model and tool time. ## Fix Each completed local tool batch bills one window: the union of the billed tools' execution intervals, persisted as `runtime_ms` on a dedicated usage record appended after the batch's tool-result rows. The record is a tool-role message with `visibility='model'`, so it never reaches the API, SSE, or clients, and prompt replay drops it because its single internal `tool-batch-usage` part converts to no provider content. Its content carries an audit payload (`billed_ms`, `billed_calls`) so the billed window is inspectable after the fact; real tool-result rows never carry batch-level runtime. Concurrent calls all start at batch start, so 5 parallel 10s reads bill 10s, not 50s; serial calls (`SerialToolCalls`) count only from their own launch, so unbilled waits before them do not count. Sub-agent orchestration tools (`spawn_agent`, `wait_agent`, `message_agent`, `interrupt_agent`, `list_agents`, `list_subagent_models`, plus the deprecated `close_agent` alias) never extend the window: every chat, including children, applies the same rules to its own runtime, so a parent's `wait_agent` window would double count. A lone `wait_agent` bills 0 and appends no usage record; `execute` 10s in parallel with `wait_agent` 60s bills 10s. A test pins the unbilled set to the registered sub-agent tool catalog so they cannot drift. `GetTotalChatMessageRuntimeMsInRange` already sums `runtime_ms` role-agnostically, so the usage record is picked up with no schema, query, or cron changes. Client-executed dynamic tools, external agents, parked/idle time, and retry backoff remain unbilled. Interrupted batches still bill only the model window; the rest of the stack adds partial-window billing via the same usage record. `ExecuteLocalToolsOptions` also gains an optional `ToolBillingRecorder` (`RecordStart`/`RecordComplete`) observing per-occurrence dispatch-order execution stamps. This PR wires no recorder; the next PR in the stack connects it to the message part buffer for interrupt billing. **NOTE**: Reported agent runtime (`hb_agent_runtime_v1`) increases from deploy forward. There is no backfill and no feature flag. |
||
|
|
d44aae37de |
refactor(coderd/x/chatd): return PersistedStep directly from ExecuteLocalTools (#28359)
**Stack**: **#28359 (this PR)** → #28360 → #28361 → #28362 `chatloop.ToolExecutionOutcome` only wrapped a `PersistedStep`, so callers had to reach through `outcome.Step` for every field. `ExecuteLocalTools` now returns the `PersistedStep` directly and the wrapper type is deleted. No behavior change. First of a four-PR stack that makes local tool execution count toward Coder Agent runtime (splitting #28211). This refactor keeps the rename noise out of the feature PRs. Refs CODAGT-928 (https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime). |
||
|
|
a48aedd924 |
fix(coderd/x): surface Gemini malformed-function-call stream deaths as retryable errors (#28470)
## Problem
When Gemini's OpenAI-compatible endpoint rejects a model-generated
function call server-side, it ends the SSE stream cleanly with the
nonstandard finish reason `function_call_filter:
MALFORMED_FUNCTION_CALL` after streaming only thought summaries; the
rejected call never reaches the wire. chatd treated this as a normal
completion: fantasy maps the unrecognized finish reason to `unknown`,
the reasoning block never closes (the only non-thought delta is the
`</thought>` marker, which the transport seam strips to an empty string,
and the openaicompat hook only ends reasoning on a non-empty content
delta), so the step accumulates no content and the generation loop
finishes the turn as complete. The user sees the model think for ~40
seconds and then nothing: no assistant message, no `last_error`, chat
status `waiting`. Observed twice in a row in production on
gemini-3.7-flash, with "Resume" reproducing it identically.
## Fix
Two independent layers:
- `coderd/x/googleopenai`: the stream rewrite now converts any chunk
whose `finish_reason` starts with `function_call_filter` into an
OpenAI-style SSE `{"error": ...}` event embedding the raw reason.
openai-go turns error-bearing events into stream errors, so the failure
rides the existing stream-error path instead of ending the stream
cleanly.
- `coderd/x/chatd/chaterror`: classifies that injected error as
retryable (kind `generic`, provider `google`) with a clear user-facing
message, so the existing generation retry machinery re-runs the step and
persists a `last_error` if retries exhaust.
- `coderd/x/chatd/chatloop`: provider-agnostic guard: a step that
produced no user-visible content and no tool calls under a finish reason
of `unknown`, `error`, `other`, or `tool-calls` (a tool-calls finish
that delivered zero calls) now returns a retryable error instead of
silently completing the turn. `stop` and `length` finishes keep their
existing semantics.
Tests cover the seam rewrite (live-capture SSE shape plus standard
finish reason passthrough), the new classification, and the chatloop
guard (error cases plus preserved stop, length, text, and tool-call
behavior). Each layer was red-green verified independently.
Remote dogfood UAT ran against this exact commit: normal reasoning and
tool-call chats on a real model complete cleanly with no spurious guard
errors and no retry loops. The Google-side failure itself is not
deterministically triggerable against live Gemini and is owned by the
unit tests.
> Xum acted on Mike's (@ibetitsmike) behalf.
<!-- xum-attribution: model=claude-opus, thinking=enabled -->
|
||
|
|
9168dbf4d3 |
refactor(coderd/x/chatd): unify LLM model-call construction behind a resolver (#28145)
Centralizes every LLM client build in chatd behind one resolver and makes all model calls behave the same, instead of preserving per-flow quirks. ## Why `chatd` built LLM clients in five places (standard turn, compaction override, quickgen, title override, advisor), each repeating config parsing, route resolution, client construction, and debug wrapping with slightly different behavior. An earlier iteration of this PR centralized construction but preserved every historical quirk behind purpose-specific spec constructors and policy flags. Review concluded the quirks themselves were the problem. ## What changed - `Server.resolveModelCall` (`modelcall.go`) is the single pipeline from a `modelCallSpec` to a ready client plus call metadata. Callers build spec literals inline; the only spec fields are data: model source (chat config, explicit config row, or fixed provider/model pair for computer use), requested reasoning effort, chatd-scoped routing, and build options. - Uniform behavior for every call: - Provider options are always derived. Whole-chat summaries, turn status labels, and chat-model compaction summaries previously sent none; they now follow the config like everything else. - `MaxOutputTokens` always defaults to 32k when the config leaves it unset. The compaction summary call nulls the cap after copying the template because it is non-streaming and the Anthropic SDK rejects non-streaming requests whose cap implies a completion over ten minutes. - Debug recording always wraps the client when chat debug is enabled, labeled with the resolved provider/model. The debug-rebuild paths (`manualTitleDebugSpec`, `quickgenDebugSpec`) are deleted; debug helpers now only create and finalize run records, since run association flows through the generate context, not the model. - Clients are always constructed with the configured model string and unmutated route, so AI Gateway validation sees the name exactly as configured. The resolved identity feeds metadata, logs, and debug labels only. - `resolvedModelCall.newCall` / `newObjectCall` are the only production constructors of `fantasy.Call` / `fantasy.ObjectCall`; object calls now carry the config's sampling parameters. - Net: -240 lines. ## Validation - Full `./coderd/x/chatd/...` suite green (PostgreSQL-backed). - Red-green toggles verified for the three key unifications (option derivation, token default, summary cap removal). - ARCHITECTURE.md: prose the refactor made stale is deleted; no new resolver documentation is added. > Mux acted on Mike's behalf for this PR. |
||
|
|
5480299cef |
fix: serialize empty MCP tool properties as {} not null for OpenAI (#28380)
## Summary
Fixed a bug where MCP tools with an empty `properties` schema had their
tool definitions sent to OpenAI with `"properties": null`, causing
OpenAI to reject the request.
## Problem
When an MCP server reports a tool with an empty input schema, e.g.:
```json
{
"type": "object",
"properties": {},
"required": []
}
```
Coder was forwarding the tool definition to OpenAI as:
```json
{
"type": "object",
"properties": null
}
```
OpenAI's function-calling API rejects `properties: null`, making these
tools unusable with OpenAI models.
## Root cause
The empty `properties` object collapsed to Go `nil` at two layers:
1. `agent/x/agentmcp/manager.go` `toolInputSchemaMap` stripped the
present-but-empty `"properties": {}` (guarded by `len(properties) > 0`),
so the wire copy lost it entirely.
2. `coderd/x/chatd/chatloop/chatloop.go` `buildToolDefinitions` embedded
`info.Parameters` directly into the input schema. When nil,
`encoding/json` marshals it as `null`.
## Fix
- `buildToolDefinitions` now substitutes an empty object when
`Parameters` is nil, mirroring the existing `required` guard (the sink,
protects all tool sources).
- `toolInputSchemaMap` now preserves an empty `properties` object
instead of dropping it (the upstream producer).
## Tests
Added regression tests (failing before, passing after) covering both
layers and asserting the schema serializes to
`{"type":"object","properties":{}}`:
- `TestBuildToolDefinitionsNilPropertiesBecomesEmptyObject`
- `TestToolInputSchemaMapPreservesEmptyProperties`
-
`TestWorkspaceMCPToolInfosFromResources/EmptyPropertiesYieldsEmptySchema`
---
*This PR was generated by Coder Agents.*
|
||
|
|
7724ee281a |
feat: defer MCP tool schemas behind a find_tools search (#28225)
## Summary When the `mcp-tool-search` experiment is enabled, chatd stops inlining connected MCP tool schemas into every generation. It instead exposes a built-in `find_tools` tool whose description carries a compact catalog of the deferred tools, and only ships full JSON schemas for tools the model has activated by searching or by calling them directly. Closes [CODAGT-760](https://linear.app/coder/issue/CODAGT-760). ## Problem Tool-heavy agent configurations (GitHub, Linear, Notion, and dev-tooling MCP servers) inline over 100k tokens of tool schema definitions into every generation. Initial uncached requests reached ~216k tokens with time-to-first-token close to nine minutes, while the model typically invokes only a handful of tools per turn. ## How it works - `decideMCPToolSearch` defers external and workspace `.mcp.json` MCP tools whenever the experiment is enabled. Native, dynamic, provider, skill, and transport tools are never deferred. - `find_tools` embeds a server-grouped catalog in its tool description (degrading to names-only, then counts-only, then a constant-size summary past a context-scaled size cap) and scores keyword matches across tool names, descriptions, parameter schemas, and server metadata. Queries can scope to one server with a `server:` prefix, and exact `names` arguments always activate. - Activation state is ephemeral: it is re-derived each generation from surviving chat history (`find_tools` results and direct calls to deferred tools), so activations naturally lapse when compaction summarizes them away. Aggregate activated schema weight is capped at 10% of the context window, shedding the least recently activated schemas first; `find_tools` shares that budget across parallel calls in one step. No new persistence. - Deferred tools stay registered for execution, so the model can call a cataloged tool directly without searching first; the schema is activated for subsequent steps. - Fail-open: the experiment being disabled, an empty candidate set, or an MCP tool named `find_tools` all disable deferral, leaving today's behavior byte-identical on the wire. - Prometheus counters/histograms track `find_tools` calls, matches, activations, and deferred token weight. - The conversation timeline renders `find_tools` calls with a collapsed search summary and expandable match list, falling back to the generic renderer on malformed payloads. ## Validation - Unit tests for the catalog, matcher, experiment-gated decision, and activation derivation; end-to-end chatd generation tests covering search-then-call, direct-call activation, experiment-off wire parity, compaction lapse, and subagent tool gating. - Storybook interaction tests for the timeline rendering and malformed-payload fallback. - Remote dogfood UAT on dev.coder.com passed: deferral with a real MCP server and Anthropic model, direct calls without prior search, activation persistence across turns, experiment-off parity, and clean UI/console. > Disclosure: Mux (AI agent) authored this PR on Mike's behalf. |
||
|
|
4b7494be72 |
feat: harden chat generation runtime instrumentation for billing (#27451)
Closes CODAGT-835 ## Summary `chat_messages.runtime_ms` becomes the billing source of truth for Coder Agents runtime (summed hourly by #27312), but it was built for debugging: the June refactor (#26270) silently stopped recording tool-step runtime, compaction was never measured, and interrupted turns lost their partial runtime entirely. This PR defines the billable metric, closes the paths that dropped it, and documents the definition where the data lives. ## The billable definition **`runtime_ms` is the wall-clock duration of the model invocation that produced the persisted message content**, measured from just before the provider stream opens until it is fully consumed. What counts: - Assistant generation steps, in top-level and sub-agent chats (sub-agents are ordinary chats on the same generation path). - Compaction summarization calls, persisted on the compaction assistant message (**new**). - Interrupted attempts: the message-part episode's lifetime is persisted on the partial assistant message committed by `FinishInterruption`, so partial generation time survives interruption (**new**; measured via a new `Buffer.EpisodeDuration`, which works even though the generation goroutine and the interrupt task are different tasks). What deliberately does not count (each is documented in code and docs): - **Local tool execution.** Tool wall time includes idle waits, most importantly `wait_agent` polling a sub-agent chat that already bills its own model invocations; billing the batch would double count, and excluding one tool from a concurrent batch's wall time is ill-defined. Pre-refactor instrumentation did include tool time; this makes the exclusion an explicit product definition instead of a silent regression. - **Failed model calls whose output is discarded** (retried attempts, terminal errors, content-filter refusals). They persist no content, so they bill nothing; billing errs toward undercounting. Notably a stream-silence timeout can burn 10 idle minutes before a retry, which should not be billable "active generation". If product later wants failed attempts billed, that needs a place to persist runtime on error turns (`FinishError` inserts no rows today) and is a deliberate follow-up, not instrumentation drift. - **Ancillary calls that produce no chat messages** (title generation, advisor, turn summaries) and all idle/parked time (`requires_action`, queueing). The definition is documented as `COMMENT ON COLUMN chat_messages.runtime_ms` (migration 000551, surfacing as a Go doc comment on `ChatMessage.RuntimeMs`), on `chatloop.PersistedStep.Runtime`, in the chatd architecture doc, and in the Spend Management docs page. ## Index for the hourly scan None needed: `GetTotalChatMessageRuntimeMsInRange` (#27312) filters an hour-wide `created_at` range, which the existing `idx_chat_messages_created_at` b-tree already serves; the residual `runtime_ms IS NOT NULL` filter applies to one hour of rows. A partial index would add permanent write amplification for a query that runs once an hour. > [!NOTE] > Migration 000551 is also claimed by #27312; whichever merges second renumbers via `fix_migration_numbers.sh`. ## Tests - End-to-end: the existing full-server generation test now asserts `RuntimeMs.Valid` on the committed assistant row (it previously read `.Int64` without checking `.Valid`, so it passed on NULL). - Interrupted turn: full task-level test (real DB, mock clock) asserting the partial assistant message persists the attempt's runtime. - Errored stream: asserts a failed invocation yields no step and no runtime. - Tool-using turn: asserts runtime lands on the assistant row only and tool rows stay NULL. - Compaction: asserts the summarization call duration is recorded and lands on the compaction assistant message only. - `messagepartbuffer.EpisodeDuration` unit coverage. Blocks: CODAGT-843 (B3), CODAGT-838 (D8). --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Hugo Dutka <hugo@coder.com> |
||
|
|
c17bed25e0 |
feat: wire chat lifecycle hooks into chatd (#27429)
Wires chat lifecycle hooks into chatd, gated by the `agent-lifecycle-hooks` experiment. Part of the lifecycle hooks stack (#27401, #27428, #27430). See `docs/admin/setup/chat-lifecycle-hooks.md` for the consumer-facing contract. ## Summary When a hook URL is configured, chatd dispatches `session_start`, `user_prompt_submit`, `pre_tool_use`, `post_tool_use`, `pre_compact`, `post_compact`, and `stop` events to the consumer and applies its responses. ## Design - **Stateless**: Coder stores no hook dispatch or decision state. Delivery is at least once; consumers deduplicate on stable payload identifiers (chat ID, event type, tool-use ID) and answer duplicates with the same decision. - **Admission-time prompt effects**: `user_prompt_submit` dispatches exactly once per submission (create, send, queue, edit, subagent spawn) and folds its effects into the stored prompt as typed message parts: original-or-overridden user parts, then model-only `hook-context`, then a user-visible `hook-notice`. Hook context is stripped from every client-facing conversion; hook notices are excluded from model prompts. The server rejects hook parts in client-submitted content. - **Tool gating**: `pre_tool_use` allow can override tool input; deny becomes a synthetic denied tool result, with any returned model context persisted as a model-only transcript row so it never reaches clients. The denial text identifies an external policy (the deployment's lifecycle hook) as the source and marks the decision as persistent, so the model explains the denial instead of retrying it or misreporting it as an infrastructure failure. - **Fail closed**: a dispatch failure rejects the triggering request or moves the chat to the error state in the same transaction as the affected step, so a runnable state is never published with unapproved content. - **Admission before persistence**: `pre_tool_use` is dispatched for the calls the model produced, before the assistant message is stored. See "Staged tool admission" below. - **Fresh dispatch per tool call**: every non-provider-executed tool call is decided by its own `pre_tool_use` dispatch; Coder never reuses an earlier decision on the consumer's behalf. Retries re-dispatch the same logical event. ## Structure All hook dispatch flows through one seam: entry points build a `chathooks.Chat` (chat identity) and a `chathooks.Message` (event details) and call `Trigger.Trigger`, the only component that talks to the dispatcher. The integration lives in the `coderd/x/chatd/chathooks` subpackage, split by responsibility: - `trigger.go`: the trigger seam; builds the wire envelope per event, normalizes deny into a typed error, and holds the package's single enabled-check. - `effects.go`: pure conversion of hook results into transcript rows and prompt parts. - `errors.go`: failure classification (dispatch error messages, denial mapping, tool-result dispatch-failure scanning). - `tooluse.go`: the tool-call gate (`pre_tool_use` preflight, `post_tool_use` payloads, applying admitted input to the step). Server-bound glue stays in `coderd/x/chatd/hook_server.go`: the chat-parking dispatch error handlers, the step-commit row insertion wrappers, and the dynamic post-tool-use state loader, which depends on chatd validation types. This PR adopts the `codersdk/x/agenthooks` and `coderd/x/agenthooks/dispatch` import paths introduced at the tip of #27401; intermediate commits still reference the pre-move paths and are not individually buildable. ## Staged tool admission `pre_tool_use` originally ran at tool execution time, which is after the assistant message carrying the tool call was already committed. An `input_override` therefore had to rewrite stored message content in place. @hugodutka pointed out that chatd treats message content as immutable, and that the rewrite was a shortcut rather than a requirement. It was also a correctness problem in its own right: the rewrite only updated the database, so the transcript could show one input while a different one had executed. The hook now runs before the step is persisted: ```text provider stream ends (tool calls complete, in memory) -> pre_tool_use dispatch per call -> ONE transaction: assistant row with admitted inputs, synthetic denials, hook rows -> execute ``` The step is inserted once, carrying the input the tool runs with. `UpdateChatMessageContentByID` and `Tx.UpdateMessageContent` are deleted from #27428, so message content stays immutable. Two consequences, both intentional: - **Clients converge rather than wait.** Tool-call parts still stream live, so a rewritten call briefly shows the model's proposed input before the committed message replaces it. The chat store already clears stream state when an assistant message arrives, so the stored input wins with no frontend change and no added latency before tool cards appear. - **A call already in history was already admitted.** Execution consumes the stored input instead of dispatching a second decision, which keeps one dispatch and one set of hook effects per call. A consumer policy change between admission and execution applies to later calls, not to calls already admitted. The per-chat debug endpoint still records the provider's original tool input. Its purpose is to report provider behavior, and it requires an explicit per-chat debug flag; the invariant here covers the transcript. ## Configuration Adds `chat-hook-url`, `chat-hook-secret`, `chat-hook-timeout`, and `chat-hook-enabled` deployment options with startup validation. The flags are hidden from `coder server --help` while the feature is experimental; the setup guide documents them. ## Tool input validation Built-in tool arguments reach a consumer as raw JSON with key spelling preserved, but the tools decode those bytes with Go, which matches struct fields case-insensitively and keeps the last match. A policy reading `path` could therefore authorize one value while the tool executed another, and a lone case variant such as `{"PATH":"/secret"}` was invisible to a policy checking for `path`. Coder now rejects a built-in tool call whose input repeats a key or spells a schema property with different capitalization, before the `pre_tool_use` dispatch, so a consumer is never asked to authorize bytes whose meaning depends on the reader. Rejected calls produce an error result the model can retry; unambiguous calls in the same batch still run. A consumer-authored `input_override` is rechecked after the dispatch and fails the turn closed, because the model cannot correct it. Dynamic and MCP inputs are excluded because the client and the workspace agent execute those calls rather than coderd. Two paths needed more than a schema check. Execution resolves a deprecated tool name to its canonical tool, so validation resolves aliases first. The `edit_files` decoder also reads `search` and `replace`, which its schema does not advertise, so those aliases are now matched exactly and their case variants ignored. A hook denial now returns a structured 403 carrying `kind: "hook_denied"`, mirroring the dispatch-failure response that already carries its own kind. Without it a client cannot tell a policy decision apart from a generic failure, and the chat UI titled a denial "Request failed". Adding a kind needs no migration: `ChatErrorKind` is persisted only inside the JSONB `chats.last_error` column, whose decoder accepts unknown kinds. The hook docs also correct the tool-input convergence window. A batch dispatches sequentially before the assistant row commits, so the original input stays visible for a span that scales with the number of tool calls in the step rather than a single hook timeout. > This PR was written by Mux, an AI coding agent, on Mike's behalf. |
||
|
|
e9951b07d4 |
fix(coderd/x/chatd/chatloop): surface reasoning-only content-filter refusals as terminal errors (#27476)
## Problem Anthropic can end a stream with `stop_reason: "refusal"` after reasoning content has already streamed. The content-filter guard in `chatloop.GenerateAssistant` only fired when the step content was completely empty, so a reasoning-only refusal bypassed it: the turn finished as `status=waiting` with `last_error=null`, and the user saw the chat silently stop mid-turn with no explanation. This looked like a Coder fault when the provider had rejected the response. Observed twice in dogfood on 2026-07-23 (chat `c72f99fc`, debug steps show `finish_reason=content-filter` with reasoning-only content). ## Change Treat a content-filter finish as terminal whenever the step produced no user-visible output. A new `hasUserVisibleContent` helper counts any non-reasoning part (text, tool call, tool result) as user-visible; reasoning-only or empty steps now return the existing `contentFilterError`, which flows through the established pipeline: classified `ChatErrorKindContentFilter` (non-retryable, refusal category/detail when provided), persisted `chats.last_error`, streamed error event, and the "Response blocked" callout in the chat UI. Behavior for steps with visible text or tool calls is unchanged, and the frontend needs no changes. ## Testing - New regression subtest `ReasoningOnlyContentSurfacesTerminalError` (reasoning stream then content-filter finish) beside the existing empty-content and partial-content subtests, which are unchanged. - `go test ./coderd/x/chatd/...` and lint pass. - Dogfood UAT against a local dev instance with a mock Anthropic upstream passed all three scenarios: reasoning-only refusal shows the "Response blocked" callout with `last_error.kind=content_filter` and no retry affordance; text-then-refusal still completes normally; empty refusal still errors. > This PR was created by Mux acting on Mike's behalf. |
||
|
|
3227cac217 |
feat: add manual chat compaction via /compact (#27081)
Adds a user-triggered `/compact` action for Coder Agents chats: typing
`/compact` in the composer (or picking it from the `/` trigger menu)
summarizes the conversation so far to free up context window space.
## How it works
- New `POST /api/experimental/chats/{chat}/compact` endpoint
(owner-only, RBAC `ActionUpdate`, excluded from the public API reference
via `x-apidocgen skip`). It marks the chat with a durable one-shot
`chats.compaction_requested_at` signal and moves it `waiting -> running`
via a new `RequestCompaction` state transition; no message row is
inserted. AI Gateway attribution needs no per-request key: generation
preparation resolves the owner's synthetic API key (#27170) like any
other turn.
- `RequestCompaction` hands off chat ownership (clears
`worker_id`/`runner_id`) so a worker acquisition hint is published;
since the transition changes no history, the previous runner could
otherwise miss the request under reordered pubsub delivery.
- The background chat worker picks the chat up like any other turn. A
pending manual request takes precedence over turn completion in the
generation decision, and forces compaction even below the automatic
threshold (and when compaction is disabled via threshold=100). The
commit step consumes the request marker in the same transaction; any
transition that ends the turn clears stale markers.
- The summary triplet reuses the automatic-compaction path, now tagged
with a `source` (`automatic` | `manual`) that is plumbed through
streamed progress parts, persisted tool JSON, and the UI label
("Summarized (manual)").
- Validation order: busy chats reject with 409 (state-machine conflict),
empty/already-compacted chats with 409 "nothing to compact", archived
chats with 400; the owner usage-limit check runs last so no-op requests
surface the specific conflict instead of a limit error.
- Web UI: the `/` trigger menu now has a built-in "Commands" group
listing `/compact`; submit intercepts exactly `/compact` and calls the
endpoint instead of sending a message. A personal or workspace skill
named `compact` takes precedence over the built-in command; while skill
collisions are still resolving, an exact `/compact` submission is
blocked with a retryable hint instead of leaking as message text.
History and queued-message edits are never intercepted. After
compaction, the context usage indicator resets to its unknown state
until the next assistant response reports fresh usage, instead of
showing the stale pre-compaction number.
- codersdk: `ExperimentalClient.CompactChat`.
Worker-path execution (rather than compacting synchronously in the
handler) reuses the existing lock fencing, live "Summarizing..."
streaming, retry accounting, restart resilience, and debug-run
observability. Rationale documented in `coderd/x/chatd/ARCHITECTURE.md`.
## Testing
- State machine: transition-matrix coverage for `RequestCompaction`,
marker lifecycle tests (carried by lease renewals/queue appends, cleared
by terminal transitions, consumed by commit), ownership handoff +
acquisition hint assertions.
- Worker: decision-ordering and forced-compaction unit tests;
active-server end-to-end test (manual compact below threshold produces a
`source=manual` summary, returns to `waiting`, no assistant follow-up;
busy chat rejected).
- API: success, archived, non-owner, RBAC-denied, empty-chat, no-daemon
cases; usage-limit ordering (at-limit owners still get
state/nothing-to-compact conflicts for no-op requests, with marker
rollback).
- Frontend: Storybook play tests for the Commands menu group, submit
intercept, skill-name collision, queued-edit passthrough, and
manual/automatic tool rendering; unit tests for command availability
resolution and the post-compaction context usage reset.
> This PR was created by Mux, an AI coding agent, working on Mike's
behalf.
|
||
|
|
f997afa220 |
feat(coderd/x/chatd/chatloop): retain user constraints in compaction summaries (#27230)
Compaction summaries drop or soften user-stated constraints, corrections, and prohibitions, so post-compaction assistants repeat behavior the user already corrected. Add a summary prompt bullet that instructs the summarizer to quote them, treat them as standing until revoked, and attribute rules to their true source instead of defaulting to the user. Validated offline on unseen human chats: holdout P1 delta +0.175 (arbitrated), 13/18 cases improve. This improves per-compaction retention only; it does not address deep-chain correction loss. |
||
|
|
cc11c8a536 |
feat: surface model content-filter refusals as a blocked chat error (#27118)
Blocked turns from a provider's content filter (Anthropic's `refusal` stop reason with empty content) previously ended silently on the "Thinking" spinner. They now end as a terminal `content_filter` error that renders as a "Response blocked" message with the provider's category and explanation. <img width="888" height="335" alt="image" src="https://github.com/user-attachments/assets/cef85a59-4091-4e62-9d45-1eb06748db48" /> Closes CODAGT-611 Follow-ups will involve implementing fallbacks, but this alone is pretty important |
||
|
|
a567f6a89f | feat: allow admins to override the chat compaction model (#27151) | ||
|
|
0ae4554de9 | fix(coderd/x/chatd/chatloop): remove compaction timeout (#27007) | ||
|
|
b21e0717d5 |
feat: remove chat chain mode (#26980)
Removes OpenAI Responses "chain mode" from chatd. Closes CODAGT-445.
- Deletes `chatopenai/responses.go` (chain detection, activation, prompt filtering, response ID extraction) and its tests.
- Deletes the `ChainBroken` classification in `chaterror` and the chatloop retry bookkeeping that disabled chain mode mid-generation.
- Drops the `chain_broken` label from the `coderd_chatd_stream_retries_total` metric.
- Stops reading and writing `chat_messages.provider_response_id`
- Deletes the dead `ClearChatMessageProviderResponseIDsByChatID` query. Dropping the column is a follow-up migration.
- Deletes three chatloop hooks no caller sets (`ReloadMessages`, `DisableChainMode`, `PrepareMessages`), the dead `const AgentChatContextSentinelPath`, and stale chain-mode comments.
🤖 Generated by Coder Agents on behalf of @johnstcn.
|
||
|
|
d6daf273aa |
fix: tighten single tool result byte budget (#26763)
## Problem #26637 caps each locally-executed tool result (built-in, global/deployment MCP, workspace MCP) at a per-result byte budget derived from the model's context window. The budget was `ContextLimit/2 * 4 bytes` — i.e. **half the window at an optimistic 4 bytes/token**. On large-context models that is far too generous. With a `1,000,000`-token `ContextLimit` the per-result cap is **~2 MB**. A user hit exactly this with a chatd (deployment-pinned) MCP tool: the result was truncated to **1,998,709 characters** and still overflowed the prompt. 2 MB of dense text (JSON/logs/code) is ~650k–1M tokens — most or all of the window for a *single* result — so the cap fired but didn't actually prevent the overflow. ## Fix Tighten the two budget constants in `tooltruncate.go`: | constant | before | after | | --- | --- | --- | | `toolResultContextDivisor` | `2` (½ window) | `3` (⅓ window) | | `bytesPerTokenEstimate` | `4` | `3` (conservative) | The budget becomes `ContextLimit/3 * 3 ≈ ContextLimit` bytes: | ContextLimit | before | after | | --- | --- | --- | | 1,000,000 | ~2 MB | ~1 MB | | 200,000 | ~400 KB | ~200 KB | | unknown (≤0) | 64 KB | 64 KB (unchanged) | The 16 KB floor and 64 KB unknown-window default are unchanged. A conservative bytes-per-token estimate is intentional: dense payloads run well under 4 B/tok, so a lower estimate yields a smaller byte budget that is less likely to underestimate the true token cost. No behavioral code paths change — only the two constants and their doc comments. The existing `tooltruncate_internal_test.go` cases derive their expectations from the constants (`LargeWindow`) or exercise the floor/default (`BelowFloor`, `Unknown`), so they remain green. <details> <summary>Investigation notes</summary> Global/deployment MCP tools (`mcpclient.ConnectAll`) are appended to `prepared.Tools` and execute locally via `ExecuteLocalTools → executeTools → executeSingleTool`, so the #26637 cap *does* apply to them for text results (`convertCallResult` joins text content into `resp.Content`). The cap was simply too large: `toolResultByteBudget(ContextLimit)` = `ContextLimit/2*4` ≈ 2 MB for a 1M-token window. Reverse-engineering the reported `1,998,709` truncated characters confirms a `ContextLimit` of ~1,000,000 tokens. Known gaps left for follow-ups (out of scope here): - **Per-step aggregate is unbounded.** MCP tools advertise `Parallel: true` and `executeSingleTool` caps each result independently, so N parallel calls in one step can sum to N × the per-result cap. - **Binary/media `Data` bypasses the cap.** Only the text payload is bounded; `image`/`media`/blob embedded-resource results are base64-encoded untouched in `executeSingleTool`. - **Compaction is reactive.** It is gated on the prior step's reported usage (`latestPromptUsage`), so it can't pre-empt a single large result appended on the current step. </details> --- Generated by Coder Agents on behalf of @kylecarbs. |
||
|
|
59fcc9c0ad |
feat: improve sub-agent orchestration tools (#26673)
Tool errors caused orchestrators to abandon spawned agents. Bare error responses and the close_agent name framed delegation as one-shot: one transient failure or timeout ended the work, and the orchestrator had no way to recover or reuse agents. Renames close_agent to interrupt_agent with a hidden backward-compatible alias. wait_agent and message_agent return structured payloads instead of bare errors, so the orchestrator can retry after a timeout, recover from an error status, or redirect an idle agent. Adds list_agents so orchestrators can rediscover spawned agents. Adds root-only orchestration guidance for error recovery. |
||
|
|
32217259b7 |
feat: cap tool output to fit the model context window (#26637)
## Problem
Local tool results were persisted and replayed to the model verbatim,
with no size cap. A single oversized result, most often a multi-megabyte
response from an MCP tool, overflows the prompt on the next request.
Every retry rebuilds the same history and fails the same way, leaving
the chat wedged in `error`. Auto-compaction is reactive (token usage is
only known after a response), so it can't catch a single result that
blows the very next request.
## Fix
Cap every locally-executed tool result at its single choke point,
`executeSingleTool` in `chatloop`, so the cap covers built-in tools,
**global (deployment-pinned) MCP**, **workspace MCP**, and provider
runners uniformly. Because this runs before the result is published to
the live stream and before it is committed, the SSE preview, the
persisted message, and the model replay all see the same bounded output.
The budget is token-aware: a single tool result may use at most half the
model's context window (`~4 bytes/token`), with a `16KB` floor and a
`64KB` default when the window is unknown. Truncation keeps the head and
tail of the output and replaces the middle with a marker telling the
model how much was removed and to narrow its query; it is UTF-8 safe and
never exceeds the budget. Binary media `Data` is passed through
untouched (only the text payload is bounded).
A `coderd_chatd_tool_result_truncated_total{provider,model,tool_name}`
counter and a warning log record each truncation.
## Out of scope
- Provider-executed results (e.g. web search) arrive via the stream, not
`executeSingleTool`.
- Dynamic/external tool results submitted through the `/tool-results`
API are validated as JSON elsewhere.
- Cumulative growth across many results is still handled by context
compaction; this change only bounds any single result.
<details>
<summary>Implementation notes</summary>
- New `coderd/x/chatd/chatloop/tooltruncate.go`:
`toolResultByteBudget(contextLimitTokens)` and
`truncateToolResultText(text, maxBytes)` (pure, unit-tested).
- `chatloop.go`: added `ContextLimit` to `ExecuteLocalToolsOptions`;
threaded a computed byte budget through `executeTools` into
`executeSingleTool`, where `resp.Content` is capped for the text,
media-text, and error branches.
- `generation.go`: passes `ContextLimit: prepared.ContextLimitFallback`
(the model's configured context limit).
- `metrics.go`: new `ToolResultTruncatedTotal` counter +
`RecordToolResultTruncated`.
- Tunable knobs live as constants in `tooltruncate.go`
(`toolResultContextDivisor = 2`, `bytesPerTokenEstimate`,
`minToolResultBytes`, `defaultToolResultBytes`).
Verified: `go build ./coderd/x/chatd/...`, `go test
./coderd/x/chatd/chatloop/...`, and the `chatd` test binary compiles.
</details>
---
Resolves CODAGT-678
Generated by Coder Agents on behalf of @kylecarbs.
|
||
|
|
4f74a7adee |
fix: enable goleak in chatd tests (#26335)
Enable goleak in chatd tests and fix some leaks. Addresses https://github.com/coder/coder/pull/26109#discussion_r3380039964 |
||
|
|
21a2652343 |
fix(coderd/x/chatd): show correct provider and clean detail for Bedrock errors (#26338)
## Problem
Bedrock errors (routed through aibridge) showed the wrong provider
("Anthropic ...")
and a doubly-wrapped detail string instead of the clean message.
## Fix (chatd only)
- **Provider label:** thread the configured provider to error
classification via
`GenerateAssistantOptions.ErrorProvider`. Transport provider still
drives prompt
prep, sanitization, and metric labels (unchanged).
- **Detail:** unwrap the SDK transport wrapper (`METHOD "URL": NNN
{body}`) to surface
the inner message; handles top-level `{"message":...}` and nested
`{"error":{"message":...}}`.
## Notes
- Surfacing a top-level `message` now applies to all providers
(intentional; nested wins when both present).
- The advisor path keeps the transport label; accepted as-is (it returns
`err.Error()`, not the classification).
- Fixes display in chatd only; the wrapper originates in aibridge (out
of scope here).
🤖 Generated by Coder Agents.
|
||
|
|
4debd23cbb |
fix: chatd refactor (#26270)
Implements the chatd stabilization RFC. Combines: - https://github.com/coder/coder/pull/25908 - https://github.com/coder/coder/pull/25923 - https://github.com/coder/coder/pull/26109 - https://github.com/coder/coder/pull/26110 - https://github.com/coder/coder/pull/26111 - https://github.com/coder/coder/pull/26112 |
||
|
|
3ab1323bc9 |
fix!: rename chat stream silence timeout error (#25973)
Renames the Agents chat stream-silence error from `startup_timeout` to `stream_silence_timeout` now that the timeout applies to any gap between provider stream parts, not just first-token startup. Updates the SDK enum, generated API docs/types, chat error copy, and Agents UI stories/status labels so the user-facing wording describes a stalled provider response instead of startup delay. > **Breaking change:** This is a very minor breaking change for the Coder Agents API: the public chat error kind enum no longer includes `startup_timeout`, so clients matching that specific value should handle `stream_silence_timeout` instead. |
||
|
|
becc858fa8 |
fix(coderd/x/chatd): retry provider stream cancellations (#26010)
Closes CODAGT-541. ## Problem An Agents chat stream could die with a terminal `context cancelled` error and surface to the user as a permanent chat failure, even when no context in our process had actually been canceled. The cancellation was a provider-returned error value (HTTP/2 RST_STREAM mid-body surfacing as `context.Canceled` from Go's net/http2), not a real caller cancel. The chain that produced the bug: - fantasy passed the provider's `context.Canceled` through unchanged. - `chaterror.Classify` short-circuited any `errors.Is(err, context.Canceled)` (or `"context canceled"` text) as terminal generic, before checking HTTP status codes or other retry signals. - `chatretry.Retry` did not retry. - The frontend rendered `type:"error"` and the chat was dead. The same short-circuit also masked retryable 5xx responses whose underlying transport error happened to wrap `context.Canceled`. ## Approach `context.Canceled` has no inherent intent. The same error value can mean a user pressing Stop, a server shutdown, the silence guard firing, or a provider-side stream reset. The only layer that can disambiguate is the one holding both the returned error and the caller context. That is `chatretry`. This PR centralizes the policy there and keeps `chaterror` context-free. ## Changes `coderd/x/chatd/chaterror/classify.go` - Add `ErrProviderTransportReset` sentinel to explicitly mark provider-side stream cancellations. - Remove the broad `context.Canceled` / `"context canceled"` short-circuit so status codes and other retry signals can win. - Classify `ErrProviderTransportReset` (with no status code) as a retryable timeout. - Keep a fallback that classifies bare `context.Canceled` as terminal-generic when no other signal is present, so legitimate caller cancels still terminate cleanly. `coderd/x/chatd/chatretry/chatretry.go` - Add `contextError(ctx)` that returns `context.Cause(ctx)` when set, falling back to `ctx.Err()`, so caller-owned cancel causes (`ErrInterrupted`, `errStreamSilenceTimeout`, server shutdown sentinels) propagate cleanly out of the retry loop. - Add `classifyProviderAttemptError(err)` that wraps a bare `context.Canceled` in `ErrProviderTransportReset` and reclassifies. Errors that already classify as retryable or carry a status code are left alone. - Restructure `Retry` so the policy is explicit and readable: check caller cancellation before attempting, run the attempt, check caller cancellation again before normalizing the provider error, then classify and retry. ## End-to-end behavior - Provider returns `context.Canceled` while caller context is healthy: classified as a retryable timeout, retried, the user sees a brief `type:"retry"` event and the chat continues. - User presses Stop: `contextError(ctx)` returns `ErrInterrupted`. Retry stops. `chatloop` flushes partial content and persists. - Stream-silence guard fires: `attemptCtx` is canceled with `errStreamSilenceTimeout`, `guardedStream` produces a classified retryable error, retry proceeds normally on the still-alive parent. - Server shutdown: parent context's cause propagates out, retry stops. |
||
|
|
8b7e040105 |
fix(coderd/x/chatd/chatloop): discourage doctrine in compaction summaries (#25850)
Two additions to the compaction summary prompt: 1. Error specificity: the "errors encountered" bullet now instructs the model to keep error notes specific (name the file, the error, the fix) and not generalize from a specific failure to a blanket tool-avoidance rule. This addresses the doctrine crystallization pattern where a single tool failure gets promoted to a standing "avoid tool X" rule that persists across compactions and model swaps. 2. Reproducibility: a new closing sentence instructs the model to reference reproducible content by path, command, or URL rather than inlining it. Content without a stable reproducer is still preserved inline with a brief summary. This targets summary bloat from inlined code blocks (worst case: 34k chars, 76 code blocks reproducing repo content verbatim). Refs CODAGT-331 |
||
|
|
6df1536256 |
fix: add missing_key error kind for missing chat api_key_id (#25783)
Refs CODAGT-486 - `codersdk/chats.go`: New `ChatErrorKindMissingKey` constant and `AllChatErrorKinds` entry - `coderd/x/chatd/chaterror/message.go`: `terminalMessage` and `retryMessage` cases - `coderd/x/chatd/model_routing_aibridge.go`: Pre-classify error with `WithClassification` - `coderd/x/chatd/model_routing_internal_test.go`: Classification assertion on production path (CRF-2) - `chatStatusHelpers.ts`: Frontend title "Chat interrupted" - `LiveStreamTail.stories.tsx`: Storybook story with `detail` assertion - `docs/ai-coder/ai-gateway/clients/coder-agents.md`: Troubleshooting entry - Tests: classification round-trip, terminal message, metrics kind enumeration > Generated with [Coder Agents](https://coder.com/agents) on behalf of @johnstcn |
||
|
|
7e2f7198dd |
fix(coderd/x/chatd/chatloop): use stream silence timeout (#25782)
Replaces the 60 second first-token timeout in the chat loop with a 10 minute stream-silence timeout. Previously, the guard bounded only the gap before the first stream part. Once any part arrived the attempt could hang indefinitely if the provider stopped streaming without closing the connection, and even normal long-running responses could be killed after 60 seconds if the provider was slow to emit the first token. The guard now arms when a model attempt opens its stream, resets on every received stream part, and fires after 10 minutes of complete silence. The existing retry path still handles the timeout, and the public `startup_timeout` error kind is preserved to avoid API and frontend churn. 10 minutes matches the default request timeout used by the Anthropic and OpenAI Python SDKs. Closes CODAGT-493 |
||
|
|
c650aabbef |
chore: standardize on *_internal_test.go for white-box tests (#25601)
My agent added `//nolint:testpackage` to a test file on one of my PRs. Again. This PR cleans it up across the entire repo and updates the in-repo conventions so future agents stop doing it. The repo already has a precedent for white-box tests that need to touch unexported symbols: `*_internal_test.go` (145+ existing files). The `testpackage` linter's default `skip-regexp` exempts that filename suffix, so the `//nolint:testpackage` directive is unnecessary in every case where someone reached for it. This PR renames 51 such files to `*_internal_test.go` via `git mv` so blame and history follow, and strips the dead directive from 2 files that were already correctly named (`coderd/oauth2provider/authorize_internal_test.go`, `coderd/x/chatd/advisor_internal_test.go`). `.claude/docs/TESTING.md` now documents the rule explicitly under *Test Package Naming*, which is imported into the root `AGENTS.md` via `@.claude/docs/TESTING.md`. The rule: prefer `package foo_test`; if you need internal access, rename the file to `*_internal_test.go` rather than adding a nolint directive. |
||
|
|
1e8c8d7dba |
fix(coderd/x/chatd): drop orphan provider tool calls on replay (#25491)
Anthropic replay can fail when stored history contains a
provider-executed tool call like `web_search` without the matching
provider-executed result. That orphaned call is incomplete
provider-internal state, so replaying it can make an otherwise usable
chat unreplayable even though there is no search result to preserve.
This fixes replay by dropping orphan provider-executed tool calls from
the model-visible prompt, preserving signed reasoning and the rest of
the assistant content, then revalidating before the request. We do not
synthesize tool results or drop reasoning. The database can retain the
historical artifact for inspection, while Anthropic only sees replayable
content.
This matches permissively licensed prior art. Vercel AI SDK
(Apache-2.0), used by mux, keeps incomplete tool state in UI/history but
omits it from model requests with `convertToModelMessages(..., {
ignoreIncompleteToolCalls: true })`. LangChain, LiteLLM, and OpenAI
Agents (MIT for the relevant open-source code) also preserve Anthropic
signed reasoning as opaque replay data. Coder applies that model-visible
replay boundary explicitly because our persisted history is already in
provider-message form.
This matches mux, is cleaner than the older idea around not persisting
the search query tool, and the model handles the repaired prompt fine.
Closes CODAGT-448
## Before
<img width="963" height="491" alt="image"
src="https://github.com/user-attachments/assets/a7788ebf-2728-4420-90cf-5e4f6905bdf7"
/>
## After
<img width="842" height="513" alt="image"
src="https://github.com/user-attachments/assets/ae39c262-7586-4e2d-b7db-1b639a7e8e15"
/>
|
||
|
|
385146000b |
feat: record created_at/completed_at on reasoning ChatMessageParts (#24789)
Records reasoning start and end times on persisted reasoning `ChatMessagePart`s so reasoning duration can be computed for stored chats. Backend-only: no SSE changes and no frontend rendering ship in this PR. The `created_at` field on `ChatMessagePart` is extended to also be present on `reasoning` parts (it previously appeared only on `tool-call` and `tool-result`), and a new `completed_at` field is added for `reasoning` parts. ### How timestamps are recorded - `StreamPartTypeReasoningStart`: stamp `startedAt = dbtime.Now()` on the active reasoning state. - `StreamPartTypeReasoningEnd`: stamp `completedAt = dbtime.Now()` and append both into parallel `[]time.Time` slices on `stepResult`. - Persistence reads the slices in occurrence order (reasoning has no provider-side ID) and applies them to the matching `ChatMessagePart` via `buildAssistantPartsForPersist`. The first reasoning block's stamps go onto the first reasoning part, and so on. - `flushActiveState` flushes partial reasoning interrupted before `StreamPartTypeReasoningEnd` with `startedAt` from the active state and `completedAt = dbtime.Now()` at the interruption. ### Why two fields, not one? Tool calls and results are point events. The frontend computes their duration by subtracting the call's `created_at` from the result's `created_at`. Reasoning is one assistant part that brackets a span, so we record both endpoints on the part itself. ### Why not stamp in `PartFromContent`? Same rationale as #24101: `PartFromContent` is called during both SSE publishing and persistence. Stamping there would yield incorrect persistence-time timestamps for reasoning blocks that finished much earlier in the step. Instead we capture in the chatloop and apply during persistence. <details><summary>Implementation plan</summary> - `codersdk/chats.go`: extend `CreatedAt`'s `variants` to include `reasoning?`; add `CompletedAt *time.Time` with `variants:"reasoning?"`. - `coderd/x/chatd/chatloop/chatloop.go`: extend `reasoningState` with `startedAt`; extend `stepResult` and `PersistedStep` with parallel `[]time.Time` reasoning slices; stamp on `ReasoningStart`/`ReasoningEnd`; thread the slices through all `PersistStep` call sites including the interrupt-safe path; record partial reasoning in `flushActiveState`. - `coderd/x/chatd/attachments.go`: walk reasoning parts in occurrence order and apply `step.ReasoningStartedAt[i]` to `part.CreatedAt` and `step.ReasoningCompletedAt[i]` to `part.CompletedAt`. ### Tests - `codersdk/chats_test.go` round-trips `created_at` + `completed_at` on reasoning parts and verifies omission when absent and partial interrupted parts. - `coderd/x/chatd/chatprompt/chatprompt_test.go` asserts `PartFromContent(ReasoningContent{})` does NOT stamp timestamps. - `coderd/x/chatd/chatloop/chatloop_test.go` `TestRun_ReasoningTimestamps` drives a stream with two reasoning blocks and verifies parallel slices, monotonicity, ordering, non-zero values, and content-block ordering. `TestRun_InterruptedReasoningFlushesTimestamps` cancels mid-reasoning and verifies `flushActiveState` records a non-zero pair. - `coderd/x/chatd/attachments_test.go` covers `buildAssistantPartsForPersist` for normal interleaved reasoning, partial (zero `completed_at`), and missing slices. </details> > Generated by Coder Agents. Co-authored-by: Coder Agent <agent@coder.com> |
||
|
|
e75bd3aca4 |
fix: preserve Anthropic replay fidelity (#25377)
Anthropic is strict about replaying the latest assistant turn once it contains signed or redacted reasoning. We were still mutating that turn in a few Coder-owned places: dropping empty reasoning blocks on replay, rewriting provider-tool history during sanitization, and in the worst case sending a prompt we already knew Anthropic would reject. This patch keeps the latest signed assistant immutable through Coder's replay and sanitization paths, preserves empty signed or redacted reasoning anywhere Coder owns the ledger, and fails before the provider call if the prompt is still unsafe. It also bumps the existing `coder/fantasy` `coder_2_33` fork that `main` already uses to the commit containing coder/fantasy#35. These fixes have also been upstreamed to charmbracelet/fantasy. Closes CODAGT-409. |
||
|
|
376fc80451 |
fix(coderd/x/chatd): discover workspace MCP tools mid-turn after create_workspace (#25169)
## Problem In `coderd/x/chatd/chatd.go` `runChat`, workspace MCP discovery is gated on `chat.WorkspaceID.Valid` at the start of each turn. New chats that bind their workspace mid-turn (via `create_workspace` or `start_workspace`) get an empty workspace tool list on the first step, and the model falls back to `execute` (bash) because no workspace MCP tools are advertised. **Repro:** new chat → "create a workspace and use MCP tools". No `/api/v0/mcp/tools` request hits the agent on turn 1; turn 2 in the same chat works fine. ## Fix - Add a `PrepareTools` callback to `chatloop.RunOptions`, analogous to `PrepareMessages`. It is invoked once before each LLM step with the current tool list. When it returns non-nil, the chatloop replaces `opts.Tools`, rebuilds the per-step tool definitions, and appends new tool names to `opts.ActiveTools` so newly injected tools are callable immediately. - Wire `PrepareTools` in `runChat` to trigger workspace MCP discovery the first time the chat snapshot reports a valid `WorkspaceID`. The previous top-of-turn discovery path is unchanged for chats that start with a workspace. - Extract the discovery logic into `Server.discoverWorkspaceMCPTools` so the top-of-turn and mid-turn paths share identical behavior (cache, agent resolution, `ListMCPTools` timeout, invalidation). Mid-turn discovery stays disabled in plan-mode turns and Explore subagents, matching the existing top-of-turn gate. The `workspaceMCPDiscovered` flag prevents redundant dials after the first successful discovery. ## Tests - `coderd/x/chatd/chatloop/chatloop_test.go`: two new `TestRun_PrepareTools*` cases covering injection on the next step and active-set merging when `ActiveTools` is non-empty. - `coderd/x/chatd/chatd_test.go`: `TestRunChat_WorkspaceMCPDiscoveryAfterMidTurnCreateWorkspace` drives `runChat` through a `create_workspace` tool call against a real Postgres + mocked agent conn and asserts the second streamed LLM request advertises the workspace MCP tool. Verified that the test fails (and pinpoints the missing tool) when the `PrepareTools` wiring is disabled. ## Validation ``` go test ./coderd/x/chatd/chatloop/... -count=1 go test ./coderd/x/chatd/... -count=1 make lint/emdash ``` <details> <summary>Decision log</summary> - Chose a per-step `PrepareTools` callback over mutating `opts.Tools` in place because `chatloop.Run` builds the `fantasy.Tool` definitions once at start; a hook is required to let the LLM see new tools on the next step. - Returned `[]fantasy.AgentTool` (not also active-tool-names) and let the chatloop derive name merges via `mergeNewToolNames`. This avoids leaking plan-mode gating decisions into the callback contract. - Kept the existing top-of-turn discovery path so chats that already have a workspace at turn start pay no extra latency. - Skipped reusing `ReloadMessages` (history reload) since this is purely a tool-availability concern; coupling it to a history reload would defeat the chatloop cache prefix optimizations. </details> --- _This pull request was generated by Coder Agents._ |
||
|
|
e8508b2d90 |
fix: recover chatd from poisoned chain anchor on retry (#25097)
When OpenAI's Responses API returns `Previous response with id ... not found` for a chained turn, classify it as a `ChainBroken` retry, clear `previous_response_id`, exit chain mode, reload full history, and let `chatretry` retry. Self-heals chats whose anchor was poisoned before #25074 stopped truncated streams from being persisted as a successful turn with a stored response id. The new state is exposed via the existing `coderd_chatd_stream_retries_total` counter as a `chain_broken="true"|"false"` label. Aggregating queries (`sum`, `rate` over `provider`/`model`/`kind`) keep working without changes; raw-series matchers without aggregation will now see two series per `(provider, model, kind)` where they previously saw one. The metric is internal-only so the blast radius should be small, but if you have dashboards that index by exact label matchers without aggregation they will need an extra `sum` or an explicit `chain_broken` selector. > 🤖 This PR was created with the help of Coder Agents, and was reviewed by a human 🧑💻 |