Commit Graph
9 Commits
Author SHA1 Message Date
Mathias Fredriksson d639ced1e0 fix: send only changed chat history over the stream (#29662)
A client could resume the chat stream only with after_id, a message
ID. A message ID does not identify what a client has received. A
reconnect could skip a message that another client sent, send no
history_reset for an edit the client missed, or resend the whole
history.

A client can now resume with the chat's snapshot_version, which the
newest messages page returns and the stream's events then include.
The stream sends what changed at that version or later. For an edit
it sends the ID of the edited message and the messages written
since, not the whole history. A stream opened without
snapshot_version sends the same events as before.

A write outside a chat state transition does not increase
snapshot_version, and a stream can miss it. The queries that only
tests used for such writes are removed.

When the stream cannot be set up, the server now returns 500 before
the WebSocket upgrade. It used to upgrade and send an error event,
which the web client shows as a chat error.

The Tasks to Chats migration guide told clients to reconnect with
after_id. It now describes snapshot_version.
2026-11-13 13:02:38 +02:00
Michael Suchacz 200fa7dea7 perf(coderd): bound the initial chat stream snapshot to the client cursor (#29761)
Opening a chat on the Agents page loads its transcript over REST and
then connects the chat stream with `after_id` set to the newest loaded
message. The first sync of the stream ignores that cursor when it
queries the database: it fetches every row of the chat on each
connection, and then sends only the rows above `after_id`.

The first sync now resolves `after_id` to that message's revision and
fetches only rows changed at or after it. The history reset rule is the
one on `main` (#29808): the stream resets only when message `after_id`
is deleted or missing. A deleted cursor still produces one
`history_reset` with the surviving history, because the bounded query
includes the cursor row. An unknown ID keeps the full query. A message
ID from another chat, live or deleted, is ignored, so the client
receives the full history. Live updates after the first sync are
unchanged.

## Performance

Measured on dogfood's database, which runs `main`, with `EXPLAIN
(ANALYZE, BUFFERS, SERIALIZE BINARY)` of the first-sync query. The chats
are the 554 chats updated in the past day, picked by row count. `main`
starts at revision 0. This PR starts at the revision of the newest live
message, which is the `after_id` the web client sends.

| Chat (rows) | `main` output | `main` first / repeat run | This PR
output | This PR time |
|---|---|---|---|---|
| p50 (41) | 189 kB | 21.5 ms / 0.6 ms | 1 kB | 0.1 ms |
| p90 (351) | 2.0 MB | 163 ms / 7.9 ms | 10 kB | 0.5 ms |
| p99 (2,200) | 7.1 MB | 425 ms / 25 ms | 14 kB | 3.0 ms |
| Largest (14,746) | 64.9 MB | 4.8 s / 193 ms | 6 kB | 17.6 ms |

- Across the 554 chats, the bounded query returns 1 row up to p99 and 5
rows at most.
- Both queries visit the same heap pages, because no index covers
`revision`. The saving is the TOAST reads, serialization, transfer, and
decoding of rows that `main` then discards. Most of `main`'s time is
serialization: 153 ms of 193 ms for the largest chat.
- The cursor lookup is one primary-key read of about 0.4 ms, so a median
chat gains almost no latency. The gain grows with chat size.
- Deleted rows are about 0.1% of rows, so they do not change the cost.

## Not changed

A tab whose stream was down can miss an edit, then send a message and
store the response, so its newest ID is above the edit. On reconnect,
message `after_id` is live and the stream sends no reset, so the tab
keeps the deleted messages. `main` has done the same since #29808, and
this PR does not change it (mafredri's reproduction gives `expected [5 6
7 8], actual [3 4 7 8]` on both). An ID cannot show which edits the
client applied. #29662 adds a client-reported version for that.

<details>
<summary>Delivery record</summary>

- Rebased on `main` at `6ebefb83254`, keeping the #29808 reset rule and
#30266 read-only snapshot reads. The cursor-bound mock test is part of
`TestStreamLoopInitialSyncTombstones`.
- Assumption checked on dogfood: no live message has a higher revision
than a later message. Messages are only updated to soft-delete them or
to set search columns, which the revision trigger ignores. 2,861 rows in
29 recently active chats break it, all created between 2026-06-26 and
2026-07-16 and none since. A client cursor cannot sit before them,
because the client sends its newest message.
- Local: `go test ./coderd/x/chatd/...`, `go test ./coderd -run
'^TestStreamChat$'`, `go test ./coderd/database/dbauthz -run
'^TestMethodTestSuite$'`. Mutants of the cursor bound (no `- 1`, no
bound, cursor from another chat kept) each fail `TestStreamLoop*`.
- Remote UAT before the rebase covered deleted, foreign, and unknown
cursors, edits, grouped commits, and reconnects. On a synthetic
5,000-message chat, dial to first snapshot went from 129.6 ms to 15.0
ms.
- Codex: [round
1](https://github.com/coder/coder/pull/29761#pullrequestreview-5284702021)
and [round
2](https://github.com/coder/coder/pull/29761#pullrequestreview-5284855535)
found cross-chat cursor bugs, fixed. Round 3 and the round on
`7bed7855e41` found nothing.

</details>

> Xum opened this PR on behalf of @ibetitsmike.
2026-10-19 19:15:56 +02:00
J. Scott Miller 939146b077 fix(coderd/x/chatd): reset stream history only if after_id is deleted (#29808)
Fixes
[CODAGT-1175](https://linear.app/codercom/issue/CODAGT-1175/prevent-chat-stream-resets-from-loading-full-history)

**tl;dr:** The chat stream no longer sends the full history on every
connect when a chat contains old edit tombstones.

On its first sync, the chat stream WebSocket treated any soft-deleted
message (edits create them) as a history reset and sent `history_reset`
plus every message, ignoring `after_id`. The client replaced its
paginated cache with that history, so every connect, reconnect and PWA
resume of a long chat that had ever been edited rendered the whole
transcript. A dogfood chat of about 3.2k messages sent 7.1 MB per
connect.

`historyResetRequired` in `stream_loop.go` now resets on the first sync
with an `after_id` only if message `after_id` is missing or deleted.
Messages are only deleted as a suffix of the chat (an edit deletes the
edited message and every later one), so while that message is live the
client holds none of the deleted rows. Otherwise the stream sends only
live messages newer than `after_id`, and `messageEvents` never sends
tombstones as messages. Deletions after the first sync still reset.
`stream_loop_internal_test.go` covers the initial-sync cases and the
deletion of a streamed message after the first sync,
`TestSubscribeAfterMessageIDWithEdit` runs a real `EditMessage` against
the database, and `ARCHITECTURE.md` gets TODOs for the new rule.

> Generated by Coder Agents on behalf of @jscottmiller.
2026-10-16 15:40:17 -05:00
Callum Styan 1a96da2b6a perf(coderd/x/chatd): make chat stream reads lock-free (#30266) 2026-10-05 11:47:24 -07:00
Thomas Kosiewski e3446c9c21 fix(coderd/x/chatd): drop late preview parts after a chat error (#30279)
When a generation attempt fails without a retry (`FinishError`), the chat moves to `error` but keeps its `history_version` and `generation_attempt`, and `retry_state` stays null. The stream loop therefore did not retire the attempt, and a part of the failed attempt that the relay delivered after the `error` event was still forwarded. The frontend clears its preview on `error` (`useChatStore.ts`), so the user saw a mid-message fragment appear under the error.

This is the error-path variant of the issue fixed in #30241 for retries. The stream loop now also retires the current attempt when a sync observes `status = error`, so its late parts are dropped. Every generation attempt increments `generation_attempt` before it creates its part episode (`beginGenerationAttempt`), so the next run after an error starts a fresh preview as before. Other statuses that leave `running` (`interrupting`, `requires_action`, `waiting`) are unchanged: the frontend does not clear the preview for them. The frontend contract is unchanged.

`TestStreamLoopLatePartAfterError` fails on main and passes with the fix; its subtests also cover the next attempt after an error and an interrupt that keeps streaming.

A separate docs commit updates step 7 of the sync algorithm in `ARCHITECTURE.md` and corrects the `preview_reset` description, which said it is emitted when the LLM call fails mid-way. The code emits it only when the history version changes or a new generation attempt starts.

The bug was found by a TLA+ model of the chat stream and confirmed with a Go reproduction.

Refs #30241

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-02 13:02:41 +02:00
Thomas Kosiewski 9ac4628947 fix(coderd): stop announcing stale chat retries and fence retried attempts (#30241)
Stop announcing a pending chat retry once the chat leaves running, and stop streaming preview parts of a failed generation attempt after its retry is recorded. UpdateChatExecutionState now clears retry_state on every non-running status write (the trigger advances retry_state_version), the stream loop only emits retry while the chat is running, and it retires the current attempt once it observes a retry payload or an advanced retry_state_version within an unchanged episode. Regression tests cover the error, interrupt and requires_action exits, late parts after a retry, and a retry cleared before the stream syncs; ARCHITECTURE.md documents the new rules.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-01 14:10:00 +02:00
Mathias Fredriksson fa5bac6e77 feat: send tool call IDs and cancel unresolved calls on interrupt (#29954)
Retries and replica takeovers ran tool calls again, and an interrupt
recorded the same generic result whether a call had finished, was
still running, or never started.

chatd now derives a tool call ID from the chat, the committed
assistant message, and the provider tool call ID, and sends it with
execute starts, edit_files, and write_file. Every replica and retry
computes the same ID, so the agent returns the first result instead
of running the tool again, and a retried foreground execute waits
only for the rest of its timeout.

On interrupt, chatd cancels the unresolved calls of those tools on
the agent before committing, within 30 seconds, and records what
happened: "tool call was canceled before it ran; no changes were
made", the output of a command canceled while running, or the real
result. The UI shows a canceled command with the existing killed
marker. If the cancel fails, including an old agent's 404, the result
stays the generic interrupted text.

Calls whose provider tool call ID is empty or repeats in the message
get no ID and behave as before.

Fixes CODAGT-757
2026-09-30 14:31:52 +03:00
Michael Suchacz 680b2f59cd refactor: remove obsolete chatd code and tests (#29228)
Chatd still carries an unreachable interruption flush path, unused state
and options, an uncalled workspace MCP cache invalidation callback, and
a stream-buffer-drop counter that production never increments. Remove
these paths and their dedicated assertions, and update the generated
metrics inventory and documentation.

Consolidate duplicate publication and rollback tests and move fixture
writers and test-only inspection helpers into test files. Retain the
interruption persistence, state-machine matrix, legacy message format,
and provider compatibility coverage.
2026-09-14 10:51:22 +02:00
Hugo Dutka 4debd23cbb fix: chatd refactor (#26270)
Implements the chatd stabilization RFC.

Combines:
- https://github.com/coder/coder/pull/25908
- https://github.com/coder/coder/pull/25923
- https://github.com/coder/coder/pull/26109
- https://github.com/coder/coder/pull/26110
- https://github.com/coder/coder/pull/26111
- https://github.com/coder/coder/pull/26112
2026-06-12 13:33:12 +02:00