Files
coder/coderd/database/dbmetrics
Michael Suchacz 200fa7dea7 perf(coderd): bound the initial chat stream snapshot to the client cursor (#29761)
Opening a chat on the Agents page loads its transcript over REST and
then connects the chat stream with `after_id` set to the newest loaded
message. The first sync of the stream ignores that cursor when it
queries the database: it fetches every row of the chat on each
connection, and then sends only the rows above `after_id`.

The first sync now resolves `after_id` to that message's revision and
fetches only rows changed at or after it. The history reset rule is the
one on `main` (#29808): the stream resets only when message `after_id`
is deleted or missing. A deleted cursor still produces one
`history_reset` with the surviving history, because the bounded query
includes the cursor row. An unknown ID keeps the full query. A message
ID from another chat, live or deleted, is ignored, so the client
receives the full history. Live updates after the first sync are
unchanged.

## Performance

Measured on dogfood's database, which runs `main`, with `EXPLAIN
(ANALYZE, BUFFERS, SERIALIZE BINARY)` of the first-sync query. The chats
are the 554 chats updated in the past day, picked by row count. `main`
starts at revision 0. This PR starts at the revision of the newest live
message, which is the `after_id` the web client sends.

| Chat (rows) | `main` output | `main` first / repeat run | This PR
output | This PR time |
|---|---|---|---|---|
| p50 (41) | 189 kB | 21.5 ms / 0.6 ms | 1 kB | 0.1 ms |
| p90 (351) | 2.0 MB | 163 ms / 7.9 ms | 10 kB | 0.5 ms |
| p99 (2,200) | 7.1 MB | 425 ms / 25 ms | 14 kB | 3.0 ms |
| Largest (14,746) | 64.9 MB | 4.8 s / 193 ms | 6 kB | 17.6 ms |

- Across the 554 chats, the bounded query returns 1 row up to p99 and 5
rows at most.
- Both queries visit the same heap pages, because no index covers
`revision`. The saving is the TOAST reads, serialization, transfer, and
decoding of rows that `main` then discards. Most of `main`'s time is
serialization: 153 ms of 193 ms for the largest chat.
- The cursor lookup is one primary-key read of about 0.4 ms, so a median
chat gains almost no latency. The gain grows with chat size.
- Deleted rows are about 0.1% of rows, so they do not change the cost.

## Not changed

A tab whose stream was down can miss an edit, then send a message and
store the response, so its newest ID is above the edit. On reconnect,
message `after_id` is live and the stream sends no reset, so the tab
keeps the deleted messages. `main` has done the same since #29808, and
this PR does not change it (mafredri's reproduction gives `expected [5 6
7 8], actual [3 4 7 8]` on both). An ID cannot show which edits the
client applied. #29662 adds a client-reported version for that.

<details>
<summary>Delivery record</summary>

- Rebased on `main` at `6ebefb83254`, keeping the #29808 reset rule and
#30266 read-only snapshot reads. The cursor-bound mock test is part of
`TestStreamLoopInitialSyncTombstones`.
- Assumption checked on dogfood: no live message has a higher revision
than a later message. Messages are only updated to soft-delete them or
to set search columns, which the revision trigger ignores. 2,861 rows in
29 recently active chats break it, all created between 2026-06-26 and
2026-07-16 and none since. A client cursor cannot sit before them,
because the client sends its newest message.
- Local: `go test ./coderd/x/chatd/...`, `go test ./coderd -run
'^TestStreamChat$'`, `go test ./coderd/database/dbauthz -run
'^TestMethodTestSuite$'`. Mutants of the cursor bound (no `- 1`, no
bound, cursor from another chat kept) each fail `TestStreamLoop*`.
- Remote UAT before the rebase covered deleted, foreign, and unknown
cursors, edits, grouped commits, and reconnects. On a synthetic
5,000-message chat, dial to first snapshot went from 129.6 ms to 15.0
ms.
- Codex: [round
1](https://github.com/coder/coder/pull/29761#pullrequestreview-5284702021)
and [round
2](https://github.com/coder/coder/pull/29761#pullrequestreview-5284855535)
found cross-chat cursor bugs, fixed. Round 3 and the round on
`7bed7855e41` found nothing.

</details>

> Xum opened this PR on behalf of @ibetitsmike.
2026-10-19 19:15:56 +02:00
..