Commit Graph
467 Commits
Author SHA1 Message Date
Thomas Kosiewski f923f4a525 chore!: remove the chat automations experiment (#30412)
Removes the experimental Chat Automations feature (`chat-automations` experiment) from main. This reverts the 16-PR stack that added it and adds two forward migrations that undo its schema and credential changes.

> [!WARNING]
> **Upgrade consequences**
>
> - **Automation data is deleted.** Migration 000615 drops the `chat_automations` table and deletes every queued message an automation delivered. Chats, their messages and ordinary queued messages stay. Messages lose their automation provenance (`automation_id`, `input_id`).
> - **Automation-only credentials are revoked.** Migration 000616 deletes API keys, OAuth2 codes and OAuth2 tokens whose only scopes (or allow-list entries) were `chat_automation:*`. Mixed credentials keep their other scopes. Custom roles lose their `chat_automation` permissions. OAuth2 apps that allowed only `chat_automation` scopes get the whitespace allowlist that grants nothing, as in 000595.
> - **A coordinated restart is required.** Stop all old replicas before the new version runs the migrations. Old replicas read the dropped table and columns, so a rolling deploy is not supported.
> - **Remove `--chat-max-automations-per-owner` before upgrading.** The flag, the YAML key `chat.maxAutomationsPerOwner` and the env var `CODER_CHAT_MAX_AUTOMATIONS_PER_OWNER` are removed. A deployment that still sets the flag or the YAML key will not start (`unknown flag` / `unknown option`). The env var is ignored. This includes dogfood. No release or tag contains the flag.
> - **Down migrations restore the schema only.** 000616 down and 000615 down recreate the 000611 schema with empty tables. They do not restore deleted automations, webhook secrets, provenance, queued deliveries, revoked credentials, stripped permissions or the deleted experiment rule.

## What is removed

- The `chat-automations` experiment and its stored runtime rule (`experiment_rule:chat-automations`).
- The `--chat-max-automations-per-owner` deployment option.
- The `chat_automation` RBAC resource, its member and chatd grants, and the `chat_automation:*` API key scopes.
- The automation sqlc queries, dbauthz wrappers and audit table entries.
- The management routes under `/api/experimental/organizations/{organization}/chat-automations`, and the webhook route `POST /api/experimental/chat-automations/{automation}/events` with its rate limiter.
- The chatd management service, webhook publish, schedule loop, the `manage_automations` tool, and the chatstate admission seam and queue promotion guard that only automations used.
- The Agents Automations page, editor, nav entry, chat labels, the Manage automations toggle and the API client.
- The `archived:any` chat search value that the automations "View chats" dialog added.
- `docs/ai-coder/agents/automations.md`, its manifest entry, the glossary entry and the related rows.

Reverted PRs: #30085, #30096, #30112, #30117, #30127, #30134, #30160, #30166, #30167, #30171, #30187, #30207, #30214, #30217, #30219, #30220.

## What stays, and why

- Migrations 000609, 000610 and 000611, their fixture, and the #30297 lock fix with `migration000609_test.go` stay unchanged. Deployments such as dogfood already ran them.
- New migration **000615** reverses the schema changes of 000609 and 000611 against the current schema. It uses the same short-wait lock-and-retry block as 000609, and it rebuilds `chats_expanded` from main's current definition minus the two automation columns. New migration **000616** follows the 000595 Tasks removal for credentials and recreates `api_key_scope` without the five `chat_automation:*` values. It keeps the `chat_project_memory:*` values from 000612.
- The `chat_automation` value stays in the `resource_type` enum, and `codersdk.ResourceTypeChatAutomation` stays. Old audit log rows stay readable, as with `task` after the Tasks removal.
- The 000615 delete fires the existing queue-version trigger, so `queue_version` moves to `snapshot_version` for each affected chat. `snapshot_version` is not bumped: the coordinated restart restarts every stream loop from an empty state, and the site does not read the version fields.
- `coderd/x/chatd/ARCHITECTURE.md` drops the sections the stack added and gets back its pre-stack wording, merged with the later main edits to the same file.

## Notes for reviewers

- #30379 (chat project page) merged while this PR was in progress. It passed `manageAutomationsEnabled` in `AgentCreatePage.test.tsx` and listed an `automations` route in `AgentsPageLayout.test.tsx`. Later main commits also added automation props in `ConversationTimeline.test.tsx` and `ConversationTimelineWorkingBlocks.stories.tsx`. This PR removes all of them.
- Remaining `automation` matches under `coderd`, `codersdk`, `enterprise`, `site/src` and `docs` are intended:
  - migrations 000609 to 000611, their fixture and `migration000609_test.go`;
  - the new 000615 and 000616 migrations and `migration000615_test.go`;
  - the `chat_automation` resource type in `dump.sql`, `models.go`, `codersdk/audit.go`, the generated API docs, `swagger.json` and `typesGenerated.ts`;
  - unrelated uses of the word "automation" (docs, `chatlabels_test.go`, `ratelimit.go`, `autostop.go`, `subagent_catalog.go`, `WorkspaceSharingSection.tsx`, `AgentChatPage.stories.tsx`).

Remote UAT on an isolated dev deployment passed. It upgraded a populated pre-removal database, then checked chats, queued messages, tokens, the 404 routes, the removed page and nav entry, and the experiments list.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_

![Agents sidebar after the upgrade, with no Automations entry](https://github.com/user-attachments/assets/bd446035-7fc8-4ca4-a735-3a4f36380738)
2026-10-11 19:57:49 +02:00
McKayla はな 86ebc72236 chore: remove unused autofill-parameters endpoint and experiment (#30525) 2026-10-09 16:42:41 -06:00
Michael Suchacz 782fb30f09 feat: load whole chat turns per history page (#30524)
Closes
[CODAGT-1372](https://linear.app/codercom/issue/CODAGT-1372/load-whole-turns-per-history-page-so-collapsed-working-blocks-fill-in).

With collapsed working blocks, a long agent turn renders as one row, but
chat history pages hold a fixed 50 messages. A 400-step turn took about
eight sequential page loads to fill, and its block showed partial labels
("at least", "or more") until then.

`GET /api/v2/chats/{chat}/messages` now returns `turn_start_id` on
newest-first pages: the ID of the user prompt that starts the turn
containing the page's oldest message. It comes from a new
`GetChatTurnStartID` query on the existing
`idx_chat_messages_user_prompts` index, so there is no migration.
Soft-deleted prompts and model-only user rows never start a turn.

The history query then fills each page back to that prompt with requests
of up to 200 messages, capped at 1000 messages per page, so a page holds
whole turns and a complete block renders without partial labels. Past
the cap, or when a fill request fails or returns no messages, the page
keeps what loaded and ends mid-turn, and the next page continues it, as
before. Because the fill runs inside the page fetch, a chat now waits
for its fill requests (one to five) before it first renders: with an
injected 500 ms per-request delay, a chat whose newest turn hits the cap
appeared after 3.8 s instead of 1.1 s.

## Screenshots

From the dogfood UAT, the same 200-step turn with its next history
request held, so each label shows only what loaded before it:

**Before**, on the base commit `29e876df4da`, after two 50-message
pages: "Worked for at least 1m 35s (48 steps or more)", partly hidden by
the loading indicator.

![Before: partial label for a 200-step
turn](https://github.com/user-attachments/assets/6b4e4abf-4bc6-4476-8ed5-52c39c8eebe6)

**After**, on `186a9b053a0`, after the first page: "Worked for 6m 39s
(200 steps)".

![After: exact label for the same 200-step
turn](https://github.com/user-attachments/assets/22ec1900-1e7b-4c98-87a9-0463e1d852c2)

A 750-step turn, larger than one 1000-message page, is exact once the
next page loads:

![After: 750-step turn,
expanded](https://github.com/user-attachments/assets/f927a848-62b1-4eb7-a548-fbdff61fc9c4)

A malformed fill response keeps the loaded messages and the partial
label instead of showing "Failed to load chat", as `ecdfb27ec29` did:

![Malformed fill response: partial label, no error
screen](https://github.com/user-attachments/assets/6d393df9-15a2-471e-9bb6-71e919aca570)

## Testing

- Go: `TestGetChatMessages_TurnStartID` (pages spanning turns, deleted
prompts, model-only user rows, polling and empty pages) and the dbauthz
suite entry.
- Vitest: fill request parameters, the 1000-message cap, failed,
malformed, and empty fill responses, and servers without
`turn_start_id`.
- Dogfood UAT through Coder Agents: PASS on `186a9b053a0` (round 4):
whole-turn first pages, the 1000-message cap and its continuation, fill
failures, a live turn, and API checks for deleted prompts and model-only
user rows. Round 3 on `ecdfb27ec29` failed: a malformed fill failed the
chat load, and an empty fill reporting `has_more` repeated forever.
`7266f468290` fixes both, with regression tests.

## Review record

- Test audit with mutation checks: the Go test now covers a page
spanning two turns, which catches a lookup keyed on the newest message.
A re-audit of later test changes changed nothing; the one surviving
mutation drops the `!older.has_more` stop, costing one extra request
when the prompt is deleted mid-fill.
- Frontend review (8 roles, 5 rounds): fixed doc, naming, loose-type,
and test findings. Its suggestions to catch only the fill request (DF5)
and drop the empty-page stop (SD4) caused the round 3 failures and are
reverted. Kept: SD1 (`MAX_MESSAGES_PER_REQUEST` is used once, named like
`MESSAGES_PAGE_SIZE`) and round 5 SD4 (`!older.has_more` is the
endpoint's pagination signal). Declined DF6: the `queued_messages`
refetch race predates this change, and a failed refetch fill only
unloads older history that reloads on scroll.
- Codex: no findings on `186a9b053a0`.
- Known: a hung fill delays the first render (no client timeout); fills
are not cancelled when the user leaves; the top of history costs one
extra empty request; injected malformed fill rows break the page as they
do on the first page.

> Xum opened this PR on behalf of @ibetitsmike.
2026-10-09 13:32:39 +02:00
Michael Suchacz 7993c73c87 feat: add organization-scoped Coder Agents instructions (#30272)
Adds organization-scoped Coder Agents instructions. Organization admins
set them in **Admin settings** > **AI** > **Coder Agents** >
**Organization settings**, or with `GET`/`PUT
/api/v2/organizations/{organization}/chats/config/system-prompt`.
Implements [CODAGT-984](https://linear.app/codercom/issue/CODAGT-984).

The instructions add to the deployment system prompt and never replace
it. When a chat is created, chatd stores the organization prompt as its
own system row right after the deployment row, only for chats in that
organization. Subagents resolve their parent organization's current
prompt when they spawn, and existing chats keep their stored rows.
Access reuses the organization `chat_model_config` permission: owners
and organization admins edit, auditors read, members get 404 on read and
403 on write. Writes reuse the deployment prompt's sanitization, size
limit (128 KiB by default), and audit behavior, with a new
`chat_organization_system_prompt` audit resource type. Migration
`000613` (the next number after `000612` on `main`) adds the
`chat_organization_system_prompts` table.

## Testing

- Go: enterprise authorization (including an update-only custom role),
validation, and audit tests; chatd row order and subagent tests.
- Frontend: Vitest interaction tests and stories for the card (loading,
load error, refetch error over the cached value, save error, read-only).
- Remote dogfood UAT, [evidence
chat](https://dogfood.cdr.dev/agents/b595a709-0709-44e6-9535-96a922974d61):
round 1 **PASS** on `b027cb5ca67` (permission matrix, validation, UI,
row order, cross-organization isolation, audit, subagents, regression).
Round 2 **PASS** on `618bff67c75` (loading and load-error states,
refetch error over the cached value with unsaved edits kept, update-only
custom role saves over the API, row order, the documented size limit,
rapid organization switching, double-click save).
- CI: rebased onto `main` (`98332c739b0`) as `3850fbaa922`; other commit
SHAs in this description are from before the rebase. Every check passed
on `3850fbaa922`, with `test-go-pg` green on the first attempt. After
the review fixes in item 7, `main` was merged in as `006c4951749` to
resolve new conflicts in generated files and renumber the migration to
`000613`. Every check passes on `006c4951749`. Before the rebase, every
check passed on `b4e76543df2`. `test-go-pg (ubuntu-latest)` passed on
its fourth attempt; the first three failed only the `cli` package
goroutine leak check, with every test passing. Two of those leaked a
coderd instance, which also happens intermittently on `main`:
`TestServer/Logging` cancels `coder server` during startup, and an early
return in `cli/server.go` skips closing coderd. That fix is in #30280.
The first attempt leaked an idle HTTP connection instead (not
investigated).

## Screenshots

From remote UAT on a test deployment (test data only).

| Organization admin saves instructions (`b027cb5ca67`) | Auditor sees
read-only instructions, no save actions (`b027cb5ca67`) |
|---|---|
| ![Organization admin saves organization instructions and sees a
success
toast](https://gist.githubusercontent.com/ibetitsmike/1027eba97c252d6f3e921d2f46debd01/raw/5a70872eae4170b4e9330c07696727071f0962e8/r1-admin-saved.png)
| ![Organization auditor sees the instructions
read-only](https://gist.githubusercontent.com/ibetitsmike/1027eba97c252d6f3e921d2f46debd01/raw/5a70872eae4170b4e9330c07696727071f0962e8/r1-auditor-readonly.png)
|
| **Loading state while the first fetch is pending (`618bff67c75`)** |
**Refetch error shown above the field; the unsaved edit is kept and Save
stays enabled (`618bff67c75`)** |
| ![Loading spinner with no textarea or
buttons](https://gist.githubusercontent.com/ibetitsmike/1027eba97c252d6f3e921d2f46debd01/raw/5a70872eae4170b4e9330c07696727071f0962e8/r2-loading.png)
| ![Error alert above the instructions field with the unsaved edit
preserved](https://gist.githubusercontent.com/ibetitsmike/1027eba97c252d6f3e921d2f46debd01/raw/5a70872eae4170b4e9330c07696727071f0962e8/r2-refetch-error-edit-kept.png)
|

## Review record

1. Codex,
[review](https://github.com/coder/coder/pull/30272#pullrequestreview-5387245297)
on `c42b5f9e05c`: 6 findings. Fixed the missing loading state, refetch
failures hiding cached instructions, and two docs style issues
(`58eab5603a4`, `cc73b714630`). Two were first declined in error, then
corrected: the read-only test now checks that typing leaves the value
unchanged (`cc73b714630`), and update-only custom roles no longer get a
500 on save (`618bff67c75`).
2. Independent security review by a clean-context agent,
[comment](https://github.com/coder/coder/pull/30272#issuecomment-5943505041)
on `c42b5f9e05c`: no P0 to P2 findings. A stale P3 comment was fixed in
`34c17aadb7f`; returning `err.Error()` in 500 details was kept to match
the deployment prompt handlers.
3. Codex,
[review](https://github.com/coder/coder/pull/30272#pullrequestreview-5387323380)
on `58eab5603a4`: 2 findings (refetch errors not surfaced, docs line
breaks), both fixed in `cc73b714630`.
4. Codex, blockers only,
[review](https://github.com/coder/coder/pull/30272#pullrequestreview-5387462070)
on `618bff67c75`: 1 P2 (a failed refresh right after a successful save
shows the previous value). Not a blocker, deferred.
5. Final fresh advisory by a clean-context agent,
[comment](https://github.com/coder/coder/pull/30272#issuecomment-5944285083)
on `618bff67c75`: **READY**, no blockers and nothing to cut. It
independently checked authorization, audit, the migration, chatd row
insertion and subagent inheritance, the tests, and soft-deleted
organization handling, and agreed with every deferral. No code changes
followed: the snapshot behavior it suggested documenting is already
documented, and its docs wording note joins the deferred docs item.
6. Codex,
[comment](https://github.com/coder/coder/pull/30272#issuecomment-5952100918)
on `b4e76543df2` (the docs-only follow-up below): no major issues. Mike
asked for a passing Codex review loop after item 5, so this review goes
past the planned stop at the final advisory. No code changes followed.
7. ethanndickson,
[review](https://github.com/coder/coder/pull/30272#pullrequestreview-5409332502)
on `3850fbaa922`: approved with 4 comments. Fixed 3: the audit target is
now empty (`91e77d3fa8c`), the docs no longer re-explain audit logs
(`a0fb6891af5`), and the organization instructions stories went from 9
to 5 (`e836097f93f`). Not fixed: the organization prompt read fails open
on a database error, like the deployment prompt, and ethanndickson is
changing both after merge
([thread](https://github.com/coder/coder/pull/30272#discussion_r4180191267)).
UAT did not rerun for these changes.

Not counted as reviews: the pre-PR frontend self-review, simplify pass,
and test audit (`de8377e94e1`), and a second test audit of
`de8377e94e1..cc73b714630` (pass with nits; the update-only test case
was added after it).

Docs-only follow-up after the advisory, requested by Mike: `b4e76543df2`
adds audit log guidance for prompt changes to the platform controls page
and an update-only custom role note to the organizations page. Codex
reviewed it (item 6). No UAT covers it, and the code is unchanged since
`618bff67c75`. The deferred claim that the Instructions page path is
outdated was wrong, so it's dropped. Item 7 removed the audit log
guidance again; the custom role note stays.

## Soft-deleted organizations

UAT found that a soft-deleted organization's prompt stays readable and
writable by UUID. This is inherited behavior, not new exposure:
`ExtractOrganizationParam` resolves UUIDs without a deleted filter, so
the sibling organization chat routes (MCP servers, model overrides,
models) behave the same. The same `chat_model_config` RBAC still
applies, nobody gains access to another organization, and writes are
audited.

## Deferred (owner: Mike)

- A failed refresh right after a successful save shows the previous
value with an error alert, so a further edit from that view starts from
the old text (Codex round 4, UAT round 2). Trigger: reports of stale
values after saving on unreliable networks.
- Update-only custom roles can save over the API, but the card stays
hidden without read access. Trigger: a customer uses such a role.
- Soft-deleted organization access by UUID, above. Trigger: the platform
starts rejecting deleted organizations in `ExtractOrganizationParam` or
purging their configuration.
- A contrived "code word" instruction was refused as possible prompt
injection; neutral instructions work. Trigger: reports of ignored
organization instructions.
- UX polish: there is no retry action after a failed first load, the
save toast can cover Cancel, there is no unsaved-changes guard, the
oversize error omits the limit, and whitespace-only text clears the
prompt without notice. Trigger: the next settings UX pass.
- Docs: the 413 response and organization name paths are undocumented.
Trigger: the next Coder Agents docs pass.
- Test audit nits: a redundant `waitFor` and an optional read-only plus
refetch story. Trigger: the next change to these tests.

> Xum, an AI coding agent, implemented, tested, and opened this PR on
behalf of @ibetitsmike.
2026-10-06 10:48:07 +02:00
Garrett DelfosseandClaude Opus 5.5 a08ba6a70c feat: add shared project memory API for chats in a project (#29301)
- Experiment, stacked on #29242. UI is in #29583.
- Adds **project memory**: `chat_project_memories` rows with a name
(lowercase slug, unique per project), a one-line description (150
characters) and a body (8KB). Up to 200 per project, shared by every
root chat in the project. Chats outside a project and subagents have no
memory.
- Memories are immutable: they are created and deleted, never edited.
Agent tools are `read_memory`, `save_memory`, `delete_memory` and
`consolidate_memory`. Saving an existing name fails and points the agent
at `consolidate_memory` to replace it.
- **Consolidation is agent-driven, in the turn**, modeled on how Claude
Code compacts its memory index when warned it is near its limit. From
160 memories (80%), save results ask the agent to get under 140 (70%)
with `consolidate_memory`, choosing what to merge and drop. That tool
deletes and saves memories in one transaction under the per-project
advisory lock, and rolls back entirely if a delete is missing, a name is
taken, or the result exceeds 200. There is no background extraction or
consolidation.
- **The memory index reaches the model as model-only conversation
messages**, not in a tool description or the system prompt. At turn
start the chat gets a full `<project-memory-index>` snapshot when its
prompt has none (first turn, or after compaction), otherwise a
`<project-memory-index-update>` listing only what changed. Tool
definitions and the system prompt carry no memory state, so memory
writes never invalidate the provider's cached prompt prefix. Writes by
other chats, or by this chat mid-turn, reach the model at its next turn.
- REST API (create, list, get, delete) under
`/api/experimental/organizations/{organization}/chats/projects/{project}/memories`.
Memory permissions follow projects onto the `agents-access` role.
- **Every memory change is audited** as `ChatProjectMemory` create or
delete. REST changes are attributed to the caller. Tool changes
(`save_memory`, `delete_memory`, `consolidate_memory`) are attributed to
the chat owner, carry the `chat_id`, and record one entry per memory
after the change commits, so a rolled-back consolidation records
nothing.

### Examples

**What the model sees.** The system prompt gets a fixed guidance block
with no memory state:

```text
<memory>
Memory is durable context shared by every chat in the project "platform".
Save facts that will matter in future chats: … The memory index is in the conversation: a <project-memory-index> message lists every saved memory, and <project-memory-index-update> messages at the start of later turns list what changed. Changes made during a turn, including your own, appear in the next turn's update. … Memories cannot be edited: replace one that no longer holds with consolidate_memory, or delete it.
</memory>
```

On the first turn, after the user's message, the chat gets a model-only
row with the full index (hidden in the UI):

```text
<project-memory-index>
- deploy-day: Deploys happen on Tuesdays
- release-owner: Alice owns releases; ping her before tagging
</project-memory-index>
```

On a later turn where memories changed, it gets only the difference:

```text
<project-memory-index-update>
Added or changed:
- deploy-day: Deploys happen on Wednesdays (changed 2026-09-30)
- staging-url: Staging lives at staging.example.com
Removed:
- old-ci-notes
</project-memory-index-update>
```

Resulting request layout, with the provider's cached prefix unchanged by
memory writes:

```text
[tools: fixed]  [system: fixed <memory> guidance]  [turn 1 user] [index snapshot] [assistant/tools …]  [turn 2 user] [index update] [assistant …]
```

**Tool calls and results** (produced by the tools in this PR):

```text
save_memory {"name":"staging-url","description":"Staging lives at staging.example.com","body":"Staging is https://staging.example.com; deploys from main."}
→ {"count":3,"saved":"staging-url"}

save_memory {"name":"deploy-day", …}   // name already exists
→ error: a memory named "deploy-day" already exists; read it, then replace it with consolidate_memory by deleting "deploy-day" and saving the new version in one call

consolidate_memory {"delete":["deploy-day"],"save":[{"name":"deploy-day","description":"Deploys happen on Wednesdays (changed 2026-09-30)","body":"The team deploys every Wednesday since 2026-09-30."}]}
→ {"count":3,"deleted":["deploy-day"],"saved":["deploy-day"]}

read_memory {"name":"deploy-day"}
→ {"name":"deploy-day","description":"…","body":"…","created_at":"2026-09-30T15:04:05Z","created_by":"alice"}
```

Near and at the cap:

```text
save_memory … → {"count":160,"saved":"one-more","warning":"memory is 160/200, approaching the limit. Consolidate it to under 140 memories now with consolidate_memory: merge overlapping memories into one, and drop stale, wrong, or superseded ones."}

save_memory … → error: memory is full (200/200); use consolidate_memory to merge overlapping memories and drop stale ones, and include this memory in its save list
```

**REST memory object** (`GET .../chats/projects/{project}/memories`):

```json
{
  "id": "59500000-0000-4000-8000-000000000001",
  "project_id": "7c0e4a3e-5b1f-4f7d-9a51-2f3c8f6d2b10",
  "organization_id": "bb640d07-ca8a-4869-b6bc-ae61ebb2fda1",
  "name": "deploy-day",
  "description": "Deploys happen on Wednesdays (changed 2026-09-30)",
  "body": "The team deploys every Wednesday since 2026-09-30.",
  "created_by": "0ed9befc-4911-4ccf-a8e2-559bf72daa94",
  "created_by_username": "alice",
  "created_at": "2026-09-30T15:04:05Z"
}
```

Create takes `{"name", "description", "body"}` and answers 409 on a
duplicate name or at the cap.

<details>
<summary>Screenshots (predate the immutable-memory rework; tool output
shape has since changed)</summary>


![save_memory](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr2-01-chat-saves-memory-raw.png)

![read_memory](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr2-02-chat-reads-memory-raw.png)
![Agent reads every
memory](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr2-agent-lists-memories-raw.png)

</details>

---
Generated with Coder Agents on behalf of @f0ssel.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 12:52:10 -04:00
Mathias Fredriksson 0f78361750 fix(coderd): cancel tool calls in messages deleted on edit (#30194)
Editing a message removes it and every later message from the chat.
Before, a tool call in those messages kept going in the workspace: a
foreground command ran to completion, and a file edit or write could
still apply, while the model no longer knew about it.

Now chatd sends the workspace agent a cancel request for each such
call before the new turn's first model call. Calls that already
returned, such as background processes the model was told about, keep
running.

Fixes CODAGT-1222
2026-10-02 22:36:59 +03:00
Thomas Kosiewski fb0951b850 feat: add the manage_automations chat tool switch and read actions (#30214)
Adds the `manage_automations` chat tool with its read and stop actions
(`list`, `get`, `disable`, `delete`), gated by an interim per-chat
switch. Tool sets have not landed, so this ships
`chats.manage_automations_enabled` as the RFC describes for that case;
tool sets later replace it and migrate its values.

Part of CODAGT-1212 (slice 12a of 3). Part of CODAGT-1135.

## What changes

- **Switch.** Migration `000611` adds `chats.manage_automations_enabled`
(default off). It is set through `POST /api/v2/chats` and `PATCH
/api/v2/chats/{chat}`. After creation only the chat owner may send the
field (403 otherwise). Turning it on returns 400 when the
`chat-automations` experiment is off for the chat owner (POST checks the
resolved `owner_id`, not the caller) or when the chat is a sub-agent
chat. Turning it off is always accepted. The audit table tracks it.
- **Offering.** The tool is offered only in root chats, outside plan
mode and explore mode, with the switch on, and with the experiment on
for the chat owner (decided once per turn, like `mcp-tool-search`). It
is not on the plan-mode or explore allowlists, and sub-agent chats are
created with the switch off.
- **Per-call recheck.** Every call reloads the chat and checks all rules
again, with a fresh experiment evaluation, before doing anything.
- **Scoping.** Owner and organization come from the chat, never from
arguments. Reads run as the owner through a new query that filters on
organization and owner. Other members' automations are "not found", even
when the chat owner is an organization admin.
- **Automation-reached turns.** Detected at call time from the full
history (compaction replays user rows without `automation_id`, so the
prompt window is not enough). In such turns only automations that target
the chat or created it (`chats.automation_id`) are visible.
- **Visibility of prompts and deletes.** `list` leaves out prompts (tool
results stay in the chat, which may be shared); `get` returns the prompt
of the automation it names. In a turn an automation reached, `delete`
only removes the triggering automation; the others can still be
disabled.
- **Audit.** `disable` and `delete` go through the management service
and write a background audit entry with `chat_id`, plus the triggering
`automation_id` and `input_id` in reached turns. Results use the API
shape and never contain a secret or secret hash.

`create`, `update`, `enable`, and `run_now` come in the next PR and
return "not available in this version" here.

Follow-up: CODAGT-1270 (bound the history read that reached-turn
detection does on each call).

## Size

About 1,380 hand-written lines: about 460 production, 680 tests, and 240
SQL that has to be there (the `chats_expanded` view rebuilt in both
migrations and the 17 inline column lists). Generated: +649/-280 (sqlc,
dbmock, dbmetrics, dump.sql, apidoc, API and audit docs,
`typesGenerated.ts`).

## Validation

- Unit and DB tests for every rule above (switch 403/400 cases, offering
rules, per-call recheck, owner/org scoping with an org-admin owner,
reached-turn detection including compaction replay, audit fields, secret
scan), plus a separate test audit. Full `make pre-push` passed before
push.
- Remote dogfood UAT passed on this slice: switch API cases, tool absent
with the switch off, in plan mode, in sub-agents and with the experiment
off for the owner (checked in the server's per-step tool lists),
list/get/disable/delete, and a secret scan of all tool output. Audit
entries are verified in Go tests only.

> [!NOTE]
> ARCHITECTURE.md prose was drafted by an agent at the user's request;
the user will review it.

## Stack

Stacked on the Automations for Coder Agents stack (30085 ... 1214b).
Next: slice 12b (write actions), then 12c (composer toggle).

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-02 15:54:55 +00:00
Thomas Kosiewski 8620361fcc feat(coderd): run chat automation schedules (#30160)
Part of CODAGT-1211. Part of CODAGT-1135.

> Stack: PR 7a of "Automations for Coder Agents", on top of 1210 5b
(#30134, `tk/codagt-1210b-automations-new-chat`). Run now (`POST
.../runs`) is split into PR 7b (#30166) on top of this one, which closes
CODAGT-1211.

Schedule automations now run. Every coderd instance scans for due
schedules at start and every 30 seconds, and publishes each occurrence
through the existing `Publish` paths (existing_chat `SendMessage(queue)`
and new_chat `Create`, with their `AdmitInTx` callbacks). There is no
second admission path.

## How a scan works

1. An unlocked read returns enabled schedule automations whose cursor
(`schedule_next_run_at`) is due, in pages of 500 by (cursor, id) until a
short page. It leaves out inactive or deleted owners and existing_chat
targets that are gone or archived.
2. The `chat-automations` experiment is evaluated once per owner per
tick, outside any callback. Owners with it off are dropped without a
cursor write. `Publish` checks the experiment for the owner again before
the transaction (webhooks map that refusal to 404, like the handler's
own check).
3. A cursor more than 60 seconds (the grace window) old is missed: a
guarded write moves it to the next future time, nothing is published,
and nothing is replayed.
4. Otherwise `Publish` runs (up to eight occurrences at once, so one
slow hook or lock wait does not hold back the rest) with the observed
revision and cursor. The pre-check refuses a stale or expired occurrence
before any hook runs. Under the chat and automation locks, the callback
requires the same revision and cursor, reads the clock after the locks,
requires the occurrence to be due and within grace, and moves the cursor
to the first cron time after that read in the same transaction.
5. A busy chat with `skip`, a full queue or automation share, a hook
denial, lost permission, or an occurrence that expired while waiting for
a lock skips the occurrence with the same guarded write and an info log.
A stale, disabled, or deleted automation is left alone. Other errors
keep the cursor, so the next scan retries within grace. That includes an
experiment-off refusal from `Publish`, because the evaluator also
reports a failed read as off.

New chats created by a schedule are titled with the occurrence's
scheduled time in the schedule's time zone, for example `Tokyo digest
2026-09-30 08:14 JST` (the ticket says acceptance time; the two differ
by at most the grace window, and the scheduled time cannot drift across
a minute while hooks or locks run). Webhook titles stay in UTC. Each
such chat gets an audit entry as created by the automation owner, with
the automation and input ids, like the chats the webhook endpoint
creates.

## Decisions reviewers should check

- **Clock (deviates from the ticket's "fresh database clock").** The
callback reads the injected quartz clock after the chat and automation
locks are held, not transaction-start `now()`. The ticket also requires
fake-clock tests, and the lower layers already read the clock after
locks. Exactly-once rests on the revision and cursor tuple, not on time.
Inter-instance clock skew cannot cause a double acceptance or a replay:
an acceptance commits only together with the conditional cursor write
under the automation lock, and every cursor write moves the cursor
strictly forward (`advanceAutomationSchedule` refuses a next run at or
before the cursor). A clock that is off by `d` only shifts the
acceptance window to `[cursor-d, cursor+grace+d]` in true time; a clock
that runs ahead can skip occurrences within `d`. See the comment on
`checkAutomationOccurrence` and the `CursorOnlyMovesForward` test.
Swapping in `clock_timestamp()` later is a one-line change.
- **Authorization of the cursor write.**
`AdvanceChatAutomationScheduleCursor` follows the object pattern of
`ConsumeChatAutomationWebhookByID` and `UpdateChatAutomationByID`: the
dbauthz wrapper loads the automation and authorizes `update` on it. The
scanner and the callback call it as chatd, so the chatd subject gains
`update` on `chat_automation` (it already had `read`). This is the
smallest grant; chatd still cannot create or delete automations (pinned
in `TestAsChatd`), and owner-only changes stay in the chatd automation
service. No 1207 migration is touched.
- **Narrow write.** The query sets only `schedule_next_run_at` and
`updated_at`, guarded by `id`, `kind = 'schedule'`, `enabled`, the
observed `schedule_revision`, and the observed cursor. It never touches
owner-controlled fields or the revision.
- **Known limitation (CODAGT-1234).** Two replicas that read the same
due row can both reach the `user_prompt_submit` hook before one loses
under the lock; only one message or chat commits. This matches the lower
layer's best-effort unlocked pre-check for webhooks. A claim-first
cursor move would break the ticket's retry-within-grace rule or the
forward-only cursor, so a lease column is proposed as a follow-up.
- The schedule tests run their servers through dbauthz, so they use the
chatd subject's real permissions (they fail without the grant).

## Tests

`coderd/x/chatd/automation_schedules_internal_test.go` uses a mock
clock, real Postgres, and a second connection that holds row locks
(waiting on `pg_stat_activity`): two instances accept one occurrence
(existing_chat, and new_chat with no orphan chat); busy `skip`,
automation share full, and queue full each advance the cursor once; a
cron edit rolls back an in-flight acceptance; restart after missed
intervals publishes nothing, restart within grace accepts once; previews
match accepted runs; an occurrence that expires while waiting for a lock
is not accepted; a DST change keeps 09:00 wall-clock time; the
experiment off gets no run (scan and Publish); the cursor only moves
forward; the loop scans at start.

Dogfood UAT ran on a real dev instance through Coder Agents and passed:
a one-minute schedule delivered one message per minute with no browser
open, a busy chat skipped its occurrence, disable stopped runs and
re-enable did not replay, new_chat titles used Asia/Tokyo, two coderd
instances produced no duplicates, and downtime was not replayed.

## Size

About 994 hand-written lines (about 560 of them tests). Generated:
`queries.sql.go`, `querier.go`, `dbmetrics`, `dbmock` (181 lines).

ARCHITECTURE.md prose was drafted by an agent at the user's request; the
user will review it.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-02 15:54:52 +00:00
Thomas Kosiewski 1e1e884c25 feat(coderd): create a chat for each webhook event of a new_chat automation (#30134)
Closes CODAGT-1210. Part of CODAGT-1135.

> Stack: PR 6 of "Automations for Coder Agents", on top of #30127
(CODAGT-1210 part 1, webhook events for `existing_chat`). This PR
delivers webhook events to `new_chat` automations and replaces part 1's
`501` for them.

## What this adds

- A webhook event for a `new_chat` automation creates a new chat as the
automation owner through chatd `CreateChat` with an `AdmitInTx`
callback, and the endpoint answers `202` with the new `chat_id`.
- The chat uses the automation's model config and reasoning effort and
client type `api`, and selects no MCP servers (the organization's Force
On servers still apply). Its title is set explicitly to the automation
name plus the acceptance time in UTC (`<name> 2006-01-02 15:04 UTC`), so
automatic title generation does not run and event data never becomes the
title.
- Before the create, the same input checks as part 1 run (enabled,
secret version, single-use unused), plus: the owner may create a chat in
the organization (`403`), and the owner can read the automation's model
config, which must be enabled and in the same organization (`409`). A
`user_prompt_submit` denial creates no chat.
- Under the automation lock, the callback rechecks all of that, sets
`chats.automation_id` on the new chat (new query
`UpdateChatAutomationIDByID`, dbauthz: update on the chat), consumes a
single-use webhook, and returns the provenance. A refusal rolls back the
whole creation.
- The events endpoint records an audit entry for each created chat
(action create, user = the automation owner, with `automation_id` and
`input_id` in the additional fields), as chat creation through the chat
API does.
- Part 1's lock and consumption code is shared by both target modes.

## Notes for reviewers

- ARCHITECTURE.md prose was drafted by an agent at the user's request;
the user will review it.
- Size: about 600 hand-written lines (about 380 of them tests).
Generated: about 65 lines (`coderd/apidoc`, `dbmock`, `dbmetrics`,
`querier.go`, `queries.sql.go`).
- Dogfood UAT ran through Coder Agents on a real dev deployment: curl
deliveries created chats with the explicit UTC title, client type `api`,
two message parts, and `automation_id` set; single-use concurrency
created exactly one chat; a disabled model config gave `409` with no
chat.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_

---------

Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-10-02 15:54:51 +00:00
Thomas Kosiewski b517db4beb feat: deliver webhook events to existing chats through chat automations (#30127)
Part of CODAGT-1210 (1 of 2). Part of CODAGT-1135.

> Stack: PR 5 of "Automations for Coder Agents", on top of #30117. The
ticket is split in two by operation: this PR delivers webhook events to
`existing_chat` targets; the stacked follow-up delivers them to
`new_chat` targets and closes CODAGT-1210. Until then a `new_chat`
automation's events endpoint answers `501`.

## What this adds

- `POST /api/experimental/chat-automations/{automation}/events`.
External systems send the automation's webhook secret as `Authorization:
Bearer <secret>` and a JSON body of at most 256 KiB. The route has no
API key middleware and sits outside the `/api/experimental` group,
behind its own rate limiter keyed by caller and a fixed endpoint key (IP
address for unauthenticated callers), so varying the automation id does
not reset the budget.
- Secret check: SHA-256 of the bearer value, compared in constant time.
Unknown ids, other kinds, and wrong secrets all get the same `401`. The
`chat-automations` experiment is checked for the automation owner only
after the secret matches (`404` when off). The body is then read: over
the limit `413`, not JSON `400`.
- `chatd.PublishAutomationWebhook` sends as the owner (`dbauthz.As` with
the owner's subject) through `SendMessage` with the `queue` busy
behavior set explicitly, so automations never interrupt or steer.
`user_prompt_submit` hooks run first: a denial is `403`, an unreachable
hook service `502`.
- One admission check runs before the send (so refused events never
reach hooks) and again in the `AdmitInTx` callback, which locks the
automation together with the chat's queued automations in one
ascending-order call. Under the lock it rechecks: enabled, secret
version (a request verified before a rotation but committed after it
gets `401`), single-use state, target chat (root chat of the owner, same
org, not archived), owner status, and the owner's `chat:update`
permission. It then applies **When busy**: `skip` refuses a busy chat
(`409`), `queue` refuses input beyond the automations' share, half the
chat queue cap rounded down and at least 1 (`429`); busy and the share
count only queued rows that promotion would still deliver. It marks a
single-use webhook consumed and returns the provenance.
- The message has two parts: the saved prompt, and a labeled part with
the body as untrusted event data inside `<automation_event_data>` tags.
`json.HTMLEscape` escapes `<`, `>`, and `&` in the body, which keeps the
JSON value and makes the closing tag impossible to forge.
- `chatd.Config.Authorizer` (wired from `options.Authorizer`) for the
explicit owner checks; publishing fails closed without one.
- New query `ConsumeChatAutomationWebhookByID` with a dbauthz wrapper.

Status codes follow the RFC's failure outcomes table; the
chat-without-a-model case returns the same `400` as a person's message.

## Notes for reviewers

- Single-use consumption runs from the chatd publish code as the owner,
because the chatd role only reads `chat_automation`. No lower-layer RBAC
change.
- ARCHITECTURE.md prose was drafted by an agent at the user's request;
the user will review it.
- Size: about 1,460 hand-written lines after review fixes (about 770 of
them tests), over the ~1,000 budget because of the review hardening.
Generated: about 360 lines (`coderd/apidoc`, `dbmock`, `dbmetrics`,
`querier.go`, `queries.sql.go`, `docs/reference/api/schemas.md`,
`typesGenerated.ts`).
- Dogfood UAT ran through Coder Agents on a real dev deployment with
curl deliveries. Round 1 found error-mapping defects, fixed here; rounds
2 and 3 passed, the last on the final code (the head after it changes
only ARCHITECTURE.md).
- Deferred: CODAGT-1233 (stale automation rows filling the physical
queue cap; lower-layer `SendMessage` path).

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-02 15:54:51 +00:00
Thomas Kosiewski 5c1662ea4c feat: disable, re-enable, and rotate chat automations with schedule preview (#30117)
Completes the chat automations management API from RFC step 3: disable
and re-enable, delete that disables first, webhook secret rotation, and
a server-side schedule preview. Stacked on #30112.

- **Disable** (`PATCH` with `{"enabled": false}`) runs in two steps. A
short transaction that touches only the automation row sets `enabled =
false` and increments `queue_generation`, the cutoff. After it commits,
the automation's queued rows below the cutoff are deleted one chat at a
time through chatd's existing `DeleteQueued` transition, run as chatd,
so queue versions and stream updates match a manual delete. If a row is
missed, for example after a crash between the steps, the queue promotion
guard from #30096 discards it, because its generation is stale. Running
turns are not interrupted.
- **Delete** deletes the row and then removes all of the automation's
queued rows the same way. Deleting the row is itself the cutoff, because
the promotion guard discards rows whose automation no longer exists. So
delete needs only delete permission, not update. This differs from the
RFC's "disable, then delete" order, with the same outcome.
- **Re-enable** (`{"enabled": true}`) moves a schedule's cursor to the
next future cron time, so the disabled period is not replayed. The
generation never resets.
- **Permissions:** only the owner can enable, edit, or rotate.
Organization admins and site owners can send exactly `{"enabled":
false}` (anything combined with it is 403), and can still list, read,
and delete.
- **Rotation:** `POST .../chat-automations/{automation}/secret/rotate`
(owner only, webhooks only) returns the new secret once and increments
`webhook_secret_version`. Only the SHA-256 hash is stored, and the
rotation is audited with the hash redacted.
- **Preview:** `POST .../chat-automations/schedule-preview` evaluates an
unsaved cron expression and time zone with create's rules and returns
the next five run times, or 400.
- `coderd/x/chatd/ARCHITECTURE.md` gets a `TODO (CODAGT-1209)` note
about disable using the delete-queued transition. **Author action before
merge:** replace that TODO with the architecture prose (agents do not
write that document).

Known gap, tracked in CODAGT-1227: when the target chat is archived,
disable and delete cannot remove its queued automation rows, because
`DeleteQueuedMessage` is not allowed from archived states. The rows
never run, since the promotion guard discards them, but they stay
visible after the chat is unarchived. The fix needs a chatstate
transition from the lower layer.

Dogfood UAT ran on a real dev instance and passed. It covered disable
removing only that automation's queued rows (the chat API agreed),
delete, re-enable without replay, the permission matrix for an
organization admin and the site owner, rotation (the stored hash matches
the new secret, and neither secret appears in get or list), preview, and
a runtime experiment rule. It also checked that a leftover row is
discarded on promotion.

Closes CODAGT-1209
Part of CODAGT-1135
Refs CODAGT-1217

Stack: PR 3b of the Automations for Coder Agents stack, on top of #30112
(PR 3a), #30096, and #30085.

> This PR was written by an AI agent (xum) and reviewed by the agent's
own test audit and UAT. A human has not reviewed it yet.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_

---------

Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-10-02 15:54:50 +00:00
Thomas Kosiewski 9316d67440 feat: add chat automations management API behind chat-automations (#30112)
Adds the management API for chat automations under
`/api/experimental/organizations/{organization}/chat-automations`,
behind the new user-scoped `chat-automations` experiment. This is the
first of two PRs for RFC step 3; nothing publishes automation messages
yet.

- **Experiment:** `chat-automations` is in `ExperimentsKnown`,
`ExperimentsUserScoped`, and the ruleguard list in `scripts/rules.go`.
Every route asks `chatd.AutomationsEnabled(ctx, api.ExperimentEvaluator,
callerID)`, which fails closed on a missing subject or evaluator; the
routes stay mounted and return 404 when the experiment is off for the
caller, so a runtime rule for one user applies on the next request
without a restart. There is no `/api/v2` alias.
- **Routes:** list, create (201), get, `PATCH`, delete (204), with
Swagger annotations and audit entries. List returns the automations in
the organization the caller can read (members see their own,
organization admins see all).
- **Service** (`coderd/x/chatd/automations.go`): create validates kind
and target shape, a non-empty prompt, a five-field cron and IANA time
zone, model access, and that an `existing_chat` target is a root chat of
the owner in the same organization. It defaults `when_busy` to `skip`
for schedules and `queue` for webhooks and computes the first schedule
cursor. `PATCH` runs the same per-field validation on each changed
field, rejects `kind` and `target_mode`, bumps `schedule_revision` on
prompt, cron, or time zone changes, and recomputes the cursor on cron or
time zone changes.
- **Owner-only:** only the owner can `PATCH` an automation, even with
organization-admin or site-owner roles (403); admins can list, read, and
delete.
- **Secrets:** a webhook's secret (`coder_automation_` prefix) is
returned once in the create response. Only its SHA-256 is stored, and it
appears in no list, get, audit, or log output.
- **Cap:** new option `--chat-max-automations-per-owner` (default 50)
caps automations per owner across all organizations; creating one more
returns 409. Creates serialize on a per-owner advisory lock.
- **Cron:** `cron.Standard(spec, timeZone)` accepts any standard
five-field expression (the existing `Weekly` rejects day-of-month and
month restrictions). Responses include the next five run times of an
enabled schedule.

PR 3b (stacked on this one) adds disable and re-enable with the queue
cutoff, makes delete run disable first, and adds secret rotation and the
schedule preview route. Until then, the queue promotion guard from
#30096 discards queued rows whose automation was deleted.

Dogfood UAT ran on a real dev instance and passed:
create/list/get/patch/delete via curl, owner-only 403s for an
organization admin and the site owner, the cap (including concurrent
creates and cross-organization counting), a runtime rule turning the
experiment off for one user without a restart, and `/api/v2` 404s.

Part of CODAGT-1209 (PR 3b closes it)
Part of CODAGT-1135
Refs CODAGT-1217

Stack: PR 3 of the Automations for Coder Agents stack, on top of #30096
(which is on top of #30085).

> This PR was written by an AI agent (xum) and reviewed by the agent's
own test audit and UAT. A human has not reviewed it yet.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_

---------

Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-10-02 15:54:49 +00:00
Thomas Kosiewski eaca2290b0 feat(coderd): add chatd automation admission seam and queue promotion guard (#30096)
Adds the backend-only seam that lets a later `Publish` operation admit
an automation message inside chatd's existing create-chat and
send-message transactions, and the guard that drops stale automation
input when the queue promotes it. Only Go test callbacks use the seam in
this PR, so ordinary chat behavior does not change.

Closes CODAGT-1208
Part of CODAGT-1135

> Stack: PR 2, on top of #30085 (schema, RBAC resource, and audit
entries). No migration changes; this PR adds and extends sqlc queries
only.

ARCHITECTURE.md prose was drafted by an agent at the user's request; the
user will review it.

## Changes

- **Admission seam.** `chatstate.AdmitFunc` is a nil-default `AdmitInTx`
field on `chatd.CreateOptions`, `chatd.SendMessageOptions`,
`chatstate.CreateChatInput`, and `chatstate.SendMessageInput`.
`CreateChat` runs it after inserting the chat and before the initial
history, and requires exactly one initial user message. `SendMessage`
runs it on the locked chat before any write. The callback returns an
`AutomationProvenance` (automation ID, input ID, queue generation),
which is validated and stamped on the message. A callback error rolls
back the whole transaction, so no chat, message, or queue row survives.
`Server.CreateChat` and `Server.SendMessage` keep their signatures.
- **Provenance.** `chatstate.Message.Automation` is written to queued
rows (`automation_id`, `input_id`, `queue_generation`) and history rows
(`automation_id`, `input_id`). `InsertChatMessages` and
`InsertChatQueuedMessageWithCreator` gain the matching parameters, and
`messageFromQueuedRow` copies them on promotion.
- **Promotion guard.** `chatstate.LockAutomations` is the only way to
lock automation rows: it locks them `FOR UPDATE` in ascending ID order,
running as chatd. The chatd subject gains read on `chat_automation`. A
queued automation row passes only if its automation exists, is enabled,
and has the row's `queue_generation`. Two helpers apply the check:
- `nextPromotableQueueHead` serves `sendMessageE1`, `FinishTurn`, and
`FinishInterruption`. A head without an automation costs no extra query.
Otherwise it locks every automation in the queue up front, deletes
failing heads until one passes, and leaves rows behind that head alone.
If the queue empties, `FinishTurn` and `FinishInterruption` land in
`waiting`, as from `R0` and `I0`.
- `guardQueuedRows` serves explicit `PromoteQueuedMessage`. A failing
row is deleted and the transition returns `Rejected` without an error,
so the delete commits. `Server.PromoteQueued` then returns
`ErrQueuedMessageNotFound` (HTTP 404) without reordering, substituting a
message, or changing status.
- The steer paths (`DeliverSteerMessages` and the steer batches from
#30073/#30074) add one `guardQueuedRows` call each, whichever PR lands
second.
- **Deadlocks.** `database.IsDeadlockError` is new. `ChatMachine.Update`
reruns the transaction, at most 3 attempts with a fresh publish buffer
each, only when an attempt that took automation locks hit a deadlock
abort. Attempts without automation locks, and nested Updates, never
retry. The `SendMessage` and `PromoteQueued` closures reset their
results so a rerun is safe.
- **Transition matrix.** Adds the outputs the guard can produce, such as
`R1 -> FinishTurn -> W` and `E1 -> SendMessage -> R0`.
- **`ARCHITECTURE.md`.** Describes the admission seam, provenance, the
queue promotion guard, the lock order, and the deadlock retry, and adds
the guard's outcomes to the state diagram.

## Validation

- Go tests on real PostgreSQL cover the ticket's checks:
  - A callback error leaves no chat, message, or queue row.
- A guard rejection deletes only the rejected heads, on every promotion
path and for each failure reason (missing, disabled, disabled with the
same generation, generation mismatch).
- Explicit promotion of a failing row returns not found, and the delete
is committed.
  - Provenance round-trips through queue and history.
  - `LockAutomations` works through dbauthz.
- `Update` retries only when an attempt that took automation locks hits
a deadlock.
- A concurrency test of acceptance, promotion, and disable across mixed
automation IDs sees zero deadlock aborts.
- Mutation checks confirmed that the guard, retry, and permission tests
fail when their behavior is removed.
- Remote dogfood UAT passed on a real dev instance: sending, queueing,
head and explicit promotion, interrupting, and deleting queued messages
all behaved as before.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-02 15:54:49 +00:00
Thomas Kosiewski f55aa6c50d feat: add chat automations schema, RBAC resource, and audit entries (#30085)
Adds the storage and authorization pieces for Automations for Coder
Agents. No route, SDK request type, or chatd code uses them yet, so
nothing is exposed.

This is the bottom PR of the Automations stack (RFC step 1). Later PRs
add the chatd acceptance seam, management routes, webhooks, and
schedules on top of these names and constraints.

**Schema** (migrations 000609, 000610)

- `chat_automations` table with enums `chat_automation_kind`,
`chat_automation_target_mode`, `chat_automation_webhook_use`, and
`chat_automation_when_busy`. It includes `created_by_chat_id`, the
webhook secret hash and version, the single-use `webhook_consumed_at`
marker, the schedule cursor and revision, and `queue_generation`. Named
CHECKs enforce the target and kind shapes from the RFC. A composite
foreign key keeps `new_chat_model_config_id` in the automation's
organization, the same pattern the model override tables use. A trigger
(`chat_automations_chat_organization`) keeps `target_chat_id` and
`created_by_chat_id` in that organization too. A composite foreign key
cannot do that while keeping `ON DELETE SET NULL` on PostgreSQL 13. The
trigger validates a reference only on insert or when it changes, so chat
deletion can still clear both references. Indexes cover the owner and
both chat references for cascades and `SET NULL`.
- Provenance columns with partial indexes and no foreign key to
`chat_automations`: `automation_id`, `input_id`, `queue_generation` on
`chat_queued_messages` (all set or all null), `automation_id` and
`input_id` on `chat_messages`, and `chats.automation_id`.
`chats_expanded` is recreated to include `automation_id`.
- `chat_automation` values for `api_key_scope` and `resource_type`. The
down migration is a no-op because enum values cannot be dropped.

**Authorization**

- New organization-scoped RBAC resource `chat_automation` (create, read,
update, delete). The `agents-access` role grants it for the member's own
automations. It is excluded from the member role's user-level
permissions, so a member without `agents-access` gets nothing. Site
owners and organization admins get it through their existing
all-permissions lists. The owner-only update, enable, and run rule from
the RFC belongs to the management service in a later PR.
- The generated scopes stay out of `externalLowLevel`, so API keys gain
no user-requestable scopes.
- `dbauthz` wrappers for the three queries this PR adds:
`InsertChatAutomation`, `GetChatAutomationByID`, and
`DeleteChatAutomationByID`. `InsertChatAutomation` also requires update
on the target chat, read on the creating chat, and read on the new-chat
model config (its user and group ACL). There is also a
`dbgen.ChatAutomation` helper.

**Audit**

- `enterprise/audit/table.go` entries for `ChatAutomation`, with
`webhook_secret_hash` and `prompt` marked `ActionSecret`. The prompt is
treated as chat content, like chat titles. The schedule cursor,
revision, and queue generation are ignored as internal bookkeeping.
`Chat.automation_id` is tracked.

**Required checks from the ticket**

- Migrations apply and roll back: `TestMigrate` and
`TestMigrateUpWithFixtures` (new fixture for 000609). Dogfood UAT also
applied the migrations on a dev instance, rolled them back, re-applied
them, and compared schema dumps. After rollback, the only differences
from main were the enum values that cannot be dropped and develop.sh's
own `_develop` schema. After re-applying, the only difference from this
branch's `dump.sql` was the `_develop` schema.
- Secret hash marked secret: `Test_chatAutomationSecretFieldsRedacted`
checks the table entries and that an audit diff contains neither the raw
hash, its base64 form, nor the prompt text.

Also covered: `TestRolePermissions` cases for own, other owner, other
organization, and a member without `agents-access`.
`TestChatAutomationShapeConstraints` covers the named CHECKs, the
cross-organization foreign key and trigger, a cross-organization
retarget, and deleting referenced chats. Under `site/`, only generated
files changed (`typesGenerated.ts`, `rbacresourcesGenerated.ts`).

Closes CODAGT-1207
Part of CODAGT-1135

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_

---------

Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-10-02 15:54:48 +00:00
Mathias Fredriksson ff158c0589 feat: preserve user-supplied chat titles (#29486)
Chats created through the API or MCP couldn't be given a title, and
automatic title generation could overwrite a title the user had set,
at creation or by renaming during generation. #30254 tried to keep
renames by comparing title text, but text can't tell a user title
from a generated one that has the same text.

Chat creation over the API and MCP now accepts a title. Each chat
records where its title came from: fallback (derived from the
prompt), generated, or user. A title can only be replaced by a title
from the same source or a higher one, so automatic generation never
overwrites a user's title. Title changes no longer reorder the chat
list, and clients apply them in the order they were written. Chats
created before this change are treated as having fallback titles.

Closes CODAGT-1047
Reverts #30254
2026-10-02 12:58:53 +00:00
Garrett DelfosseandClaude Opus 5.5 7cd1b2040b feat: add chat projects API (#29242)
- Experiment, behind `chat-projects` (off by default). First of three
stacked PRs: #29301 (memory), #29583 (UI). This PR is API only; the
screenshots below are from the UI in #29583.
- CRUD under
`/api/experimental/organizations/{organization}/chats/projects/{project}`.
`GET /api/experimental/chats/projects` lists the caller's projects
across all organizations.
- `project_id` on chat create and as a chat list filter. A chat joins a
project only at creation; there is no move.
- Projects are identified by ID. Names are display labels (64
characters, description 1024) with no uniqueness constraint. An optional
`icon` URL (usually an emoji image under `/emojis`) is capped at 256
characters and audited.
- A user can own at most 100 projects across all organizations; creating
another answers 409. Creates count and insert under a per-owner advisory
lock so concurrent requests cannot overshoot.
- Projects are private to their `owner_id`. Members may only create
their own, and a chat can only be bound to a project its owner owns. An
unreadable or foreign project on chat create answers 404 so IDs cannot
be probed; a project from another organization answers 400.
- Permissions ride on the `agents-access` role next to chats (after
#30004), so whoever can use Coder Agents can use projects. Owners and
org admins reach every project.

### Examples

Create a project in an organization:

```http
POST /api/experimental/organizations/{organization}/chats/projects
{"name": "Platform", "description": "Infra and deploy work", "icon": "/emojis/1f680.png"}
```

```json
{
  "id": "7c0e4a3e-5b1f-4f7d-9a51-2f3c8f6d2b10",
  "organization_id": "bb640d07-ca8a-4869-b6bc-ae61ebb2fda1",
  "owner_id": "0ed9befc-4911-4ccf-a8e2-559bf72daa94",
  "name": "Platform",
  "description": "Infra and deploy work",
  "icon": "/emojis/1f680.png",
  "created_at": "2026-09-30T15:04:05Z",
  "updated_at": "2026-09-30T15:04:05Z"
}
```

List the caller's projects across organizations with `GET
/api/experimental/chats/projects` (an array of the object above). Update
with `PATCH .../chats/projects/{project}` and any of `name`,
`description`, `icon`; delete with `DELETE`, which detaches its chats.

Create a chat in the project, and list only its chats:

```http
POST /api/v2/chats
{"organization_id": "bb640d07-…", "project_id": "7c0e4a3e-…", "content": [{"type": "text", "text": "Plan the deploy"}]}

GET /api/v2/chats?project_id=7c0e4a3e-…
```

At the limit:

```json
409 {"message": "You can have at most 100 chat projects. Delete a project to create another."}
```

<details>
<summary>Screenshots (UI from #29583)</summary>

![Create
project](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr1-01-create-project-dialog.png)
![Sidebar
folders](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr1-02-sidebar-folders.png)
![Project
page](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr1-03-project-page.png)
![Folder
menu](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr1-05-folder-menu.png)
![Another member sees no
projects](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr5-member-sees-no-projects.png)

</details>

---
Generated with Coder Agents on behalf of @f0ssel.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:49:21 -04:00
Thomas Kosiewski 7b40aa7fa6 fix(coderd): keep user chat renames made during title generation (#30254)
Automatic title generation captured the chat title when it was
scheduled, ran the model call, then wrote the generated title with an
unconditional UPDATE. A rename that landed while the model call ran
was silently overwritten.

Add UpdateChatTitleByIDIfTitle, a compare-and-set variant that only
writes when the stored title still equals the title generation started
from. When it matches no rows, generation keeps the current title and
skips recording and publishing the generated one.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-01 17:41:56 +02:00
Thomas Kosiewski d11809f095 fix(coderd): enforce chatd concurrent-agent capacity on every path into running (#30243)
## Problem

The concurrent-agent cap (5 root chats and 10 subagent chats per
deployment by default) could be exceeded. Three paths put more chats in
`running` with a fresh lease than the pool allows:

1. **Interrupting a waiting chat.** A user sends a message with
`busy_behavior=interrupt` to a chat that is waiting for capacity. The
chat becomes `interrupting` while unowned, a worker acquires it without
admission, and `FinishInterruption` promotes the queued message into a
running turn under that ownership.
2. **Taking over an orphaned `requires_action` chat.** A replica dies
while its chat is in `requires_action`, its lease goes stale and another
chat takes the slot. A worker acquires the `requires_action` chat
without admission, and a tool result, an interrupt or the action
deadline returns it to `running` under that ownership.
3. **Heartbeat reviving a stale lease.** A replica loses the database
for longer than the 30 second stale threshold and another replica admits
a second chat into the slot. When the first replica comes back, its
heartbeat upsert renews the stale lease unconditionally, and its runner
keeps generating until the turn ends.

## Root cause

`agentCapacityLimiter.Admit` returned `true` without taking the
admission lock or counting for every status other than `running`. The
count query already treats every owned chat with a fresh lease as
holding a slot, so the shortcut let chats acquire a slot uncounted and
later move back to `running` under that ownership. Separately,
`BatchUpsertChatHeartbeats` had no freshness or ownership condition.

## Fix

- **Admission for every runnable status.** `Admit` now counts for
`running`, `interrupting` and `requires_action`, and a refused pool
skips the rest of its candidates in the pass regardless of status. An
owned chat holds its slot until it releases ownership, so every later
transition back to `running` stays within capacity.
- **Fenced heartbeat renewal.** `BatchUpsertChatHeartbeats` is replaced
by `RenewChatHeartbeats`, which only extends rows that are still fresh
and still own their chat, using `statement_timestamp()` read after the
capacity admission lock is granted. The renewal runs under that lock
with a `lock_timeout` (`HeartbeatLockTimeout`) and a per-tick timeout
(`HeartbeatRenewalTimeout`, a third of the stale threshold). Admission
also waits at most 5 seconds for the lock.
- **Stop runners that lost their lease.** `heartbeatOnce` requests
cleanup of every runner whose lease was not renewed, which cancels its
generation. The chat stays owned by the stale lease until an admitted
worker takes it over. A tick that fails or times out cleans up no
runners and retries on the next tick.
- **Instant Stop on a queued chat.** Requiring admission for
`interrupting` would have made Stop on a chat waiting for capacity hang
until a slot frees. When `Interrupt`, or `SendMessage` with
`busy_behavior=interrupt`, runs on a running chat with no owner, the
transition now applies `FinishInterruption` in the same transaction and
cancels orphaned tool calls. An empty queue lands in `waiting`;
otherwise the queue head is promoted into `running`, still unowned, so
it goes through admission.
- **Queued metric.** `CountChatCapacityQueuedByPool` (behind
`coderd_chatd_agents_queued_for_capacity`) now also counts
`interrupting` and `requires_action` chats without a fresh lease, since
they also wait for admission.

## How it was found

A TLA+ model of capacity admission found a counterexample for each path.
Go tests reproduced all three against the previous code (each failed
with `"2" is not less than or equal to "1"` for a root pool of capacity
1), and the model with the fix enabled passes the safety invariant and
the liveness properties. The model itself is not part of this PR.

## Tests

The three reproductions are kept as regression tests in
`coderd/x/chatd/agentcapacity_regression_internal_test.go`:
`TestAdmission_InterruptWithMessageOnWaitingChatStaysWithinCapacity`,
`TestAdmission_RequiresActionTakeoverStaysWithinCapacity` and
`TestRunnerManager_HeartbeatDoesNotReviveStaleLease`. The original
reproductions failed on the base commit. Restoring the admission
shortcut makes the `requires_action` test fail, and restoring the
unconditional heartbeat upsert, or dropping the runner cleanup, makes
the heartbeat test fail. New coverage also includes
`TestAdmission_StopWaitingChatNeedsNoCapacity`,
`TestRenewChatHeartbeats_OnlyRenewsFreshOwnedLeases`,
`TestRunnerManager_FailedHeartbeatTickKeepsRunners`, and chatstate tests
for the unowned interrupt paths. Existing tests that encoded the old
behavior (`requires_action` chats holding no slot, interrupting chats
always acquirable, unconditional heartbeat renewal) were updated.

## Behavior / API change

- `POST /api/v2/chats/{chat}/messages` with `busy_behavior=interrupt` on
a chat that no worker owns (for example one waiting for capacity) now
returns the promoted message with `queued=false` instead of a queued
message, and the chat stays in `running` instead of `interrupting`. If
the chat already had queued messages, the queue head is promoted and the
new message stays queued.
- `POST /api/v2/chats/{chat}/interrupt` on such a chat lands it in
`waiting` immediately (or in `running` with the queue head promoted)
instead of `interrupting`.
- An unowned `interrupting` or `requires_action` chat (for example after
its replica died) now waits for a free slot before its runner starts,
which also delays enforcing its action deadline while the pool is full.
- New internal chat worker settings `HeartbeatRenewalTimeout` and
`HeartbeatLockTimeout`, defaulting to a third of the stale threshold and
half of that.
- Heartbeat ticks now take the capacity admission advisory lock briefly
on each replica.

## Out of scope / follow-ups

- Interrupting a chat still owned by a dead runner (stale lease,
`worker_id` set) still waits for a slot.
- An admission lock timeout should end the acquisition pass instead of
logging a warning per candidate.
- The per-chat `queued_for_capacity` flag in the chat API still reports
only `running` chats.
- Alert on failed heartbeat ticks.

`coderd/x/chatd/ARCHITECTURE.md` gains TODO lines for the affected
sections; the prose is left for the owner.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-01 16:42:51 +02:00
Thomas Kosiewski cdc87ee047 fix(coderd): recheck auto-archive candidates under the family lock (#30242)
Auto-archive read its candidates without locks and then archived each family without rechecking, so a chat that became active after selection (a new message in the root or a child, a pin, a status change) could still be archived. SetFamilyArchivedInput gains an optional Precondition that runs after the root and every family member row are locked root-first; auto-archive uses it to rerun the candidate filters for that root through the new GetAutoArchiveInactiveChatCandidateByID query and skips chats that no longer qualify. A parity test keeps the batch and by-ID queries in sync, regression tests cover root, pin and child activity races, and ARCHITECTURE.md describes the recheck. Builds on #30240.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-01 16:40:24 +02:00
Danielle Maywood 63a95d87f3 feat(coderd): key chat diff statuses by ref (#29081)
Part 1 of 6.

`chat_diff_statuses` had one row per chat, so a chat could only track
one PR, and switching branches overwrote the old branch's PR data. The
primary key is now `(chat_id, git_remote_origin, git_branch)`, so each
ref the agent reports gets its own row. The API still returns a single
`diff_status`: the first row of the chat's statuses, ordered by
`updated_at`, newest report first. The primary ref changes only when the
agent reports a ref: refreshes, claims, backoffs and URL writes do not
advance `updated_at`, so they cannot reorder the primary.

The gitsync worker now claims, backs off and clears rows by the full
key, not the whole chat. When the diff GET discovers a PR URL, it
updates the existing ref row and no longer inserts one. Telemetry picks
one row per chat.

**NOTE**: Migration `000606` swaps the primary key under an ACCESS
EXCLUSIVE lock. The table is small. Existing rows already fit the new
key, so no data is rewritten. The down migration keeps only the primary
row per chat. Stop old coderd instances before migrating: their `ON
CONFLICT (chat_id)` upserts fail once the old key is gone.

Generated by Coder Agents.
2026-10-01 10:59:27 +01:00
Thomas Kosiewski a0b00b5e4b feat: store experiment rules with revisions and audit types (#29898)
Slice 2 of 4 for runtime experiment rules: persist one rule per
experiment in `site_configs`, with revisions that only increase, and add
an audit record type. Nothing calls this code yet, so behavior does not
change.

## What's in it

- **Queries** in `siteconfig.sql`, all checked by dbauthz against
`ResourceDeploymentConfig`:
- `GetExperimentRules` (read) lists `experiment_rule:` keys with
`starts_with`, not `LIKE`, because `_` is a `LIKE` wildcard.
  - `GetExperimentRule` (read) reads one row.
  - `UpsertExperimentRule` (update) writes one row.
- **`experiments.WriteRule(ctx, db, actor, ex, rule,
expectedRevision)`** validates the rule first, then does everything in
one `READ COMMITTED` transaction:
  - It takes an advisory lock for that experiment, waiting at most 5 s.
  - It reads the current row; a missing row counts as revision 0.
- It returns `*RevisionConflictError` when the expected revision is
stale.
- An identical mode and condition writes nothing and returns
`changed=false`. The first `inherit` on a missing row is a change and
stores revision 1.
- Otherwise it writes `revision+1` with the actor and time, and returns
the old and new rule as read and written inside the transaction.
- **`experiments.NewDBStore(db)`** implements the evaluator's `Store`.
It reads rules by prefix, plus the user's id, username, email, explicit
site roles, member org names, and `<org>/<group>` group names (including
`<org>/Everyone`). It runs as the system.
- **Audit:**
- A new `resource_type` enum value `experiment_rule` (migration
`000602`; the down migration is a comment because PostgreSQL cannot drop
enum values).
- A new struct `database.ExperimentRule`. `experiments.AuditRecord`
derives its ID from the experiment name, so audit history groups by
experiment.
- In the audit table, `experiment`, `mode` `revision` and `condition`
are tracked (`ActionTrack`), so the audit diff shows the old and new
condition. In Enterprise, the slog audit backend also writes that diff
to the server log. Admins and auditors can already read the condition
through the rules API.
  - The audit page filter includes the new resource type.

## Testing

- `WriteRule` and `NewDBStore` tests run against real PostgreSQL. They
cover stale revisions after a reset and a recreate, two concurrent
writers with the same expected revision (exactly one wins), no-ops, and
rollback after a forced error.
- dbauthz method tests and an audit diff test that shows both condition
texts.

Remote dogfood UAT through Coder Agents passed the upgrade from slice 1
(existing state kept, enum value added), a fresh install, unchanged
`/api/v2/experiments`, and the `WriteRule` tests under `-race`. The
licensed audit-page check could not run: the test workspace has no
premium license. That round tested `015979ca96`. The commit after it
only adds `AuditRecord`, and a follow-up UAT round will cover the final
head.

> [!NOTE]
> Migration `000604` was the next free number when this was written.
Recheck it before merging.

## Stack

1. #29869: evaluator library
2. **This PR:** storage and audit types
3. Enforcement and discovery (`/api/v2/experiments`, meta tag, chatd
`mcp-tool-search`, ruleguard)
4. Control plane (experimental API, `coder exp experiment-rules` CLI,
docs)

> [!NOTE]
> This PR was written by an AI agent (Xum).

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_

---------

Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-09-29 12:22:59 +00:00
Michael Suchacz 756ffb97af feat: add a collapse_assistant_steps user preference (#29416)
## Stack context

Bottom of a six-PR stack that adds the **Collapse assistant steps** chat
preference (Agents > Settings, off by default). With it on, each turn's
consecutive tool-step rows fold into one disclosure: "Worked for 2m 14s
(7 steps)" once the turn completes, or a ticking "Working for 12s" while
it runs. The stack was split out of #29115 for review size; #29115 now
sits at the top with the full feature description and the UAT record.

1. #29416 feat: add a collapse_assistant_steps user preference (this PR)
2. #29417 feat(site): group assistant steps into working blocks
3. #29418 feat(site): add the working block disclosure
4. #29419 feat(site): fold assistant steps in the conversation timeline
5. #29420 fix(site): keep streaming text alive inside expanded working
blocks
6. #29115 feat(site): add the collapse assistant steps chat preference

## Why

The chat UI reads a per-user boolean to decide whether to fold steps.
This layer stores it and exposes it through the preference settings API
and SDK.

`collapse_assistant_steps` follows the existing
`task_notification_alert_dismissed` boolean pattern: a `user_configs`
key with dbauthz wrappers, no migration. The generated API docs and
TypeScript types are regenerated, and the frontend preference mocks gain
the new required field.

> This PR was prepared by Xum, an AI coding agent, on behalf of
@ibetitsmike.
2026-09-27 22:15:10 +02:00
Ethan acebdf5b60 feat: add chat_mcp_servers table with encrypted headers (#29517)
This is the second PR in the stack that adds experimental support for
attaching MCP servers to a chat;
https://github.com/coder/coder/pull/29466 is the first. This one is just
the persistence layer, nothing reads from the table yet.

It adds a `chat_mcp_servers` table, unique on `(chat_id, slug)` and
cascading on chat deletion. Each row stores the server URL, a JSON
object of request headers, per-server tool allow and deny lists, and
three flags that later PRs in the stack use to decide when the server
gets connected: `allow_in_plan_mode` (mirroring the existing column on
`mcp_server_configs`), `allow_in_subagents`, and
`forward_coder_headers`. All three default to `false`.

Headers are the sensitive part, so they're encrypted at rest via
`dbcrypt` in the same way `external_auth_links` are: `headers_key_id`
records which key was used, and `NULL` means plaintext. Empty or `{}`
headers are stored as `{}` with no key, so servers that don't need auth
never produce ciphertext. The `dbcrypt` wrappers cover the chat-scoped
reads and writes, plus `GetChatMCPServersByChatOwnerID` and
`UpdateEncryptedChatMCPServerHeaders`, which follow the per-user shape
that `Rotate` and `Decrypt` in `cliutil.go` use. I haven't wired those
two into `coder server dbcrypt rotate` and `decrypt` in this PR.

For authz, reading a chat's servers requires read on the chat, and
upserting or deleting them requires update on the chat. The two
owner-scoped queries above are system-only.


Part of CODAGT-1131
2026-09-25 18:51:54 +10:00
Bobby Ho 9fd0131d04 feat: accept redirect_uris on the OAuth2 app admin API and deprecate callback_url (#29502)
- Add `redirect_uris` to admin OAuth2 app responses and create/update requests.
- Deprecate `callback_url` while preserving compatibility; it maps to the first redirect URI.
- Reject empty `redirect_uris` lists instead of silently ignoring them.
- Update fixtures, tests, and docs.

Part of PLAT-619. Builds on #29440; admin UI support follows in #29603.
2026-09-22 19:12:27 +00:00
Ehab Younes 817fc77612 feat: store session usage per app and family in template usage stats (#29109)
Store per-app session minutes in a child table instead of fixed family-minute columns, so adding an IDE no longer needs a rollup schema change. API, Prometheus, and telemetry output are unchanged.

- Add `template_usage_stats_session_apps`, keyed by start time, template, user, and app name, replacing the `ssh_mins`, `sftp_mins`, `reconnecting_pty_mins`, `vscode_mins`, and `jetbrains_mins` columns.
- Group app names into families at read time through the `codersdk` registry, so the registry never reaches SQL and attribution covers all history rather than only the buckets the rollup still revisits.
- Group `GetTemplateInsightsByTemplate` by app name so that Prometheus and the API report a family identically.
- Drop the `@app_families` query parameter and its dbauthz validation.
- Report a family total as the sum of its apps rather than the distinct minutes any of them was active. Numbers are unchanged today, because every agent reports one canonical name per family.
- Keep the child table window index-usable and count minutes with `COUNT(DISTINCT)` in the grouping, so a dashboard-sized range uses an index scan and a large deployment does not spill the sort to disk.

Part of #27410
2026-09-18 15:26:47 +02:00
Jake Howell dd4f5ae48a fix: load AI Gateway session provider filter without AIProvider read (#28966)
> 🤖 This PR was modified by Coder Agents on behalf of Jake Howell.

The Provider filter on `/ai-gateway` called the owner-only
provider-config API (`GET /api/v2/ai/providers`). Site auditors can read
sessions but not that config, so the dropdown was empty, and typing
`provider_name:…` failed the whole sessions list because search looked
up the name with the same permission.

- Add `GET /api/v2/ai-gateway/providers`, returning display metadata
only (`name`, `type`, `display_name`, `icon`) from `ai_providers`,
including disabled and soft-deleted rows that past sessions still
reference. When a name was reused, the live row wins.
- Authorize it in `dbauthz` on `aibridge_interception` read rather than
`ai_provider` read, so owners and auditors get labels and icons without
access to configuration. Members get 404.
- Match `provider_name` as a plain string in session search instead of
looking up `ai_providers`. Unknown names return an empty page, not a
validation error.
- Point the Provider dropdown at the new endpoint; filter by name, show
`display_name`, and render the icon from `type`/`icon`.

The endpoint reads the small `ai_providers` table instead of grouping
`aibridge_interceptions`, so it needs no new index and does not inherit
the scan cost noted for the models/clients lists. It is registered only
under `/ai-gateway`, not the legacy `/aibridge` alias.
2026-09-17 15:29:21 +00:00
Michael Suchacz 2e03073938 feat(coderd/database): add paginated organization AI spend user query (#29219)
First of a four-PR stack that reworks the AI spend page onto the
released organization spend export, `GET
/api/v2/organizations/{organization}/ai/spend/export`, and a sibling
per-user endpoint, `GET
/api/v2/organizations/{organization}/ai/spend/users`, instead of the
deployment-wide `/api/v2/ai-gateway/spend/*` endpoints this stack
previously proposed. Every PR in the stack was rewritten in place from
current main; the head branch name predates the rework. The export's
accounting rules are unchanged: token usage by `created_at` in
`[period_start, period_end)`, attributed to the organization through the
effective budget group, so the page reconciles with the CSV and with
daily budget accounting.

Adds `ListOrganizationAISpendUsers`, the per-user aggregate behind the
users endpoint. It shares the export query's joins and predicates,
groups to one row per user with priced cost, a count of unpriced usage,
and the sorted distinct providers, clients, and models the user spent
through (a missing client is reported as `Unknown`, as in the sessions
list), carries the window totals (`count`, `total_cost_micros`,
`total_unpriced_usage_count`) as window aggregates ahead of
`LIMIT`/`OFFSET`, and orders by cost, then username, then user ID.
Because every row carries those totals, dbauthz requires read access to
every group member in the organization before running the query, instead
of filtering rows by the caller's own membership.

Includes the dbauthz wrapper and test, the dbmetrics and dbmock
regeneration, and querier tests for grouping, totals across pages,
filters, providers, clients, and models, shifted period windows
(including token usage recorded outside the window of the interception
that produced it), and organization isolation.

> Xum prepared this PR on behalf of @ibetitsmike.
2026-09-17 04:32:15 +00:00
Cian Johnston 2cf819bfca feat: add break-glass user email update (#29256)
Adds an owner-only experimental API and hidden CLI command for
break-glass user email corrections.

The operation verifies the existing email, updates it transactionally,
clears outstanding password-reset state, revokes the target user's Coder
sessions and API tokens, and records successful and failed authenticated
attempts in the audit log. External-auth links remain unchanged.

Generated by Coder Agents.
2026-09-14 14:33:36 +01:00
Bobby Ho 7a57b17b7b fix: make OAuth2 refresh token redemption single-use under concurrency (#28752)
Make OAuth2 refresh token redemption single-use under concurrency using DELETE ... RETURNING, ensuring exactly one request succeeds and losing requests return invalid_grant. Remove the redundant API-key lookup to prevent 500s on missing keys, explicitly use READ COMMITTED for both grant transactions, and replace whitespace-only stored-scope echoes with a fixed error message. Add regression coverage for concurrent redemption, revoked or missing keys, and invalid stored scopes.
2026-09-11 12:47:55 -07:00
Steven Masley d5fcd99213 revert(coderd/database): purge identified terraform modules via dbpurge (#29234)
<!-- Coder Agents generated on behalf of @Emyrk. -->

Reverts #28802.

- Migration `000589_delete_tf_modules` (#28798) already deletes the
identified cached module archives for the same window.
- The recurring dbpurge query is redundant with a one-time migration and
is no longer needed.
- Removes the `DeleteCachedModuleFilesCreatedBetween` query, its
dbauthz, dbmetrics, and dbmock wiring, and the dbpurge tests.

---

<sub>Coder Agents on behalf of @Emyrk.</sub>
2026-09-11 11:32:15 -05:00
Michael Suchacz b957e66ccc feat!: drop the Coder Tasks tables and queries (#29086)
Drops the `tasks`, `task_workspace_apps` and `task_snapshots` tables,
the `tasks_with_status` view and the `task_status` enum, and recreates
`workspaces_expanded` without the `tasks` join and `task_id` column.
Deletes `queries/tasks.sql` and every wrapper, fixture and test that
existed only for those queries (dbauthz, dbmock, dbmetrics, dbfake,
dbgen, querier_test), plus the `task_id` projection in
`GetWorkspaceAgentAndWorkspaceByAuthToken`.

**Breaking change:** migration 000594 deletes all task rows. The down
migration restores the schema (tables, constraints, indexes, views), not
the data.

**Review changes:** the schema-only migration test was removed per
review (`TestMigrate/UpDownUp` still exercises the down path), the test
file no longer reorders existing tests, and the migration moved from
000593 to 000594. The `dump.sql` reorder is pg_dump dependency ordering:
`tasks_with_status` pulled its four workspace tables ahead of it, and
without the view they return to alphabetical positions.

**Stack context:** Coder Tasks removal stack (CODAGT-642): #28743
frontend → #28788 CLI → #28789 API → #28790 provisioner → #29084
notifications → #29085 build reasons → **#29086 tables** → #28750
permissions. The last four PRs replace the single data-drop migration
that used to live in #28750; each carries one concern and its own
migration number.

> Created by Xum (AI agent) on behalf of @ibetitsmike.


## Delivery record

**Decision (2026-09-11 ~14:40Z): merge** through the stack merge
endpoint, squash, pinned to head
`f1d236a0c82f123b8a0d6878799c227ef9bd5776` (merge SHA appended below
once landed).

**Independent substantive reviews on this PR (3):**

| # | Kind | Reviewed commit | Findings and disposition | Source |
|---|---|---|---|---|
| 1 | Codex code review (bot), PR opened | `24535f8410` | No findings,
no threads |
[summary](https://github.com/coder/coder/pull/29086#issuecomment-5583877627)
|
| 2 | Human review, spikecurtis (COMMENTED) | `24535f8410` | 4 findings:
dump.sql reorder question (answered: pg_dump dependency hoisting by
`tasks_with_status`); two moved tests (undone, the test file is no
longer touched); schema-only migration test (deleted). All resolved. |
[review](https://github.com/coder/coder/pull/29086#pullrequestreview-5142045164)
|
| 3 | Human review, spikecurtis (APPROVED) | `f1d236a0c8` (this head) |
Approval on the exact final head |
[review](https://github.com/coder/coder/pull/29086#pullrequestreview-5179353695)
|

The four COMMENTED reviews by @ibetitsmike on `f35688f40b` are
thread-reply wrappers and are not counted.

**Pre-split history relevant to this diff:** this layer's content lived
in the monolithic #28750 until 2026-09-08, which received 10 completed
Codex code reviews (2026-08-28 to 2026-09-01) and spikecurtis's
CHANGES_REQUESTED review of 2026-09-04 that asked for the split.
Counting those, the completed-review total relevant to this diff exceeds
the six-review budget, so no further discretionary review or fresh final
advisory was commissioned. The usable final recommendation for the final
diff is review 3 above: the reviewer who authored every finding on this
layer approved the exact head.

**Exact-head checks:** `required` SUCCESS in [run
34580949353](https://github.com/coder/coder/actions/runs/34580949353/job/103205933212);
rollup 15 success, 8 skipped, 1 failure. The failure is the non-required
`Analyze PR for Documentation Updates Needed` job
([103204101137](https://github.com/coder/coder/actions/runs/34580949071/job/103204101137)),
caused by the dormant `doc-check-pr-29086` review workspace
(CODAGT-1025), not a code finding; GitHub flags only `required` as
required for this PR. Merge-ref freshness: `refs/pull/29086/merge` has
the exact head as second parent and `25af21b5d5d` (the #29085 squash) as
first parent; main has since gained #29231, #28639 and #29218 (12 files,
none overlapping this PR). The composed tree (`git merge-tree` of
current main and this head) builds (`go build ./...`) and vets (`go vet`
on `coderd`, `coderd/database/...`, `coderd/wsrelated/...`) cleanly.
Layer diff parity with the previously pushed layer verified by numstat
after both GitHub restacks. `gh stack push` did not run the local
pre-push suite; CI is the validation evidence.

**Deferred:** the dormant doc-check workspace (CODAGT-1025) is outside
this PR. Next in the stack, #28750, stays blocked: CHANGES_REQUESTED,
its approval and CI predate the restack, and its review budget is
exhausted.
2026-09-11 14:24:32 +00:00
Michael Suchacz f809776229 feat!: remove the task notification templates (#29084)
Removes the six Coder Tasks notification templates (their preferences
and inbox history go with them via cascade) and the
`preference_task_notification_alert_dismissed` user setting with its two
queries. The `task` value of `ai_seat_usage_reason` stays: seats
consumed through tasks remain consumed, and remapping their reason would
misstate billing history.

**Breaking change:** migration 000592 deletes data. The down migration
restores the template definitions, not the deleted preferences or
notification history.

**Review changes:** dropped the `ai_seat_state` remap and enum swap and
the migration unit test (both per review); moved the migration from
000591 to 000592 because main took 000591.

**Stack context:** Coder Tasks removal stack (CODAGT-642): #28743
frontend → #28788 CLI → #28789 API → #28790 provisioner → **#29084
notifications** → #29085 build reasons → #29086 tables → #28750
permissions. The last four PRs replace the single data-drop migration
that used to live in #28750; each carries one concern and its own
migration number.

> Created by Xum (AI agent) on behalf of @ibetitsmike.
2026-09-11 08:05:53 +00:00
Ehab Younes 0aae890d03 feat: return per-app session counts from workspace agent stats queries (#29134)
- Make all six raw session-count queries return a session_counts JSONB
object keyed by app name instead of fixed per-family columns. Fold the
results into existing family fields in Go using the shared SDK registry,
including unrecognized names under unknown.
- Remove the registry parameters and associated dbauthz validation from
these queries, which no longer perform family attribution in SQL.
Preserve existing deployment stats, Prometheus, and telemetry outputs.
- Select session maps directly in the two latest-row queries instead of
decomposing and reaggregating a single row. Keep byte, connection, and
latency aggregates independent of per-app expansion.
- Add regressions for latest eligible rows, empty session maps, distinct
same-family app names, labels, and unchanged traffic aggregates.
2026-09-10 14:07:03 +02:00
Ehab Younes da45488208 feat: attribute session counts by app family (#28337)
- Centralize app-to-family attribution in the SDK registry so recognized
app aliases contribute to their family's session counts and usage
minutes without duplicating name lists across SQL queries.
- Pass the registry into agent stats queries, insights, and usage rollups.
- Validate its structure in dbauthz so missing or renamed families fail
explicitly instead of silently producing zero counts.
- Decompose session maps once per row for per-agent sums while retaining
the key-presence checks used by insights. Update reporting consumers
and add coverage for alias attribution and registry validation.
2026-09-10 13:54:11 +02:00
Bobby Ho 96ee8d171a fix: make OAuth2 code redemption single-use under concurrency (#28744)
TL;DR

Third PR in the PLAT-480 stack, on top of #28740. Fixes a race where the same authorization code could be redeemed twice concurrently.

The redeeming delete now returns the deleted row. If another request already redeemed the code, the delete returns sql.ErrNoRows and redemption fails with invalid_grant.

Each redemption runs in its own transaction, so concurrent requests serialize on the code row and only one succeeds.

The test now races two concurrent redemptions to verify single-use behavior.

No scope logic, new endpoints, or schema changes.

Depends on #28740.

Stack: #28237, #28740, this PR.
2026-09-08 16:42:34 -07:00
Danielle Maywood a6938e69db fix: re-resolve PR when stored URL's branch is stale (#28983) 2026-09-04 17:04:13 +01:00
dylanhuff-at-coder 4875ae218a feat: enforce user secret file policy (#28518)
Adds the deployment option `--user-secrets-disable-file-path` /
`CODER_USER_SECRETS_DISABLE_FILE_PATH` / `userSecretsDisableFilePath`
(default `false`). When it is set, the create and batch-create
user-secret endpoints reject requests with a non-empty file path, and
PATCH validates the post-update state under a `SELECT ... FOR UPDATE`
row lock, rejecting a new or changed file path as well as enabled states
that would have no effective delivery target. Legacy stored paths are
preserved, so users can still edit, disable, clear the path, or migrate
the secret to env delivery.

This is PR 1 of a 4-PR stack; follow-ups cover manifest filtering, the
dashboard, and CLI/docs.

Refs #27488

Reviewed and updated by Coder Agents on behalf of @dylanhuff-at-coder.
2026-09-02 09:21:36 -07:00
Michael Suchacz 6db09ed9a6 fix: sync pinned chat MCP context resources on agent push (#28796) 2026-09-02 13:38:14 +02:00
Bartek Gatz 55340de360 feat: add deployment setting to disable the Codernauts game (#27662)
Adds a `codernauts_enabled` runtime setting (default: enabled) that
controls whether the Codernauts game link appears in the user dropdown
menu. A toggle at the bottom of the Deployment > Appearance page
controls it, and the value persists in `site_configs`, so it survives
restarts and upgrades without any CLI flags or environment variables.

The setting works on all licenses: `GET /api/v2/appearance` reports it
from both the enterprise and the default (AGPL) appearance fetchers, and
`PUT /api/v2/appearance` persists it gated only by the deployment-config
RBAC permission, not by an entitlement. On the Appearance page the
toggle renders outside the Premium paywall introduced in #27948; the
other appearance fields keep their existing licensing behavior.

Note: a pure AGPL-only build of coderd does not register the
`/api/v2/appearance` routes at all, so the toggle cannot be saved there.
The standard `coder server` binary is unaffected regardless of license
state.

<details>
<summary>Implementation notes</summary>

- `coderd/database/queries/siteconfig.sql`: new `GetCodernautsEnabled` /
`UpsertCodernautsEnabled` queries backed by a `codernauts_enabled` key
in `site_configs`; defaults to `true` when unset (no migration needed).
- `coderd/database/dbauthz`: read has no authz checks (matching other
appearance reads); write requires `ResourceDeploymentConfig` update.
Coverage added to `dbauthz_test.go`.
- `codersdk`: `codernauts_enabled` added to `AppearanceConfig` and
`UpdateAppearanceConfig`.
- `coderd/appearance`: the default fetcher takes a `database.Store` and
reads the setting from the database so unlicensed deployments serve it
too.
- `enterprise/coderd/appearance.go`: fetches the value in `Fetch` and
persists it in `putAppearance`.
- Frontend: "Codernauts game" toggle section on Deployment > Appearance
saving immediately via the existing appearance mutation;
`codernauts_enabled` threaded from `useDashboard().appearance` through
Navbar to `UserDropdownContent`, which renders the menu item only when
enabled. Storybook stories with `play` functions cover toggling
(entitled and not) and the dropdown hiding the link when disabled.

</details>

---

🤖 This PR was generated by Coder Agents on behalf of @bartekgatzcoder.
2026-09-02 11:29:35 +02:00
Steven Masley dc1a54cd37 chore: purge identified terraform modules via dbpurge (#28802) 2026-08-31 19:24:27 -05:00
Jaayden HalkoandMathias Fredriksson 8bf271c503 feat: add Prometheus metrics for the usage publisher (#28368)
The Tallyman usage publisher (`enterprise/coderd/usage/publisher.go`)
had no metrics; failures were only visible in logs and the
`usage_events.failure_message` column.

This adds five metrics under `coderd_usage_events_*`:

| Metric | Type | Meaning |
|---|---|---|
| `publish_results_total{result, event_type}` | counter | Per-event
outcomes from real Tallyman responses (`accepted`,
`rejected_temporarily`, `rejected_permanently`; events missing from the
response count as temporary) |
| `publish_send_errors_total` | counter | Ingest requests that failed
entirely (HTTP error, non-200, decode error), one per request |
| `pending` | gauge | Unpublished events still inside the 30-day
publishing window |
| `pending_oldest_age_seconds` | gauge | Age of the oldest pending event
(0 when none) |
| `expired` | gauge | Unpublished events older than 30 days that will
never be published |

Send errors and per-event results never double-count: when a request
fails, the publisher fakes an all-temporarily-rejected response for the
DB update, but only `publish_send_errors_total` increments, so
`publish_results_total` reflects only real Tallyman verdicts.

The gauges are backed by a new read-only `GetUsageEventsStats` query
(authorized as `ActionRead` on `ResourceUsageEvent`) and refreshed from
the publish loop after each attempt, so they update roughly every 17
minutes.

Metrics are created with `promauto.With(reg)` where the registerer
defaults to nil, so existing callers and tests need no changes;
`enterprise/cli/server.go` wires in the deployment's
`PrometheusRegistry` via the new `PublisherWithPrometheusRegisterer`
option.

Closes
https://linear.app/codercom/issue/CODAGT-833/e3-prometheus-metrics-for-the-usage-publisher

---------

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2026-08-28 15:44:31 +07:00
Yevhenii Shcherbina 0bc2858b6e feat: report unpriced AI models to owners (#28419)
Closes
[AIGOV-568](https://linear.app/codercom/issue/AIGOV-568/notify-admins-about-unpriced-ai-models).

AI Bridge records token usage for a model with no price at a NULL cost,
so its spend is neither reported nor enforced against any budget. Until
now that only reached admins through an info log and a Prometheus
metric.

Owners now receive a weekly report listing models used without a price
in the past week. The report links to documentation explaining how Coder
calculates spend from the default price book and how owners can
configure missing prices with `coder exp ai-model-prices`.

## Screenshots

<img width="726" height="531" alt="image"
src="https://github.com/user-attachments/assets/168ae378-60cb-4c96-8f60-36c6edc0b268"
/>
<img width="470" height="417" alt="image"
src="https://github.com/user-attachments/assets/7dc71a1e-0c57-44f1-86f7-df800603c865"
/>



## Design

**Derived, not tracked.** The unpriced set is computed at report time
from interceptions that recorded token usage, joined against the current
`ai_model_prices` table. Requests that produced no token usage are
excluded.

**No new loop or table.** `reportUnpricedAIModels` is a sibling of
`reportFailedWorkspaceBuilds` in the existing report generator, reusing
its ticker and `notification_report_generator_logs`. Each report runs in
its own transaction under a separate advisory lock, so failures and lock
contention do not couple otherwise independent reports. The weekly
frequency is enforced by the persisted timestamp rather than by the
ticker, which restarts with the process and runs on a different phase in
each replica.

## Behaviour

| Situation | Outcome |
|---|---|
| Model used without a price | Listed in the next weekly report |
| Price set | Disappears from the next report |
| Still unpriced and still in use | Reported again each week |
| Model stops being used | Drops out of the report |
| Nothing unpriced | No notification; window still advances |
| More than 100 unpriced models | Top 100 by usage, with the total
reported alongside |
| openai-compat models | Excluded, they cannot be priced |
| Many replicas | Exactly one report per week |
| First ever run | Reports models used in the preceding week immediately
|

## Design choices

1. **openai-compat** is excluded because it cannot be priced and would
produce permanent, unactionable notifications.
2. **Token volume** orders the list but is not shown. The 100-model cap
therefore prioritizes models responsible for the most unpriced usage.
3. **Placement** remains in the generic report generator rather than
`aibridgedserver`. This keeps one periodic-report lifecycle at the cost
of coupling the notifications package to an AI cost-control query.
4. **No license gate.** The reporter runs even when AI Gateway is
disabled. With no recorded token usage the query is empty and no
notification is sent.

---

Created by Coder Agents on behalf of @evgeniy-scherbina.
2026-08-27 21:04:13 +00:00
Garrett Delfosse 378ac91b49 fix: stem chat search terms with the english text search config (#28319)
Chat message search used the `simple` text search config, which folds
case but does no stemming, so `refactor` did not match messages
containing `refactoring`. Message bodies now index and query with the
`english` config, which stems both sides. Titles, PR titles, and all
other filters are unchanged.

Migration `000585` adds a `chat_messages.search_tsv_config` enum column
recording which config produced each stored vector and stamps existing
vectors `'simple'` (a column-only update; vectors and indexes are
untouched, since rewriting `search_tsv` would block message writes on
large tables). The dbpurge sweep gains a bounded
`ReindexStaleChatMessagesSearchTsv` pass that rewrites stale vectors
newest first and stops for the process lifetime once drained. Until a
row is rewritten, `GetChats` matches it with the config that produced
it, so pre-migration exact-form searches keep working during the drain.
Vectors written by old binaries mid rolling upgrade cannot stamp the
config, stay pending, and self-heal on the next upgraded sweep.
Steady-state DB load is unchanged.

The down migration resets `english` vectors to `NULL` so the parent
version's existing sweep rewrites them with `simple`.

Closes
[CODAGT-867](https://linear.app/codercom/issue/CODAGT-867/backend-search-improvements-to-handle-partial-matches)

<details>
<summary>Decision log</summary>

- **`english` stemming over prefix matching.** On a 300k-row corpus a
stemmed lookup is a single GIN entry-tree probe, while `term:*` prefix
matching unions posting lists of every lexeme sharing the prefix (up to
a full scan for short prefixes), is asymmetric, and loses the
`websearch_to_tsquery` phrase/OR/negation UX. `pg_trgm` rejected as a
heavier index with different semantics.
- **Scope to message bodies only** (review). Titles and PR titles keep
their `simple` FTS expressions and indexes, identical to main.
- **Config-versioned queue over migration-time reset** (review). An
unqualified `UPDATE ... SET search_tsv = NULL` rewrites every indexed
row in the migration transaction, and an old replica could backfill
reset rows with `simple` vectors that permanently leave the `search_tsv
IS NULL` queue. The config column makes staleness explicit and both
problems disappear.
- **Enum over `text`** (review) for the config column.
- **Trade-offs.** English stopwords are dropped from message search
(stopword-only queries match no messages); stemming is English-specific;
titles do not stem; prefix matching remains out of scope.

</details>

---

🤖 This PR was generated by Coder Agents on behalf of @f0ssel.
2026-08-25 09:08:52 -04:00
Spike Curtis 6a58d50f1d feat(coderd): add queue-position-free provisioner job query (#28367)
Follow-up to #28302 ([GRU-82](https://linear.app/codercom/issue/GRU-82), [RFC](https://app.notion.com/p/coderhq/Lite-Workspace-API-Requests-3aad579be5928060b5e7cc65c539717f)). That PR added related-data selection but noted that selecting `latest_build.job` still ran the expensive `GetProvisionerJobsByIDsWithQueuePosition` query, which computes queue position and size with window functions over pending jobs and provisioner daemons and is consistently the top resource consumer in scale tests.

This PR adds a cheaper `GetProvisionerJobsByIDs` query that fetches jobs by ID without the queue-position computation, and selects between the two based on whether `latest_build.job.queue_position` is requested. When the queue position is not selected, `QueuePosition` and `QueueSize` are left zero and the rest of the response is unchanged.

Both paths go through a single `provisionerJobsByIDs` API method that takes the `jobRelated` selection, switches on it, and reshapes the cheaper query's rows into `GetProvisionerJobsByIDsWithQueuePositionRow` so downstream conversion is uniform.

The new query has the same dbauthz properties as the queue-position variant (system-level pass-through pending the proper provisioner-job RBAC check tracked in #16160), with a matching authorization test.

No behavior change for existing callers: they select `queue_position` (via `allLatestBuildRelated`), so they continue to use the queue-position query.


---

*Generated by Coder Agents on behalf of @spikecurtis.*
2026-08-24 16:17:55 +02:00
Ethan c275327fb7 feat: audit operational agent settings (#28369)
Depends on #28394.

Audit changes to seven operational agent settings through a typed
`chat_operational_settings` resource. Each handler serializes baseline
capture and mutation with a per-setting advisory lock, keeps both
operations in one transaction, and records field-specific diffs.

Effective no-op requests now skip both the database upsert and the
success audit, including absent rows whose effective value is already
the default. The change also retains the allowlisted raw `site_configs`
query, updates generated audit and API surfaces, adds failure and
concurrency coverage, and reuses the existing AuditPage resource-filter
story pattern.

Relates to CODAGT-720.
2026-08-24 11:53:47 +01:00
Michael SuchaczandEthan Dickson 0ef331fcd0 feat!: scope chat model override settings to organizations (#28442)
Moves chat model override settings from deployment scope to organization
scope, following the org-scoping of chat model configs in #28440.

Depends on #28440 (based on
`ethan/chat-models-availability-consolidation`).

Remote dogfood UAT passed on this change, including multi-organization
scenarios on a licensed deployment with real models.

## Problem

Chat model configs are now owned by organizations, but the override
settings that reference them stayed deployment-wide: admin
subagent/title/compaction overrides in `site_configs`, personal
overrides in `user_configs`, and the advisor model embedded in the
advisor runtime JSON. A single deployment-level override can only point
at one organization's model config, so overrides effectively worked for
the default organization only, and title generation had to soften its
hard-failure contract to tolerate unusable overrides.

## Fix

Admin overrides (general, explore, title generation, compaction,
advisor) and personal overrides (root, general, explore) become
organization-scoped rows in two new typed tables,
`chat_organization_model_overrides` and `chat_user_model_overrides`.
Composite foreign keys on `(organization_id, model_config_id)` make
cross-organization model references unrepresentable. The migration
deletes the legacy serialized `site_configs`/`user_configs` override
keys instead of migrating them; admins and users re-select models in the
new organization-scoped settings.

New endpoints live under
`/api/experimental/organizations/{organization}/chats/model-overrides`
(admin defaults) and
`/api/experimental/organizations/{organization}/members/{user}/chats/model-overrides`
(personal overrides); the legacy deployment-level routes are removed.
chatd resolves overrides via the chat's organization, restoring the
hard-failure contract for configured-but-unusable title generation
overrides. The deployment-level `AllowUsers` switch and advisor runtime
limits stay deployment-scoped.

The Coder Agents settings UI gains a per-organization "Defaults &
overrides" page under AI Settings > Models, and the personal overrides
page gains an organization selector for users in multiple organizations.
Advisor telemetry now reports per-organization override rows.

**BREAKING CHANGE**: Existing deployment-level overrides are dropped,
not migrated. Every organization starts with no overrides after upgrade,
and model selections must be re-applied in the new settings. The legacy
deployment-level override endpoints are removed.

Closes https://linear.app/coder/issue/CODAGT-872

> [!NOTE]
> Xum (AI agent) authored this PR on behalf of @ibetitsmike.

<!-- xum-attribution: model=claude-opus-4-6 thinking=high -->

---------

Co-authored-by: Ethan Dickson <ethan@coder.com>
2026-08-24 11:20:26 +01:00
Ethan aefc761fcf feat!: enforce organization scope for chat models (#27959)
## Summary

Move chat model management and runtime availability to
organization-scoped API routes. The API now identifies the organization
in every model-management operation, applies chat model RBAC and ACLs
within that organization, and returns only models the caller can use
there.

Depends on #27958.

## Problem

Chat model configurations now belong to organizations, but the existing
experimental management and discovery APIs are deployment-shaped. Their
routes do not identify an organization, SDK methods do not accept an
organization ID, and item operations address a model configuration by ID
alone.

That contract cannot safely expose organization-owned model
configurations. It also does not provide the organization-specific
provider information, authorization behavior, ACL management, or
recovery guidance needed by the settings and chat clients later in the
stack.

## Fix

Add organization-scoped chat model endpoints under:

```text
/api/experimental/organizations/{organization}/chats/models
```

The new API provides:

- Organization-scoped model listing and creation.
- Model get, update, and delete operations scoped by both organization
and model ID.
- Model ACL read and update operations.
- Organization-scoped runtime availability and provider status.
- Redacted provider descriptors for model-management clients.
- Chat model RBAC and row ACL enforcement for writes, runtime discovery,
and Chatd model selection.
- Recovery guidance when a chat's organization has no usable local
model.

Historical chats can retain a model reference owned by another
organization, but runtime selection uses an authorized local default
when one is available. Foreign, inaccessible, disabled, and otherwise
unusable models are not exposed as usable selections for the
organization.

The existing collection and availability routes remain as
default-organization compatibility endpoints in this layer:

```text
GET  /api/experimental/chats/model-configs
POST /api/experimental/chats/model-configs
GET  /api/experimental/chats/models
```

These compatibility routes do not provide access to models in
non-default organizations. The previous unscoped item update and delete
routes are removed because item operations must include the owning
organization.

## Breaking change

This changes the experimental HTTP API and Go SDK contracts for chat
model management.

Clients must migrate model-management and availability requests to the
organization-scoped routes and provide an organization identifier. In
the Go SDK, the following methods now require an organization ID:

- `ChatModelAvailability`
- `ChatModels`
- `CreateChatModel`
- `UpdateChatModel`
- `DeleteChatModel`

`ChatModels` also changes its return value from `[]ChatModel` to
`OrganizationChatModelsResponse`, which includes the organization models
and the redacted provider descriptors required by management clients.
New item and ACL methods likewise require both the organization ID and
model ID.

The default-organization compatibility routes reduce the immediate
impact for collection and availability callers, but they do not preserve
the previous item API or SDK method signatures. Generated clients and
direct consumers of the experimental API must be updated before adopting
this change.

_This pull request description was generated by Coder Agents._
2026-08-24 11:20:25 +01:00
Ethan cbe24a3da5 feat: add chat model config RBAC resource (#27957)
Add the `chat_model_config` RBAC resource, actions, and API key scopes.
This layer includes the scope migration and generated API types. Model
list queries apply token scopes, SQL authorization filters, and row
ACLs.

Write and default-management handlers keep deployment configuration
authorization until the next API layer replaces it.

Depends on #27956

_This pull request description was generated by Coder Agents._
2026-08-24 09:43:45 +00:00
Ethan f32592ec6b feat!: scope chat model configs to organizations (#27955)
## Summary

Scope chat model configurations and runtime model resolution to
organizations. Each model configuration now belongs to exactly one
organization, each organization can have its own default model, and
Chatd only uses models that belong to the chat's organization.

Depends on #27968.

## Problem

Chat model configurations are currently deployment-wide even though
chats belong to organizations. A default model, explicitly selected
model, or configured model override can therefore reference the same
model configuration from every organization. This prevents model
catalogs and defaults from being isolated by organization and allows a
chat to retain or select a model that its organization does not own.

Existing model configurations also have no organization ownership or
organization-scoped ACL data, and the database enforces a single live
default across the entire deployment.

## Fix

Add required organization ownership and ACL storage to chat model
configurations. The migration assigns every existing configuration to
the default organization, seeds organization-wide read access, and
replaces the deployment-wide default constraint with a per-organization
constraint.

Scope Chatd lookup, admission, fallback, and queue-promotion behavior to
the chat's organization:

- New chats use the usable default model from their organization.
- Explicit model selections from another organization are rejected.
- Deployment or personal overrides that point to a foreign model are not
used for the chat.
- Historical cross-organization model references remain unchanged in the
database, but runtime generation falls back to a usable local default.
- Queue promotion resolves unavailable, disabled, missing, or foreign
stored models to the usable local default and fails when no usable local
default exists.

The existing model configuration management and availability routes
remain compatibility routes for the default organization in this layer.
A later layer in the stack adds organization-scoped management routes
and updates clients to use them.

## Breaking change

This changes chat model behavior from deployment-wide to
organization-scoped. Existing configurations are migrated only to the
default organization; they are not copied into every organization.

As a result:

- A non-default organization no longer inherits the deployment's
existing default model.
- Creating a chat in an organization without a usable local default
fails with `400: No default chat model config is configured.`
- Explicitly selecting a configuration owned by another organization
fails with `400: Model config does not belong to the specified
organization.`
- Existing chats whose stored model belongs to another organization can
generate only if a usable local default is available; otherwise
generation fails.

Deployments using Agents in non-default organizations must configure a
usable default chat model separately for each organization after the
organization-scoped management layers in this stack are available.

_This pull request description was generated by Coder Agents._
2026-08-24 09:30:01 +00:00
Ethan cd1362abfd fix(coderd): prevent stale chat model config writes (#27968)
Chat model updates could merge PATCH fields from a row read before the
serialized transaction. Deletes could also succeed after another request
deleted the row.

Merge updates from the row read after the write lock. Make soft deletes
report missing rows, and validate provider availability and model
compatibility in the same transaction.

This PR also removes `DeleteChatModelConfigsByAIProviderID`. The query
has no production callers, including the AI provider deletion path, and
only remained in generated database wrappers and mocks. Its removal is
intentional dead-code cleanup and does not change provider deletion
behavior.

_This pull request description was generated by Coder Agents._
2026-08-24 09:30:00 +00:00