Commit Graph
254 Commits
Author SHA1 Message Date
Michael Suchacz 2b875068e0 feat: send RFC 8707 resource parameter on MCP server OAuth2 flows
Store an oauth2_resource indicator on MCP server configs. Auto-discovery
fills it from the Protected Resource Metadata entry that covers the
server URL (else the canonical server URL); admins can set or clear it.
When set, Coder sends it as the resource parameter on authorization,
code exchange, and refresh requests. Manual configs without it send no
resource parameter, so providers that reject it keep working.
2026-10-12 11:23:57 +00:00
Thomas Kosiewski f923f4a525 chore!: remove the chat automations experiment (#30412)
Removes the experimental Chat Automations feature (`chat-automations` experiment) from main. This reverts the 16-PR stack that added it and adds two forward migrations that undo its schema and credential changes.

> [!WARNING]
> **Upgrade consequences**
>
> - **Automation data is deleted.** Migration 000615 drops the `chat_automations` table and deletes every queued message an automation delivered. Chats, their messages and ordinary queued messages stay. Messages lose their automation provenance (`automation_id`, `input_id`).
> - **Automation-only credentials are revoked.** Migration 000616 deletes API keys, OAuth2 codes and OAuth2 tokens whose only scopes (or allow-list entries) were `chat_automation:*`. Mixed credentials keep their other scopes. Custom roles lose their `chat_automation` permissions. OAuth2 apps that allowed only `chat_automation` scopes get the whitespace allowlist that grants nothing, as in 000595.
> - **A coordinated restart is required.** Stop all old replicas before the new version runs the migrations. Old replicas read the dropped table and columns, so a rolling deploy is not supported.
> - **Remove `--chat-max-automations-per-owner` before upgrading.** The flag, the YAML key `chat.maxAutomationsPerOwner` and the env var `CODER_CHAT_MAX_AUTOMATIONS_PER_OWNER` are removed. A deployment that still sets the flag or the YAML key will not start (`unknown flag` / `unknown option`). The env var is ignored. This includes dogfood. No release or tag contains the flag.
> - **Down migrations restore the schema only.** 000616 down and 000615 down recreate the 000611 schema with empty tables. They do not restore deleted automations, webhook secrets, provenance, queued deliveries, revoked credentials, stripped permissions or the deleted experiment rule.

## What is removed

- The `chat-automations` experiment and its stored runtime rule (`experiment_rule:chat-automations`).
- The `--chat-max-automations-per-owner` deployment option.
- The `chat_automation` RBAC resource, its member and chatd grants, and the `chat_automation:*` API key scopes.
- The automation sqlc queries, dbauthz wrappers and audit table entries.
- The management routes under `/api/experimental/organizations/{organization}/chat-automations`, and the webhook route `POST /api/experimental/chat-automations/{automation}/events` with its rate limiter.
- The chatd management service, webhook publish, schedule loop, the `manage_automations` tool, and the chatstate admission seam and queue promotion guard that only automations used.
- The Agents Automations page, editor, nav entry, chat labels, the Manage automations toggle and the API client.
- The `archived:any` chat search value that the automations "View chats" dialog added.
- `docs/ai-coder/agents/automations.md`, its manifest entry, the glossary entry and the related rows.

Reverted PRs: #30085, #30096, #30112, #30117, #30127, #30134, #30160, #30166, #30167, #30171, #30187, #30207, #30214, #30217, #30219, #30220.

## What stays, and why

- Migrations 000609, 000610 and 000611, their fixture, and the #30297 lock fix with `migration000609_test.go` stay unchanged. Deployments such as dogfood already ran them.
- New migration **000615** reverses the schema changes of 000609 and 000611 against the current schema. It uses the same short-wait lock-and-retry block as 000609, and it rebuilds `chats_expanded` from main's current definition minus the two automation columns. New migration **000616** follows the 000595 Tasks removal for credentials and recreates `api_key_scope` without the five `chat_automation:*` values. It keeps the `chat_project_memory:*` values from 000612.
- The `chat_automation` value stays in the `resource_type` enum, and `codersdk.ResourceTypeChatAutomation` stays. Old audit log rows stay readable, as with `task` after the Tasks removal.
- The 000615 delete fires the existing queue-version trigger, so `queue_version` moves to `snapshot_version` for each affected chat. `snapshot_version` is not bumped: the coordinated restart restarts every stream loop from an empty state, and the site does not read the version fields.
- `coderd/x/chatd/ARCHITECTURE.md` drops the sections the stack added and gets back its pre-stack wording, merged with the later main edits to the same file.

## Notes for reviewers

- #30379 (chat project page) merged while this PR was in progress. It passed `manageAutomationsEnabled` in `AgentCreatePage.test.tsx` and listed an `automations` route in `AgentsPageLayout.test.tsx`. Later main commits also added automation props in `ConversationTimeline.test.tsx` and `ConversationTimelineWorkingBlocks.stories.tsx`. This PR removes all of them.
- Remaining `automation` matches under `coderd`, `codersdk`, `enterprise`, `site/src` and `docs` are intended:
  - migrations 000609 to 000611, their fixture and `migration000609_test.go`;
  - the new 000615 and 000616 migrations and `migration000615_test.go`;
  - the `chat_automation` resource type in `dump.sql`, `models.go`, `codersdk/audit.go`, the generated API docs, `swagger.json` and `typesGenerated.ts`;
  - unrelated uses of the word "automation" (docs, `chatlabels_test.go`, `ratelimit.go`, `autostop.go`, `subagent_catalog.go`, `WorkspaceSharingSection.tsx`, `AgentChatPage.stories.tsx`).

Remote UAT on an isolated dev deployment passed. It upgraded a populated pre-removal database, then checked chats, queued messages, tokens, the 404 routes, the removed page and nav entry, and the experiments list.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_

![Agents sidebar after the upgrade, with no Automations entry](https://github.com/user-attachments/assets/bd446035-7fc8-4ca4-a735-3a4f36380738)
2026-10-11 19:57:49 +02:00
Garrett DelfosseandClaude Opus 5.5 a08ba6a70c feat: add shared project memory API for chats in a project (#29301)
- Experiment, stacked on #29242. UI is in #29583.
- Adds **project memory**: `chat_project_memories` rows with a name
(lowercase slug, unique per project), a one-line description (150
characters) and a body (8KB). Up to 200 per project, shared by every
root chat in the project. Chats outside a project and subagents have no
memory.
- Memories are immutable: they are created and deleted, never edited.
Agent tools are `read_memory`, `save_memory`, `delete_memory` and
`consolidate_memory`. Saving an existing name fails and points the agent
at `consolidate_memory` to replace it.
- **Consolidation is agent-driven, in the turn**, modeled on how Claude
Code compacts its memory index when warned it is near its limit. From
160 memories (80%), save results ask the agent to get under 140 (70%)
with `consolidate_memory`, choosing what to merge and drop. That tool
deletes and saves memories in one transaction under the per-project
advisory lock, and rolls back entirely if a delete is missing, a name is
taken, or the result exceeds 200. There is no background extraction or
consolidation.
- **The memory index reaches the model as model-only conversation
messages**, not in a tool description or the system prompt. At turn
start the chat gets a full `<project-memory-index>` snapshot when its
prompt has none (first turn, or after compaction), otherwise a
`<project-memory-index-update>` listing only what changed. Tool
definitions and the system prompt carry no memory state, so memory
writes never invalidate the provider's cached prompt prefix. Writes by
other chats, or by this chat mid-turn, reach the model at its next turn.
- REST API (create, list, get, delete) under
`/api/experimental/organizations/{organization}/chats/projects/{project}/memories`.
Memory permissions follow projects onto the `agents-access` role.
- **Every memory change is audited** as `ChatProjectMemory` create or
delete. REST changes are attributed to the caller. Tool changes
(`save_memory`, `delete_memory`, `consolidate_memory`) are attributed to
the chat owner, carry the `chat_id`, and record one entry per memory
after the change commits, so a rolled-back consolidation records
nothing.

### Examples

**What the model sees.** The system prompt gets a fixed guidance block
with no memory state:

```text
<memory>
Memory is durable context shared by every chat in the project "platform".
Save facts that will matter in future chats: … The memory index is in the conversation: a <project-memory-index> message lists every saved memory, and <project-memory-index-update> messages at the start of later turns list what changed. Changes made during a turn, including your own, appear in the next turn's update. … Memories cannot be edited: replace one that no longer holds with consolidate_memory, or delete it.
</memory>
```

On the first turn, after the user's message, the chat gets a model-only
row with the full index (hidden in the UI):

```text
<project-memory-index>
- deploy-day: Deploys happen on Tuesdays
- release-owner: Alice owns releases; ping her before tagging
</project-memory-index>
```

On a later turn where memories changed, it gets only the difference:

```text
<project-memory-index-update>
Added or changed:
- deploy-day: Deploys happen on Wednesdays (changed 2026-09-30)
- staging-url: Staging lives at staging.example.com
Removed:
- old-ci-notes
</project-memory-index-update>
```

Resulting request layout, with the provider's cached prefix unchanged by
memory writes:

```text
[tools: fixed]  [system: fixed <memory> guidance]  [turn 1 user] [index snapshot] [assistant/tools …]  [turn 2 user] [index update] [assistant …]
```

**Tool calls and results** (produced by the tools in this PR):

```text
save_memory {"name":"staging-url","description":"Staging lives at staging.example.com","body":"Staging is https://staging.example.com; deploys from main."}
→ {"count":3,"saved":"staging-url"}

save_memory {"name":"deploy-day", …}   // name already exists
→ error: a memory named "deploy-day" already exists; read it, then replace it with consolidate_memory by deleting "deploy-day" and saving the new version in one call

consolidate_memory {"delete":["deploy-day"],"save":[{"name":"deploy-day","description":"Deploys happen on Wednesdays (changed 2026-09-30)","body":"The team deploys every Wednesday since 2026-09-30."}]}
→ {"count":3,"deleted":["deploy-day"],"saved":["deploy-day"]}

read_memory {"name":"deploy-day"}
→ {"name":"deploy-day","description":"…","body":"…","created_at":"2026-09-30T15:04:05Z","created_by":"alice"}
```

Near and at the cap:

```text
save_memory … → {"count":160,"saved":"one-more","warning":"memory is 160/200, approaching the limit. Consolidate it to under 140 memories now with consolidate_memory: merge overlapping memories into one, and drop stale, wrong, or superseded ones."}

save_memory … → error: memory is full (200/200); use consolidate_memory to merge overlapping memories and drop stale ones, and include this memory in its save list
```

**REST memory object** (`GET .../chats/projects/{project}/memories`):

```json
{
  "id": "59500000-0000-4000-8000-000000000001",
  "project_id": "7c0e4a3e-5b1f-4f7d-9a51-2f3c8f6d2b10",
  "organization_id": "bb640d07-ca8a-4869-b6bc-ae61ebb2fda1",
  "name": "deploy-day",
  "description": "Deploys happen on Wednesdays (changed 2026-09-30)",
  "body": "The team deploys every Wednesday since 2026-09-30.",
  "created_by": "0ed9befc-4911-4ccf-a8e2-559bf72daa94",
  "created_by_username": "alice",
  "created_at": "2026-09-30T15:04:05Z"
}
```

Create takes `{"name", "description", "body"}` and answers 409 on a
duplicate name or at the cap.

<details>
<summary>Screenshots (predate the immutable-memory rework; tool output
shape has since changed)</summary>


![save_memory](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr2-01-chat-saves-memory-raw.png)

![read_memory](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr2-02-chat-reads-memory-raw.png)
![Agent reads every
memory](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr2-agent-lists-memories-raw.png)

</details>

---
Generated with Coder Agents on behalf of @f0ssel.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 12:52:10 -04:00
Thomas Kosiewski fb0951b850 feat: add the manage_automations chat tool switch and read actions (#30214)
Adds the `manage_automations` chat tool with its read and stop actions
(`list`, `get`, `disable`, `delete`), gated by an interim per-chat
switch. Tool sets have not landed, so this ships
`chats.manage_automations_enabled` as the RFC describes for that case;
tool sets later replace it and migrate its values.

Part of CODAGT-1212 (slice 12a of 3). Part of CODAGT-1135.

## What changes

- **Switch.** Migration `000611` adds `chats.manage_automations_enabled`
(default off). It is set through `POST /api/v2/chats` and `PATCH
/api/v2/chats/{chat}`. After creation only the chat owner may send the
field (403 otherwise). Turning it on returns 400 when the
`chat-automations` experiment is off for the chat owner (POST checks the
resolved `owner_id`, not the caller) or when the chat is a sub-agent
chat. Turning it off is always accepted. The audit table tracks it.
- **Offering.** The tool is offered only in root chats, outside plan
mode and explore mode, with the switch on, and with the experiment on
for the chat owner (decided once per turn, like `mcp-tool-search`). It
is not on the plan-mode or explore allowlists, and sub-agent chats are
created with the switch off.
- **Per-call recheck.** Every call reloads the chat and checks all rules
again, with a fresh experiment evaluation, before doing anything.
- **Scoping.** Owner and organization come from the chat, never from
arguments. Reads run as the owner through a new query that filters on
organization and owner. Other members' automations are "not found", even
when the chat owner is an organization admin.
- **Automation-reached turns.** Detected at call time from the full
history (compaction replays user rows without `automation_id`, so the
prompt window is not enough). In such turns only automations that target
the chat or created it (`chats.automation_id`) are visible.
- **Visibility of prompts and deletes.** `list` leaves out prompts (tool
results stay in the chat, which may be shared); `get` returns the prompt
of the automation it names. In a turn an automation reached, `delete`
only removes the triggering automation; the others can still be
disabled.
- **Audit.** `disable` and `delete` go through the management service
and write a background audit entry with `chat_id`, plus the triggering
`automation_id` and `input_id` in reached turns. Results use the API
shape and never contain a secret or secret hash.

`create`, `update`, `enable`, and `run_now` come in the next PR and
return "not available in this version" here.

Follow-up: CODAGT-1270 (bound the history read that reached-turn
detection does on each call).

## Size

About 1,380 hand-written lines: about 460 production, 680 tests, and 240
SQL that has to be there (the `chats_expanded` view rebuilt in both
migrations and the 17 inline column lists). Generated: +649/-280 (sqlc,
dbmock, dbmetrics, dump.sql, apidoc, API and audit docs,
`typesGenerated.ts`).

## Validation

- Unit and DB tests for every rule above (switch 403/400 cases, offering
rules, per-call recheck, owner/org scoping with an org-admin owner,
reached-turn detection including compaction replay, audit fields, secret
scan), plus a separate test audit. Full `make pre-push` passed before
push.
- Remote dogfood UAT passed on this slice: switch API cases, tool absent
with the switch off, in plan mode, in sub-agents and with the experiment
off for the owner (checked in the server's per-step tool lists),
list/get/disable/delete, and a secret scan of all tool output. Audit
entries are verified in Go tests only.

> [!NOTE]
> ARCHITECTURE.md prose was drafted by an agent at the user's request;
the user will review it.

## Stack

Stacked on the Automations for Coder Agents stack (30085 ... 1214b).
Next: slice 12b (write actions), then 12c (composer toggle).

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-02 15:54:55 +00:00
Thomas Kosiewski eaca2290b0 feat(coderd): add chatd automation admission seam and queue promotion guard (#30096)
Adds the backend-only seam that lets a later `Publish` operation admit
an automation message inside chatd's existing create-chat and
send-message transactions, and the guard that drops stale automation
input when the queue promotes it. Only Go test callbacks use the seam in
this PR, so ordinary chat behavior does not change.

Closes CODAGT-1208
Part of CODAGT-1135

> Stack: PR 2, on top of #30085 (schema, RBAC resource, and audit
entries). No migration changes; this PR adds and extends sqlc queries
only.

ARCHITECTURE.md prose was drafted by an agent at the user's request; the
user will review it.

## Changes

- **Admission seam.** `chatstate.AdmitFunc` is a nil-default `AdmitInTx`
field on `chatd.CreateOptions`, `chatd.SendMessageOptions`,
`chatstate.CreateChatInput`, and `chatstate.SendMessageInput`.
`CreateChat` runs it after inserting the chat and before the initial
history, and requires exactly one initial user message. `SendMessage`
runs it on the locked chat before any write. The callback returns an
`AutomationProvenance` (automation ID, input ID, queue generation),
which is validated and stamped on the message. A callback error rolls
back the whole transaction, so no chat, message, or queue row survives.
`Server.CreateChat` and `Server.SendMessage` keep their signatures.
- **Provenance.** `chatstate.Message.Automation` is written to queued
rows (`automation_id`, `input_id`, `queue_generation`) and history rows
(`automation_id`, `input_id`). `InsertChatMessages` and
`InsertChatQueuedMessageWithCreator` gain the matching parameters, and
`messageFromQueuedRow` copies them on promotion.
- **Promotion guard.** `chatstate.LockAutomations` is the only way to
lock automation rows: it locks them `FOR UPDATE` in ascending ID order,
running as chatd. The chatd subject gains read on `chat_automation`. A
queued automation row passes only if its automation exists, is enabled,
and has the row's `queue_generation`. Two helpers apply the check:
- `nextPromotableQueueHead` serves `sendMessageE1`, `FinishTurn`, and
`FinishInterruption`. A head without an automation costs no extra query.
Otherwise it locks every automation in the queue up front, deletes
failing heads until one passes, and leaves rows behind that head alone.
If the queue empties, `FinishTurn` and `FinishInterruption` land in
`waiting`, as from `R0` and `I0`.
- `guardQueuedRows` serves explicit `PromoteQueuedMessage`. A failing
row is deleted and the transition returns `Rejected` without an error,
so the delete commits. `Server.PromoteQueued` then returns
`ErrQueuedMessageNotFound` (HTTP 404) without reordering, substituting a
message, or changing status.
- The steer paths (`DeliverSteerMessages` and the steer batches from
#30073/#30074) add one `guardQueuedRows` call each, whichever PR lands
second.
- **Deadlocks.** `database.IsDeadlockError` is new. `ChatMachine.Update`
reruns the transaction, at most 3 attempts with a fresh publish buffer
each, only when an attempt that took automation locks hit a deadlock
abort. Attempts without automation locks, and nested Updates, never
retry. The `SendMessage` and `PromoteQueued` closures reset their
results so a rerun is safe.
- **Transition matrix.** Adds the outputs the guard can produce, such as
`R1 -> FinishTurn -> W` and `E1 -> SendMessage -> R0`.
- **`ARCHITECTURE.md`.** Describes the admission seam, provenance, the
queue promotion guard, the lock order, and the deadlock retry, and adds
the guard's outcomes to the state diagram.

## Validation

- Go tests on real PostgreSQL cover the ticket's checks:
  - A callback error leaves no chat, message, or queue row.
- A guard rejection deletes only the rejected heads, on every promotion
path and for each failure reason (missing, disabled, disabled with the
same generation, generation mismatch).
- Explicit promotion of a failing row returns not found, and the delete
is committed.
  - Provenance round-trips through queue and history.
  - `LockAutomations` works through dbauthz.
- `Update` retries only when an attempt that took automation locks hits
a deadlock.
- A concurrency test of acceptance, promotion, and disable across mixed
automation IDs sees zero deadlock aborts.
- Mutation checks confirmed that the guard, retry, and permission tests
fail when their behavior is removed.
- Remote dogfood UAT passed on a real dev instance: sending, queueing,
head and explicit promotion, interrupting, and deleting queued messages
all behaved as before.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_
2026-10-02 15:54:49 +00:00
Thomas Kosiewski f55aa6c50d feat: add chat automations schema, RBAC resource, and audit entries (#30085)
Adds the storage and authorization pieces for Automations for Coder
Agents. No route, SDK request type, or chatd code uses them yet, so
nothing is exposed.

This is the bottom PR of the Automations stack (RFC step 1). Later PRs
add the chatd acceptance seam, management routes, webhooks, and
schedules on top of these names and constraints.

**Schema** (migrations 000609, 000610)

- `chat_automations` table with enums `chat_automation_kind`,
`chat_automation_target_mode`, `chat_automation_webhook_use`, and
`chat_automation_when_busy`. It includes `created_by_chat_id`, the
webhook secret hash and version, the single-use `webhook_consumed_at`
marker, the schedule cursor and revision, and `queue_generation`. Named
CHECKs enforce the target and kind shapes from the RFC. A composite
foreign key keeps `new_chat_model_config_id` in the automation's
organization, the same pattern the model override tables use. A trigger
(`chat_automations_chat_organization`) keeps `target_chat_id` and
`created_by_chat_id` in that organization too. A composite foreign key
cannot do that while keeping `ON DELETE SET NULL` on PostgreSQL 13. The
trigger validates a reference only on insert or when it changes, so chat
deletion can still clear both references. Indexes cover the owner and
both chat references for cascades and `SET NULL`.
- Provenance columns with partial indexes and no foreign key to
`chat_automations`: `automation_id`, `input_id`, `queue_generation` on
`chat_queued_messages` (all set or all null), `automation_id` and
`input_id` on `chat_messages`, and `chats.automation_id`.
`chats_expanded` is recreated to include `automation_id`.
- `chat_automation` values for `api_key_scope` and `resource_type`. The
down migration is a no-op because enum values cannot be dropped.

**Authorization**

- New organization-scoped RBAC resource `chat_automation` (create, read,
update, delete). The `agents-access` role grants it for the member's own
automations. It is excluded from the member role's user-level
permissions, so a member without `agents-access` gets nothing. Site
owners and organization admins get it through their existing
all-permissions lists. The owner-only update, enable, and run rule from
the RFC belongs to the management service in a later PR.
- The generated scopes stay out of `externalLowLevel`, so API keys gain
no user-requestable scopes.
- `dbauthz` wrappers for the three queries this PR adds:
`InsertChatAutomation`, `GetChatAutomationByID`, and
`DeleteChatAutomationByID`. `InsertChatAutomation` also requires update
on the target chat, read on the creating chat, and read on the new-chat
model config (its user and group ACL). There is also a
`dbgen.ChatAutomation` helper.

**Audit**

- `enterprise/audit/table.go` entries for `ChatAutomation`, with
`webhook_secret_hash` and `prompt` marked `ActionSecret`. The prompt is
treated as chat content, like chat titles. The schedule cursor,
revision, and queue generation are ignored as internal bookkeeping.
`Chat.automation_id` is tracked.

**Required checks from the ticket**

- Migrations apply and roll back: `TestMigrate` and
`TestMigrateUpWithFixtures` (new fixture for 000609). Dogfood UAT also
applied the migrations on a dev instance, rolled them back, re-applied
them, and compared schema dumps. After rollback, the only differences
from main were the enum values that cannot be dropped and develop.sh's
own `_develop` schema. After re-applying, the only difference from this
branch's `dump.sql` was the `_develop` schema.
- Secret hash marked secret: `Test_chatAutomationSecretFieldsRedacted`
checks the table entries and that an audit diff contains neither the raw
hash, its base64 form, nor the prompt text.

Also covered: `TestRolePermissions` cases for own, other owner, other
organization, and a member without `agents-access`.
`TestChatAutomationShapeConstraints` covers the named CHECKs, the
cross-organization foreign key and trigger, a cross-organization
retarget, and deleting referenced chats. Under `site/`, only generated
files changed (`typesGenerated.ts`, `rbacresourcesGenerated.ts`).

Closes CODAGT-1207
Part of CODAGT-1135

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_

---------

Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-10-02 15:54:48 +00:00
Mathias Fredriksson ff158c0589 feat: preserve user-supplied chat titles (#29486)
Chats created through the API or MCP couldn't be given a title, and
automatic title generation could overwrite a title the user had set,
at creation or by renaming during generation. #30254 tried to keep
renames by comparing title text, but text can't tell a user title
from a generated one that has the same text.

Chat creation over the API and MCP now accepts a title. Each chat
records where its title came from: fallback (derived from the
prompt), generated, or user. A title can only be replaced by a title
from the same source or a higher one, so automatic generation never
overwrites a user's title. Title changes no longer reorder the chat
list, and clients apply them in the order they were written. Chats
created before this change are treated as having fallback titles.

Closes CODAGT-1047
Reverts #30254
2026-10-02 12:58:53 +00:00
Garrett DelfosseandClaude Opus 5.5 7cd1b2040b feat: add chat projects API (#29242)
- Experiment, behind `chat-projects` (off by default). First of three
stacked PRs: #29301 (memory), #29583 (UI). This PR is API only; the
screenshots below are from the UI in #29583.
- CRUD under
`/api/experimental/organizations/{organization}/chats/projects/{project}`.
`GET /api/experimental/chats/projects` lists the caller's projects
across all organizations.
- `project_id` on chat create and as a chat list filter. A chat joins a
project only at creation; there is no move.
- Projects are identified by ID. Names are display labels (64
characters, description 1024) with no uniqueness constraint. An optional
`icon` URL (usually an emoji image under `/emojis`) is capped at 256
characters and audited.
- A user can own at most 100 projects across all organizations; creating
another answers 409. Creates count and insert under a per-owner advisory
lock so concurrent requests cannot overshoot.
- Projects are private to their `owner_id`. Members may only create
their own, and a chat can only be bound to a project its owner owns. An
unreadable or foreign project on chat create answers 404 so IDs cannot
be probed; a project from another organization answers 400.
- Permissions ride on the `agents-access` role next to chats (after
#30004), so whoever can use Coder Agents can use projects. Owners and
org admins reach every project.

### Examples

Create a project in an organization:

```http
POST /api/experimental/organizations/{organization}/chats/projects
{"name": "Platform", "description": "Infra and deploy work", "icon": "/emojis/1f680.png"}
```

```json
{
  "id": "7c0e4a3e-5b1f-4f7d-9a51-2f3c8f6d2b10",
  "organization_id": "bb640d07-ca8a-4869-b6bc-ae61ebb2fda1",
  "owner_id": "0ed9befc-4911-4ccf-a8e2-559bf72daa94",
  "name": "Platform",
  "description": "Infra and deploy work",
  "icon": "/emojis/1f680.png",
  "created_at": "2026-09-30T15:04:05Z",
  "updated_at": "2026-09-30T15:04:05Z"
}
```

List the caller's projects across organizations with `GET
/api/experimental/chats/projects` (an array of the object above). Update
with `PATCH .../chats/projects/{project}` and any of `name`,
`description`, `icon`; delete with `DELETE`, which detaches its chats.

Create a chat in the project, and list only its chats:

```http
POST /api/v2/chats
{"organization_id": "bb640d07-…", "project_id": "7c0e4a3e-…", "content": [{"type": "text", "text": "Plan the deploy"}]}

GET /api/v2/chats?project_id=7c0e4a3e-…
```

At the limit:

```json
409 {"message": "You can have at most 100 chat projects. Delete a project to create another."}
```

<details>
<summary>Screenshots (UI from #29583)</summary>

![Create
project](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr1-01-create-project-dialog.png)
![Sidebar
folders](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr1-02-sidebar-folders.png)
![Project
page](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr1-03-project-page.png)
![Folder
menu](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr1-05-folder-menu.png)
![Another member sees no
projects](https://raw.githubusercontent.com/coder/coder/f0ssel/chat-projects-assets/v3/pr5-member-sees-no-projects.png)

</details>

---
Generated with Coder Agents on behalf of @f0ssel.

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 11:49:21 -04:00
Susana Ferreira 34db9ca638 feat: price AI Gateway token usage by provider model (#30090)
## Description

When the requested model has no price, price AI Gateway token usage by
the model the provider reports instead of leaving it unpriced. This
covers providers that resolve an alias to a specific model, such as
`codex-auto-review` served as `gpt-5.6-luna`.

Stacked on #30086, which records the provider model.

Closes https://linear.app/codercom/issue/AIGOV-650

## Changes

- Look up the price for the requested model first, then the provider
model, preferring a custom price over the price book for each model.
- Store the provider model and the model used for pricing
(`priced_model`) on each token usage.
- Add a `provider_model` label to `unpriced_token_usage_records_total`.
- Document the lookup order and the new metric label.

> [!NOTE]
> Generated by Claude Code on behalf of @ssncferreira.
2026-09-29 17:47:28 +01:00
Asher 957aefa475 feat: add client_session_id to connection_logs table (#29686)
The client session ID is not populated yet; this just adds the ability
for agents to send a client session ID with the rest of the connection
data, once it gets piped through.
2026-09-27 22:38:36 -08:00
Ethan acebdf5b60 feat: add chat_mcp_servers table with encrypted headers (#29517)
This is the second PR in the stack that adds experimental support for
attaching MCP servers to a chat;
https://github.com/coder/coder/pull/29466 is the first. This one is just
the persistence layer, nothing reads from the table yet.

It adds a `chat_mcp_servers` table, unique on `(chat_id, slug)` and
cascading on chat deletion. Each row stores the server URL, a JSON
object of request headers, per-server tool allow and deny lists, and
three flags that later PRs in the stack use to decide when the server
gets connected: `allow_in_plan_mode` (mirroring the existing column on
`mcp_server_configs`), `allow_in_subagents`, and
`forward_coder_headers`. All three default to `false`.

Headers are the sensitive part, so they're encrypted at rest via
`dbcrypt` in the same way `external_auth_links` are: `headers_key_id`
records which key was used, and `NULL` means plaintext. Empty or `{}`
headers are stored as `{}` with no key, so servers that don't need auth
never produce ciphertext. The `dbcrypt` wrappers cover the chat-scoped
reads and writes, plus `GetChatMCPServersByChatOwnerID` and
`UpdateEncryptedChatMCPServerHeaders`, which follow the per-user shape
that `Rotate` and `Decrypt` in `cliutil.go` use. I haven't wired those
two into `coder server dbcrypt rotate` and `decrypt` in this PR.

For authz, reading a chat's servers requires read on the chat, and
upserting or deleting them requires update on the chat. The two
owner-scoped queries above are system-only.


Part of CODAGT-1131
2026-09-25 18:51:54 +10:00
Thomas Kosiewski 2de1f8b111 feat: link promoted chat messages to their queue entry (#29859)
Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-09-25 09:51:25 +02:00
Michael Suchacz fd5b37576a feat(coderd): record provider response IDs on chat messages (#29763)
Chat messages could only be linked to AI Gateway records per chat, so
nothing recorded which model response produced a given message.

fantasy surfaces the response ID the model endpoint returns on the
stream's finish part: the Anthropic Messages ID as the part ID, and the
OpenAI Responses ID in provider metadata. chatd now stores that ID in
the existing `chat_messages.provider_response_id` column for assistant
steps and the compaction summary row. No migration and no aibridge
changes.

What the stored value is depends on the route:

| Route | Stored value | Joins to |
|---|---|---|
| Anthropic Messages API through the AI Gateway (Anthropic and Bedrock
Claude) | The gateway's interception ID. The gateway replaces the
message ID because injected MCP tool loops can fold several upstream
messages into one response. | `aibridge_interceptions.id` |
| OpenAI Responses API through the AI Gateway | The upstream `resp_` ID,
passed through unchanged | `provider_response_id` on
`aibridge_token_usages`, `aibridge_user_prompts`, and
`aibridge_tool_usages` |
| Direct to the provider | The provider's own ID (`msg_`, `resp_`) |
Nothing in the gateway tables |

To resolve messages to interceptions, then read usage by
`interception_id`:

```sql
SELECT cm.id AS chat_message_id, i.id AS interception_id
FROM chat_messages cm
JOIN aibridge_interceptions i ON i.id::text = cm.provider_response_id
UNION
SELECT cm.id, tu.interception_id
FROM chat_messages cm
JOIN aibridge_token_usages tu ON tu.provider_response_id = cm.provider_response_id;
```

Not covered, and left NULL:
- Partial rows saved on interrupt, because fantasy only reports the ID
in the finish part.
- Chat Completions providers (OpenAI-compatible, OpenRouter, Vercel),
because fantasy does not surface the chunk ID there.
- User, tool, and system rows, and all existing messages.

<details>
<summary>Delivery record</summary>

- `96a896c` replaced the earlier interception-link design (`59fa58f`,
`7f93c18`, `9d04ccd`) in place: the aibridge hook, the
`aibridge_interception_id` column and migration, the message part buffer
tracking, and the `ARCHITECTURE.md` section are all removed. The net
diff against the base touches only `coderd/database` and
`coderd/x/chatd`.
- Local validation on `96a896c`: pre-commit hook green (gen, fmt, lint,
build); `go test ./coderd/x/chatd/... ./coderd/database/` green.
Red-green: removing the finish-part capture fails both new end-to-end
tests (`TestActiveServer_PersistsAnthropicMessageID`,
`TestOpenAIResponsesPersistsProviderResponseID`). CI on `96a896c`:
green.
- Remote UAT round 2 on `96a896c`
([chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)):
**FAIL** against this description's earlier claim that every stored ID
joins the gateway usage rows directly. Anthropic rows routed through the
gateway stored the interception ID, so 0 of 18 joined usage rows
directly, while every checked Anthropic row equaled
`aibridge_interceptions.id`. Both OpenAI Responses rows joined directly.
Interrupted partial rows, Chat Completions rows, and user and tool rows
were NULL, with no empty strings or duplicate IDs. Multi-step turns,
manual compaction, edit and regenerate, a provider switch, and a retried
429 behaved as expected. One unattributed anomaly: a single duplicate
user row after interrupt and continue, which did not reproduce and is
outside this diff.
- `2e889ba` corrects the doc comment on
`PersistedStep.ProviderResponseID` and this description to match;
behavior is unchanged. Pre-commit hook and CI green.
- Remote UAT round 3 on `2e889ba` against the corrected contract ([same
chat](https://dogfood.cdr.dev/agents/75353a23-cd9e-4c81-a99f-472c78f3ba32)):
**PASS**. The query above resolved all 40 stamped rows to exactly one
interception from the same chat, with none unresolved. Reading usage by
interception recovered every usage row for 38 of them; the last two
resolved but their usage rows were not counted. Fresh Anthropic and
Responses calls, a seven-step tool turn, manual compaction, an Anthropic
to Responses to Anthropic switch in one chat, and a UI reload behaved as
described, and the NULL cases held. Three interrupt and continue
attempts did not reproduce the round-2 duplicate, so it stays
unattributed. Not tested: gateway-injected MCP tool loops (unreachable
from chatd, whose delegated requests skip the gateway's MCP proxy),
direct Anthropic `msg_` IDs, Bedrock, and threshold-triggered
compaction. Both rounds had a verified isolation proof and a clean
dogfood audit.
- Codex rounds 1 to 3 reviewed the old design. Codex round 4 on
`2e889ba`
([request](https://github.com/coder/coder/pull/29763#issuecomment-5796193371),
13:58Z): no major issues
([result](https://github.com/coder/coder/pull/29763#issuecomment-5796267231),
reviewed commit `2e889ba05c`). No unresolved review threads. Review
passes so far: 4 of 6.
- Final head `2e889ba` is the exact commit remote UAT round 3 and Codex
round 4 covered; required checks green.

</details>

> Xum updated this PR on behalf of @ibetitsmike.
2026-09-23 19:01:24 +02:00
Thomas Kosiewski 60cc826f2b feat: sign forwarded MCP identity headers (#29669) 2026-09-22 11:11:08 +02:00
Cian Johnston 19076beb8d feat: attribute AI Gateway usage to workspaces (#29235)
- Adds nullable `workspace_id` to AI Gateway interceptions without a
foreign key or index.
- Derives direct attribution from strictly parsed server-minted workspace
session-token names, excluding user-created tokens and validating the
embedded owner.
- Derives delegated Coder Agents attribution from the persisted chat
workspace ID.
- Records `workspace_id` through the additive v1.3 gateway protocol and
structured interception logs.
- Exposes attribution in session details through
`interception_attributions`, including tool-less interceptions and null
values for unknown attribution.

Closes AIGOV-637

Generated by Coder Agents.
2026-09-18 15:22:02 +01:00
Michael Suchacz d2e94e299a chore: delete dead functions and skip-only or duplicated tests (#29461) 2026-09-17 18:14:08 +02:00
Michael Suchacz b957e66ccc feat!: drop the Coder Tasks tables and queries (#29086)
Drops the `tasks`, `task_workspace_apps` and `task_snapshots` tables,
the `tasks_with_status` view and the `task_status` enum, and recreates
`workspaces_expanded` without the `tasks` join and `task_id` column.
Deletes `queries/tasks.sql` and every wrapper, fixture and test that
existed only for those queries (dbauthz, dbmock, dbmetrics, dbfake,
dbgen, querier_test), plus the `task_id` projection in
`GetWorkspaceAgentAndWorkspaceByAuthToken`.

**Breaking change:** migration 000594 deletes all task rows. The down
migration restores the schema (tables, constraints, indexes, views), not
the data.

**Review changes:** the schema-only migration test was removed per
review (`TestMigrate/UpDownUp` still exercises the down path), the test
file no longer reorders existing tests, and the migration moved from
000593 to 000594. The `dump.sql` reorder is pg_dump dependency ordering:
`tasks_with_status` pulled its four workspace tables ahead of it, and
without the view they return to alphabetical positions.

**Stack context:** Coder Tasks removal stack (CODAGT-642): #28743
frontend → #28788 CLI → #28789 API → #28790 provisioner → #29084
notifications → #29085 build reasons → **#29086 tables** → #28750
permissions. The last four PRs replace the single data-drop migration
that used to live in #28750; each carries one concern and its own
migration number.

> Created by Xum (AI agent) on behalf of @ibetitsmike.


## Delivery record

**Decision (2026-09-11 ~14:40Z): merge** through the stack merge
endpoint, squash, pinned to head
`f1d236a0c82f123b8a0d6878799c227ef9bd5776` (merge SHA appended below
once landed).

**Independent substantive reviews on this PR (3):**

| # | Kind | Reviewed commit | Findings and disposition | Source |
|---|---|---|---|---|
| 1 | Codex code review (bot), PR opened | `24535f8410` | No findings,
no threads |
[summary](https://github.com/coder/coder/pull/29086#issuecomment-5583877627)
|
| 2 | Human review, spikecurtis (COMMENTED) | `24535f8410` | 4 findings:
dump.sql reorder question (answered: pg_dump dependency hoisting by
`tasks_with_status`); two moved tests (undone, the test file is no
longer touched); schema-only migration test (deleted). All resolved. |
[review](https://github.com/coder/coder/pull/29086#pullrequestreview-5142045164)
|
| 3 | Human review, spikecurtis (APPROVED) | `f1d236a0c8` (this head) |
Approval on the exact final head |
[review](https://github.com/coder/coder/pull/29086#pullrequestreview-5179353695)
|

The four COMMENTED reviews by @ibetitsmike on `f35688f40b` are
thread-reply wrappers and are not counted.

**Pre-split history relevant to this diff:** this layer's content lived
in the monolithic #28750 until 2026-09-08, which received 10 completed
Codex code reviews (2026-08-28 to 2026-09-01) and spikecurtis's
CHANGES_REQUESTED review of 2026-09-04 that asked for the split.
Counting those, the completed-review total relevant to this diff exceeds
the six-review budget, so no further discretionary review or fresh final
advisory was commissioned. The usable final recommendation for the final
diff is review 3 above: the reviewer who authored every finding on this
layer approved the exact head.

**Exact-head checks:** `required` SUCCESS in [run
34580949353](https://github.com/coder/coder/actions/runs/34580949353/job/103205933212);
rollup 15 success, 8 skipped, 1 failure. The failure is the non-required
`Analyze PR for Documentation Updates Needed` job
([103204101137](https://github.com/coder/coder/actions/runs/34580949071/job/103204101137)),
caused by the dormant `doc-check-pr-29086` review workspace
(CODAGT-1025), not a code finding; GitHub flags only `required` as
required for this PR. Merge-ref freshness: `refs/pull/29086/merge` has
the exact head as second parent and `25af21b5d5d` (the #29085 squash) as
first parent; main has since gained #29231, #28639 and #29218 (12 files,
none overlapping this PR). The composed tree (`git merge-tree` of
current main and this head) builds (`go build ./...`) and vets (`go vet`
on `coderd`, `coderd/database/...`, `coderd/wsrelated/...`) cleanly.
Layer diff parity with the previously pushed layer verified by numstat
after both GitHub restacks. `gh stack push` did not run the local
pre-push suite; CI is the validation evidence.

**Deferred:** the dormant doc-check workspace (CODAGT-1025) is outside
this PR. Next in the stack, #28750, stays blocked: CHANGES_REQUESTED,
its approval and CI predate the restack, and its review budget is
exhausted.
2026-09-11 14:24:32 +00:00
Michael Suchacz 25af21b5d5 feat!: remove has_ai_task and the task build reasons (#29085)
Drops the `has_ai_task` column from `template_versions` and
`workspace_builds` (with its index and view projections) and removes the
three task build reasons (`task_auto_pause`, `task_manual_pause`,
`task_resume`) from the `build_reason` enum. Existing rows are remapped
before the enum swap: automatic pauses become `autostop`, manual pauses
and resumes become `initiator`, and
`workspace_build_orchestrations.child_reason` follows the same mapping.
The query filters, provisionerdserver flag writes, and audit ignore
entries for the column go with it.

**Migration cost:** recreating `build_reason` rewrites
`workspace_builds`. `TestMigration000593TaskBuildReasonRewriteTiming`
seeds 1,000 workspaces, 100,000 builds (30,000 with task reasons) and
5,000 orchestrations and asserts the migration stays under a two-minute
bound; locally it takes about 0.7 to 1.0 s. The test is skipped under
`-short`.

**Breaking change:** the down migration re-adds `has_ai_task` as `false`
and restores the enum values; remapped reasons keep their new values.

**Review changes:** the migration test now asserts only the row remaps
(schema and enum introspection removed per review); renumbered from
000592 to 000593.

**Stack context:** Coder Tasks removal stack (CODAGT-642): #28743
frontend → #28788 CLI → #28789 API → #28790 provisioner → #29084
notifications → **#29085 build reasons** → #29086 tables → #28750
permissions. The last four PRs replace the single data-drop migration
that used to live in #28750; each carries one concern and its own
migration number.

> Created by Xum (AI agent) on behalf of @ibetitsmike.
2026-09-11 10:47:43 +02:00
Ehab Younes 90c7533204 perf: store workspace agent session counts as JSONB (#28126)
Replace fixed session_count_* columns with a sparse session_counts
JSONB object so new session types require no schema changes.

Convert retained and unrolled-up activity, warning on backlogs spanning
more than 24 hours. Update batching, queries, and rollups while preserving
existing API shapes and compatibility with older agents.

Add migration, query, and batcher coverage.
2026-09-07 22:31:42 +02:00
McKayla はな 29995c9ef8 feat: add a per-template workspace rename setting (#27558) 2026-08-25 11:05:42 -06:00
Ethan f32592ec6b feat!: scope chat model configs to organizations (#27955)
## Summary

Scope chat model configurations and runtime model resolution to
organizations. Each model configuration now belongs to exactly one
organization, each organization can have its own default model, and
Chatd only uses models that belong to the chat's organization.

Depends on #27968.

## Problem

Chat model configurations are currently deployment-wide even though
chats belong to organizations. A default model, explicitly selected
model, or configured model override can therefore reference the same
model configuration from every organization. This prevents model
catalogs and defaults from being isolated by organization and allows a
chat to retain or select a model that its organization does not own.

Existing model configurations also have no organization ownership or
organization-scoped ACL data, and the database enforces a single live
default across the entire deployment.

## Fix

Add required organization ownership and ACL storage to chat model
configurations. The migration assigns every existing configuration to
the default organization, seeds organization-wide read access, and
replaces the deployment-wide default constraint with a per-organization
constraint.

Scope Chatd lookup, admission, fallback, and queue-promotion behavior to
the chat's organization:

- New chats use the usable default model from their organization.
- Explicit model selections from another organization are rejected.
- Deployment or personal overrides that point to a foreign model are not
used for the chat.
- Historical cross-organization model references remain unchanged in the
database, but runtime generation falls back to a usable local default.
- Queue promotion resolves unavailable, disabled, missing, or foreign
stored models to the usable local default and fails when no usable local
default exists.

The existing model configuration management and availability routes
remain compatibility routes for the default organization in this layer.
A later layer in the stack adds organization-scoped management routes
and updates clients to use them.

## Breaking change

This changes chat model behavior from deployment-wide to
organization-scoped. Existing configurations are migrated only to the
default organization; they are not copied into every organization.

As a result:

- A non-default organization no longer inherits the deployment's
existing default model.
- Creating a chat in an organization without a usable local default
fails with `400: No default chat model config is configured.`
- Explicitly selecting a configuration owned by another organization
fails with `400: Model config does not belong to the specified
organization.`
- Existing chats whose stored model belongs to another organization can
generate only if a usable local default is available; otherwise
generation fails.

Deployments using Agents in non-default organizations must configure a
usable default chat model separately for each organization after the
organization-scoped management layers in this stack are available.

_This pull request description was generated by Coder Agents._
2026-08-24 09:30:01 +00:00
McKayla はな 5ca0143992 refactor: upgrade golangci-lint to v2.0.2 (#28416) 2026-08-21 23:46:27 -06:00
Michael Suchacz 7ca7c30f40 feat: add group and user ACLs to MCP server configs (#27944)
Adds group and user ACLs to org-scoped MCP server configs so
organizations can restrict specific MCP servers to subsets of members,
mirroring the template ACL pattern.

## Summary

- Migration adds `group_acl`/`user_acl` JSONB columns (nested `{"<id>":
{"permissions": [...]}}` shape) and seeds every existing config with its
organization's Everyone group read entry (the Everyone group ID equals
the org ID), so member access is unchanged by default. Creation seeds
the same entry.
- The blanket org-member read grant from the base PR is replaced by ACL
evaluation: Rego requires org membership for every ACL grant, and
`GetAuthorizedMCPServerConfigs` compiles ACL-aware SQL filters.
- New `ActionShare` (org admins) gates `GET|PATCH
/api/experimental/organizations/{organization}/mcp-servers/{mcpserverconfig}/acl`
(nested under the organization like the rest of the config surface).
PATCH validates principals against the config's organization, merges
sparse updates under a row lock, stamps `updated_by`/`updated_at`, and
is audited as a Write with `Old` captured before authorization. A config
deleted concurrently between the middleware fetch and the locked
re-fetch is concealed as 404, matching the update and delete handlers.
ACL columns are tracked in the audit table.
- ACL management is available in all editions (no enterprise
entitlement), documented in the MCP servers page. Revoking an ACL does
not retro-strip already-selected configs from existing chats; new
selection is blocked at chat create.
- Force On respects the ACL: the forced set is loaded as the chat owner,
so a `force_on` server whose ACL denies the owner never attaches at
create, send, or generation time.
- No rolling-upgrade machinery: upgrades run in scheduled maintenance
downtime, so the migration only backfills existing rows; the API sets
the Everyone read grant explicitly on every insert.

Stacked on #27943. Part of the MCP org-separation stack.

Closes https://linear.app/codercom/issue/CODAGT-712

UAT: verified on a dogfood instance with two members and a custom group:
Everyone-seed default visibility, group grant with Everyone removal
(non-member loses list/fetch/selection), foreign-principal rejection,
non-admin share denial (audited 403 / concealed 404), user_acl restore,
and audited ACL diffs.

> Mux (AI agent) authored this PR on Mike's behalf.

<!-- mux-attribution: model=claude-fable-5 thinking=high -->
2026-08-19 19:25:18 +00:00
Michael SuchaczandMathias Fredriksson 443e3b9b80 feat!: org-scope MCP server configs with RBAC (#27942)
Moves `mcp_server_configs` from deployment scope to organization scope
so each organization fully controls the MCP servers its members can use
with Coder Agents.

## Summary

- Migration: adds `organization_id` (NOT NULL, FK) and keeps existing
rows as the default organization's originals with credentials intact.
Other organizations start with no MCP servers and configure their own;
nothing is copied across organizations. Chats outside the default
organization keep any now-cross-organization `mcp_server_ids` entries;
the runtime already ignores IDs that do not resolve in the chat's
organization, so no data rewrite is needed. Slug uniqueness becomes
`(organization_id, slug)`.
- RBAC: new org-scoped `ResourceMCPServerConfig` with regosql converter
and `GetAuthorizedMCPServerConfigs`; org admins get in-org CRUD, org
members get read (replaced by ACL evaluation in the follow-up ACL PR in
this stack).
- API: all config routes nest under the organization, matching
templates: `POST|GET
/api/experimental/organizations/{organization}/mcp-servers` and
`GET|PATCH|DELETE .../mcp-servers/{mcpserverconfig}` (plus
`oauth2/connect`), resolved by a read-only param middleware that
conceals read-denied and cross-organization access as 404. Two routes
stay on the frozen `/api/experimental/mcp/servers/{mcpServer}` block:
the OAuth2 callback (the redirect URI baked into existing AS-side client
registrations) and `oauth2/disconnect`, which must remain reachable by
users removed from the organization so they can still revoke their token
grant.
- Chat runtime: selection validation and generation resolve configs
strictly by IDs, enabled state, and the chat's organization in SQL;
requested duplicates are normalized; invalid or cross-org IDs are
rejected with the precise ID list. IDs already persisted on a chat are
exempt from message-time rejection so disabling a selected server never
blocks sends; generation skips servers that are no longer usable.
- Frontend: API layer and admin settings pages target the new endpoints.
The admin page manages the default organization's servers; the org
picker is tracked separately (CODAGT-714).

- Security hardening from review: OAuth user grants are additionally
bound to `oauth2_revocation_url` (changing it invalidates grants, and a
racing OAuth callback gets 409 instead of recreating a grant).
Stack-wide SSRF protection for MCP config-directed traffic was split
into its own PR at the top of this stack (#28242) to keep this diff
reviewable; this PR keeps main's existing discovery IP-range guard.
OAuth2 auto-discovery now completes before the config row is inserted: a
failed discovery persists nothing, and there is no provisional row that
concurrent updates could race against.

Two follow-up PRs in this stack were split out to keep this diff
reviewable: #28064 completes the swagger annotations for the moved
routes (main already ships these experimental MCP handlers unannotated),
and #28065 carries hardening fixes and regression pins on top of the
cutover.

- Force On enforcement (landed on main mid-review) is org-scoped: the
forced set is read per chat organization
(`GetForcedMCPServerConfigsByOrganization`), so another organization's
`force_on` server never attaches to a chat.

## Breaking changes (experimental API)

The MCP server config endpoints move from the deployment-scoped
`/api/experimental/mcp/servers` block to organization-nested paths:
`POST|GET /api/experimental/organizations/{organization}/mcp-servers`
and `GET|PATCH|DELETE
/api/experimental/organizations/{organization}/mcp-servers/{mcpserverconfig}`
(plus `oauth2/connect`). The old paths are removed, so API consumers
must supply an organization. Two routes intentionally stay on the frozen
`/api/experimental/mcp/servers/{mcpServer}` block: the OAuth2 callback
(its redirect URI is baked into existing AS-side client registrations)
and `oauth2/disconnect` (must remain reachable by users removed from the
organization). These endpoints are under `/api/experimental`, so no
deprecation window is provided.

## Rolling upgrades

During a rolling deploy, an old replica creating an MCP config can fail
the new `NOT NULL organization_id` constraint until it is upgraded
(reads are unaffected: old binaries' generated queries select their own
column lists). This matches the repo's existing precedent for additive
NOT NULL migrations (000562) and affects only the admin config-create
path in the upgrade window.

Upgrades are expected to run in scheduled maintenance downtime with the
database locked during migration, so the migration ships no
rolling-upgrade compatibility machinery. The down migration deletes
organization-created configs (their chat references are cleaned by the
000510 delete trigger) and restores deployment-wide slug uniqueness.

Part of the MCP org-separation stack (CODAGT-711 -> CODAGT-717 audit ->
CODAGT-712 ACLs -> CODAGT-806 token RBAC).

Closes https://linear.app/codercom/issue/CODAGT-711

UAT: validated end to end on a two-org dogfood deployment, including a
real pre-migration to post-migration upgrade, cross-org isolation
(404s), same-slug-two-orgs, chat selection gating, and a live MCP tool
call through the org-scoped generation path. The migration was later
revised to keep existing rows in the default organization only (no
per-organization copies); that revision is covered by the migration test
suite.

> Mux (AI agent) authored this PR on Mike's behalf.

<!-- mux-attribution: model=claude-fable-5 thinking=high -->

---------

Co-authored-by: Mathias Fredriksson <mafredri@gmail.com>
2026-08-19 18:04:13 +00:00
Michael Suchacz affeeaf9c8 feat: extend agent chat MCP tools for remote UAT evidence loops (#28233)
Extends the Agent-chat MCP tools so an unattended UAT evidence loop can
fetch artifacts, monitor long runs, and find prior runs without burning
model context.

## Backend

- New `chat_files_token` crypto key feature (migration 000571) with
rotator support and a dedicated signing keycache on coderd.
- `POST /api/experimental/chats/files/{file}/download-url`
(authenticated) mints a short-lived (5 min) signed URL and returns it
with `sha256`, `size_bytes`, `name`, `mime_type`, and `expires_at`.
- `GET /api/experimental/chats/files/{file}/download?token=` (no session
token) redeems the signed URL: verifies the JWS, requires the token's
`file_id` to match the path, and re-checks the minting user's RBAC
access live at redemption. Clients can `curl -o` artifacts with zero
credentials in the URL consumer.
- `ChatFileMetadata` gains `size_bytes` (via `octet_length`, no bytes
fetched).

## MCP tools (`codersdk/toolsdk`)

- `coder_download_chat_file`: by `file_id` or `chat_id`+`file_name`;
returns the signed URL plus checksum and size instead of base64.
- `coder_await_chat`: blocks (bounded `wait_secs`, 1-120) until a chat
leaves `running`/`interrupting`, using the existing watch stream with
subscribe-before-read.
- `coder_list_chats`: label, query, and limit filtering; chat
projections now include labels.
- `coder_get_chat_messages`: `after_id` forward cursor with
`next_after_id` (exact incremental reads), plus per-message `files`
metadata so artifact-bearing messages are identifiable.
- `coder_get_chat`: file listings now include `size_bytes` and
`created_at`.
- `coder_list_templates`: exposes `agents_allowed` for pre-flight
checks.

## Testing

- coderd: mint/redeem happy path with an unauthenticated client,
expired/tampered/file-mismatched tokens, auth still required on the
plain file endpoint, non-owner mint rejection.
- toolsdk: harness + integration coverage for all new/changed tools,
including signed-URL redemption with checksum verification,
forward-cursor exactness, await transition/timeout paths, and label
filtering.
- Remote dogfood UAT (dev.coder.com Coder Agent) passed all six
acceptance scenarios end to end over both MCP transports.

Note: `go test ./codersdk/toolsdk/` has a pre-existing goleak flake on
main (leaked `agentssh` non-PTY session goroutines from SSH exec tests;
reproduced 3/3 on clean `b4971bc49f1`). It is unrelated to this diff.

> Mux acted on Mike's behalf to create this PR.

<!-- mux-attribution: model=claude-sonnet-4-6 thinking=high -->
2026-08-18 19:15:30 +02:00
Bobby HoandClaude Opus 5 990d24dc42 feat: add oauth2 scope columns and single-use delete queries (#28007)
OAuth2 tokens issued by Coder ignore scope entirely. The authorize
endpoint parses the `scope` parameter and then discards it, and both
grant paths mint API keys with full API access regardless of what the
client requested or what the app's allowlist permits. There is also
nowhere to put a negotiated scope: nothing carries one from the
authorize step to the token it produces.

Schema and query groundwork for that pipeline. No behavior change on its
own.

- Migration `000569` adds a `scope` column to
`oauth2_provider_app_codes` and `oauth2_provider_app_tokens`, so a
negotiated scope can travel from a code to the token it is exchanged
for, and from a token to its refreshed successor.
- Existing rows are backfilled to `coder:all`, then both columns become
NOT NULL with a non-empty CHECK. Every OAuth2 key is unrestricted in
fact today, so the backfill only writes that down, and a caller that
omits the column now fails instead of silently issuing full access.
- Adds `DeleteOAuth2ProviderAppCodeByIDReturningRow` and
`DeleteAPIKeyByIDReturningRow`, which return `sql.ErrNoRows` when the
row is already gone. Postgres serializes concurrent deletes on the row
lock, so exactly one caller gets a row back, which is what will let the
grant paths enforce single use of a code or refresh token without a
read-then-write race.
- No callers yet. The existing blind deletes and all of their call sites
are untouched, and codes and tokens record `coder:all` until a later
phase negotiates a real value.

Phase 1 of [PLAT-470](https://linear.app/codercom/issue/PLAT-470),
tracked as
[PLAT-478](https://linear.app/codercom/issue/PLAT-478/phase-1-schema-and-queries).
Scope validation at authorize, applying the negotiated scope in the code
grant, and refresh narrowing follow as separate PRs.

Verified locally: `make gen` and `make lint` clean, the migrations suite
passes both up and down, and dbauthz's `TestMethodTestSuite` passes.

<details>
<summary>End-to-end scope enforcement flow (green marks what this PR
touches)</summary>

```mermaid
flowchart TD
    subgraph authorize["/oauth2/authorize"]
        AZ1["ShowAuthorizePage (GET)<br/>renders consent page"]
        AZ2["ProcessAuthorize (POST)<br/>scope parsed, then discarded"]
        Q1["InsertOAuth2ProviderAppCode<br/>gains a Scope param"]
        AZ1 --> AZ2 --> Q1
    end

    Q1 --> CODES[("oauth2_provider_app_codes<br/>new column: scope text NOT NULL")]

    subgraph codegrant["POST /oauth2/token, grant_type=authorization_code"]
        G1["authorizationCodeGrant"]
        Q2["GetOAuth2ProviderAppCodeByPrefix<br/>now returns Scope"]
        Q4["DeleteOAuth2ProviderAppCodeByIDReturningRow<br/>added, no caller yet"]
        G2["apikey.Generate + UserRBACSubject<br/>hardcoded to full access"]
        G1 --> Q2 --> G2
        G1 -.-> Q4
    end

    CODES --> G1
    G2 --> Q3

    Q3["InsertOAuth2ProviderAppToken<br/>gains a Scope param"]
    Q3 --> TOKENS[("oauth2_provider_app_tokens<br/>new column: scope text NOT NULL")]

    subgraph refresh["POST /oauth2/token, grant_type=refresh_token"]
        G3["refreshTokenGrant"]
        Q5["GetOAuth2ProviderAppTokenByPrefix<br/>now returns Scope"]
        Q6["DeleteAPIKeyByIDReturningRow<br/>added, no caller yet"]
        G3 --> Q5
        G3 -.-> Q6
    end

    TOKENS --> G3
    Q5 --> Q3

    subgraph enforce["Every authenticated API request"]
        E1["httpmw ExtractAPIKey"] --> E2["APIKey.ScopeSet()"] --> E3["UserRBACSubject"] --> E4["dbauthz authorize"]
    end

    TOKENS --> E1

    classDef changed fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,color:#1b3c1e
    classDef dormant fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px,stroke-dasharray:5 3,color:#1b3c1e
    class Q1,Q2,Q3,Q5,CODES,TOKENS changed
    class Q4,Q6 dormant
```

Solid green is added or changed here. Dashed green exists but has no
caller yet. Everything else is unchanged, including the enforcement
engine at the bottom, which already reads a key's scopes correctly and
only needs real data fed into it.

</details>

<details>
<summary>Suggested reading order</summary>

Most of the diff is generated. `dump.sql`, `models.go`, `querier.go`,
`queries.sql.go`, `check_constraint.go`, and the dbmock and dbmetrics
packages all come from `make gen`.

1. `migrations/000569_oauth2_scope_columns.{up,down}.sql`: additive
column, backfill, NOT NULL, CHECK, and a `COMMENT ON COLUMN` on each.
2. `queries/oauth2.sql` and `queries/apikeys.sql`: `scope` added to both
insert column lists, plus the two new returning-row deletes alongside
the untouched originals. The `Get...ByPrefix` selects needed no edit,
since they are `SELECT *`.
3. `dbauthz/dbauthz.go`: hand-written wrappers for the two new queries,
each fetching by ID, authorizing delete against the fetched object, then
delegating. The generic `deleteQ` helper does not fit, since it requires
the delete to return only `error`.
4. `oauth2provider/authorize.go` and `oauth2provider/tokens.go`: the
only production changes, all behavior-neutral.
5. `dbgen/dbgen.go` and `dbauthz/dbauthz_test.go`: seed threading, plus
a case per new query. `MethodTestSuite` fails with "Method never called"
for anything untested.

Neither type needs to become auditable, which `make lint` confirms by
not erroring on `enterprise/audit/table.go`.

</details>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-13 09:58:51 -07:00
Bobby HoandClaude Opus 5 16c58770f8 feat: constrain the OAuth2 client type column (#27931)
Extracted from #27873 so the schema change can be reviewed for migration
safety on its own. #27873 will rebase onto this.

`client_type` decides whether the token endpoint validates a client
secret at all, and the column accepts any text: nullable, no `CHECK`, no
enum. No Go path can write a bad value today, and `IsPublic` fails
closed on anything unrecognized, so the read side is safe. What the
schema still permits is the problem: a future migration writing
`'public'` onto a row that holds a secret turns off client
authentication for that app with nothing to catch it, no constraint, no
log, no audit entry, no test.

`000565` adds `CHECK (client_type IN ('confidential', 'public'))` and
`NOT NULL`. The `UPDATE` ahead of it should touch zero rows, since
migration `000344` added the column with a default of `'confidential'`
and backfilled with `COALESCE`; it is there so `SET NOT NULL` cannot
fail on an unexpected row. Both `ALTER`s take `ACCESS EXCLUSIVE` and
scan a table holding one row per registered OAuth2 client, so the lock
is brief.

## The second migration, and why it aligns the way it does

Two columns describe the same fact and can currently contradict each
other.

`token_endpoint_auth_method` is the client's own declaration: registered
client metadata under RFC 7591 §2, where `"none"` is defined to mean the
client is public and has no secret. `client_type` is Coder's derived
copy, and it is what the token endpoint enforces on. RFC 7591 defines no
`client_type` metadata field; the column exists only as a
denormalization.

Registration used to persist the declaration verbatim while hardcoding
`client_type` to `'confidential'`, so rows exist declaring `"none"` on a
client stored confidential that was issued, and still requires, a real
secret. A client that reads its own metadata and believes it is public
will drop that secret and stop being able to exchange codes.

`000566` aligns the declaration to what is enforced, not the reverse.
Deriving enforcement from the declaration would reclassify every such
client as public and stop requiring the secret it holds, which is a
silent authentication downgrade. The down migration is deliberately
empty: the previous values are not recorded, and restoring them would
only reinstate metadata that tells a client to authenticate in a way the
server rejects.

## Application changes

`SET NOT NULL` changes the generated field from `sql.NullString` to
`string`, so the three write sites are updated to match. That is the
entire application diff and no behavior depends on it.

Refs
https://linear.app/codercom/issue/ENG-3029/oauth2-support-public-client

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 10:49:02 -07:00
Ethan b3485d9b3a chore: add agents_allowed to templates (#27284)
Relates to CODAGT-713

This adds `templates.agents_allowed` as a default-true, auditable template attribute, along with nullable database filtering. Migration `000562` translates the effective legacy `agents_template_allowlist` state for existing templates: a valid nonempty list allows matching templates and blocks the rest, missing or empty values leave templates allowed, whilst corrupt values fail closed by blocking all existing templates. As per the linear issue, new templates deliberately default to allowed under the per-template model.

This is the database-only first PR in the stack. #27285 makes the field authoritative in the API and chatd whilst temporarily retaining the compatibility routes needed by the shipped frontend. Later PRs migrate the UI, remove the legacy storage, routes, SDK types, and utility, then add CLI flags.
2026-08-06 14:04:23 +10:00
Bobby HoandClaude Sonnet 5 d814dfad88 feat(coderd): support public OAuth2 client tokens at the schema layer (#27712)
Layer 1 of a multi-PR split of #27195 (public/secretless PKCE-only
OAuth2 clients), broken up for easier review: **database schema (this
PR)** → oauth2provider handler logic → API/e2e integration tests.

## Goal

Coder's OAuth2 provider only works correctly for confidential clients
today. Public clients — native apps that can't safely hold a shared
secret, such as the CLI's browser-based login flow, IDE plugins (VS
Code, JetBrains), desktop apps, and MCP clients — cannot complete a real
OAuth2 flow against Coder, even though OAuth 2.1 §2.1 explicitly defines
this client type and RFC 8252 §8.5 requires PKCE alone to be sufficient
authentication for it. Every MCP client, CLI login flow, and IDE plugin
is a public client by construction, and none of them can complete a
secretless flow against Coder today: dynamic registration always
classifies a client as confidential regardless of what it asks for, the
token endpoint unconditionally requires a `client_secret`, and discovery
metadata never advertises `"none"` as a supported auth method.

Full write-up:
[ENG-3029](https://linear.app/codercom/issue/ENG-3029/oauth2-support-public-client)

### Overall design (end state across the full PR stack)

`[PR2]` marks handler-layer changes landing in the next PR in this
stack. The green box is what this PR implements.

```mermaid
sequenceDiagram
    autonumber
    participant C as Public Client (CLI/MCP/IDE plugin)
    participant S as coderd (chi router)
    participant H as oauth2provider handlers
    participant DB as PostgreSQL

    Note over C,S: Discovery
    C->>S: GET /.well-known/oauth-authorization-server
    S->>H: GetAuthorizationServerMetadata()
    Note over H: [PR2] add "none" to<br/>the returned auth methods list
    H-->>C: [PR2] 200 { token_endpoint_auth_methods_supported:<br/>[..., "none"] }

    Note over C,S: Dynamic Client Registration
    C->>S: POST /oauth2/register<br/>{redirect_uris, token_endpoint_auth_method: "none"}
    S->>H: CreateDynamicClientRegistration()
    Note over H: [PR2] client type now reads<br/>the request -> "public"
    Note over H: [PR2] skip secret generation<br/>for public clients
    H->>DB: [PR2] INSERT app row<br/>(client_type = 'public')
    DB-->>H: app row
    Note over H: [PR2] skip secret insert entirely
    H-->>C: [PR2] 201 { client_id }<br/>(no client_secret field)

    Note over C,S: Authorization Code + PKCE flow
    C->>S: GET /oauth2/authorize?client_id=...&code_challenge=...
    C->>S: POST /oauth2/tokens (grant_type=authorization_code)<br/>no client_secret
    S->>H: extractTokenRequest()
    Note over H: [PR2] client_secret no longer required<br/>for public clients
    H->>H: authorizationCodeGrant()
    Note over H: [PR2] skip secret lookup for public clients
    Note over H: PKCE verification — already mandatory, unchanged
    rect rgb(198, 239, 206)
    Note over H,DB: [THIS PR] oauth2_provider_app_tokens.app_id<br/>column added (NOT NULL, populated at insert<br/>time from app.ID) and app_secret_id loosened<br/>to nullable. Revocation now checks app_id<br/>directly. Confidential-client behavior is<br/>unchanged — no public client can be created yet.
    H->>DB: [PR2] INSERT refresh token row<br/>(no secret reference, for public clients)
    end
    DB-->>H: token row
    H-->>C: 200 { access_token, refresh_token }
```

## This PR: database schema

A public client has no `client_secret`, so it has nothing to put in
`oauth2_provider_app_tokens.app_secret_id`, which was `NOT NULL`. This
PR makes that column nullable and instead attributes a token to its
owning app through a new, always-populated `app_id` column — so
ownership checks (e.g. revocation) work identically for public and
confidential clients without joining through a secret that may not
exist.

| Column | Before | After (this PR) |
|---|---|---|
| `app_secret_id` | `uuid NOT NULL` | **nullable** |
| `app_id` | — | **new**: `uuid NOT NULL`, `FOREIGN KEY →
oauth2_provider_apps(id) ON DELETE CASCADE`, backfilled for every
existing row and populated on every new insert from that point on |

This is a single, complete migration — not staged across multiple PRs.
An earlier version of this branch deferred `app_secret_id`'s nullability
and the insert-time population of `app_id` to a later PR, keeping this
PR's diff limited to `coderd/database`. [Automated
review](https://github.com/coder/coder/pull/27712#discussion_r3686851911)
correctly flagged that as unsafe: the migration would backfill existing
rows once, but nothing would populate `app_id` for rows written
afterward, so the moment this PR merged, new tokens would start
accumulating a permanently `NULL` app_id — and if a release happened to
be cut before the follow-up PR landed, that gap could ship to customers
and would need a second, later backfill to close. Doing the full
migration now avoids that: `app_id` is correct from the first row
written, and the promised `NOT NULL` constraint requires no data repair
because it's already enforced.

Closing that gap requires a few mechanical, non-branching touches
outside `coderd/database`:
- `revoke.go`'s two ownership checks now compare `dbToken.AppID`
directly instead of looking up the app through `app_secret_id` — a
genuine simplification (and slightly less code), not a temporary shim.
- `tokens.go`'s two `InsertOAuth2ProviderAppToken` call sites supply the
new `app_id` column and wrap `app_secret_id` as a `NullUUID`.
- `oauth2_test.go`'s one direct-insert test fixture does the same.

None of these introduce client-type branching or new capability — every
client today is still confidential-only, still always presents a secret,
and behavior is unchanged. The full repo builds, vets, and all existing
tests pass unmodified in behavior.

## Coming next

- **PR2 (handler layer)**: `codersdk`'s `DetermineClientType()` reading
the requested `token_endpoint_auth_method`; `registration.go` skipping
secret generation for public clients (and wrapping the app+secret insert
in a single transaction, fixing a pre-existing
orphan-row/visibility-race gap); `tokens.go` making the secret check
conditional so PKCE alone authenticates a public client; `metadata.go`
advertising `"none"` in discovery. No further migration is needed — the
schema this PR ships is already final.
- **PR3 (API/e2e layer)**: integration tests through the real HTTP API
(`coderd/oauth2_test.go`), the MCP OAuth2 e2e flow
(`coderd/mcp/mcp_e2e_test.go`), and the manual test script
(`scripts/oauth2/test-mcp-oauth2.sh`).

Depends on: #27195 (original combined PR, being superseded by this
stack)

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-05 10:41:06 -07:00
Michael Suchacz 6b8f820493 feat: remove native chat cost tracking in favor of AI Gateway cost data (#27330)
## Stack Context

This stack makes AI Gateway data and budgets the source of truth for AI
spend controls.

1. Re-back the per-chat cost endpoint with AI Gateway data (#27328,
merged).
2. Remove native chat usage limits (#27329, merged).
3. **This PR, now based on `main`:** remove native chat cost tracking
and its dedicated admin UI.

## Summary

Removes native per-message price calculation, model pricing fields, cost
persistence, aggregate cost queries, and admin cost API types. It also
deletes the Analytics and Spend pages plus their legacy redirects. The
AI Gateway-backed per-chat cost row and compact budget indicators
remain.

The spend documentation is renamed to `spend-management.md` and updated
for the remaining surfaces, group budget APIs, CSV export, upgrade
handling for native pricing and cost history, and the absence of a
deployment-wide spend dashboard. The per-chat cost API documents that
data follows AI Gateway retention and reports zero after all matching
requests are purged.

No schema is dropped in this release. `chat_messages.total_cost_micros`
remains nullable and unwritten so replicas from the previous release can
continue inserting messages during rolling upgrades. #27600 tracks
removal after the compatibility window.

> Mux prepared this PR on Mike's behalf.
2026-08-04 12:27:38 +02:00
Michael Suchacz 91c7232d97 feat: add chat suffix messages, idle failure, and content update support (#27428)
Adds generic chat state and query capabilities that the lifecycle hooks
integration (#27429) builds on. Part of the lifecycle hooks stack
(#27401, #27429, #27430).

- `chatstate`: `EditMessage` accepts caller-provided suffix messages
inserted after the replacement in the same transaction, transitions can
carry a typed error kind, and `FinishError` is also allowed from waiting
chats so admission-time failures can park an idle chat in error.
- `chatstate`: `ValidateToolResults` holds the submitted-tool-result
rules (duplicate, invalid JSON, missing, unexpected) in one place, so
`CompleteRequiresAction` and API-level prechecks reject the same
payloads with the same typed causes.
- `database`: `InsertChat` accepts an optional caller-provided ID.

No hook-specific state or behavior is introduced here; these primitives
are usable by any caller.

An earlier revision added a message-content rewrite primitive so a
`pre_tool_use` override could update an already-committed tool call.
Message content is immutable by design, and @hugodutka pushed back on
changing that. The rewrite is gone: #27429 now dispatches the hook
before the assistant message is stored, so the stored input is the one
that runs and nothing needs updating.

> This PR was written by Mux, an AI coding agent, on Mike's behalf.
2026-07-29 13:16:29 +02:00
Michael Suchacz e96d8646e2 fix(coderd): give chat message ids an append-order guarantee (#27495)
Chat message ordering was derived from `created_at`, which is `now()`
and therefore the transaction start time. That makes it unusable as an
append-order column for two independent reasons: every row in one
`InsertChatMessages` batch shares a single timestamp, and two concurrent
transactions can commit in the opposite order to the one they started
in.

This PR gives `chat_messages.id` a real append-order guarantee and moves
the history reads onto it.

## Changes

**`InsertChatMessages` had no input-order guarantee.** Callers index the
returned slice by input position. That only worked because PostgreSQL
happens to evaluate the `BIGSERIAL` default in row order. Ids are now
allocated up front and the k-th smallest is assigned to input index k,
so the pairing does not depend on where the column default is evaluated.
Returned rows are explicitly `ORDER BY id`.

**Three history reads now order by `id`.**

| Query | Was | Now |
|---|---|---|
| `GetChatMessagesByChatID` | `created_at ASC` | `id ASC` |
| `GetChatMessagesByRevisionForStream` | `created_at ASC, id ASC` | `id
ASC` |
| `GetLastChatMessageByRole` | `created_at DESC, id DESC` | `id DESC` |

`GetChatMessagesByChatID` paginated by `id` while ordering by
`created_at`, which is incoherent on its own terms.

The other two matter because of who consumes them. The stream query
supplies incremental updates on the same socket that emits a full
`GetChatMessagesByChatID` snapshot on history reset, so once that
snapshot moved to `id` the two disagreed under timestamp skew.
`GetLastChatMessageByRole` returns an id that is then used as an id
cursor, both as `AfterID` when synthesizing tool cancellations and as
`chats.last_read_message_id`, where a stale anchor leaves later
assistant messages permanently unread.

A tie-breaker would not have fixed either one. It only resolves equal
timestamps; leading with `created_at` is the actual defect.

**`GetLastChatMessageByRole` loses its index, so this adds one.** `ORDER
BY created_at DESC, id DESC` could take an ordered scan of
`idx_chat_messages_chat_created`. Nothing in the schema can supply
`ORDER BY id DESC LIMIT 1` for a given `chat_id` and `role`, so the
planner switches to a backward scan of the primary key and filters every
newer row in the table, scanning all of it when the chat has no message
in that role, which is the routine case for a fresh chat. Migration
`000559` adds `(chat_id, role, id DESC) WHERE deleted = false`, the same
shape as the existing `idx_chat_messages_user_prompts`. This matters
because the query is hot: it runs on every stream connect and
disconnect, and once per turn when synthesizing tool cancellations.

`GetChatMessagesForPromptByChatID` has the same defect and is fixed in
the stacked PR, because its compaction boundary change is semantic and
deserves a separate review. Auto-archive stays timestamp-based
deliberately: it measures activity, not order.

Wrapping the insert in a CTE (needed because `INSERT` cannot take `ORDER
BY`) makes sqlc synthesize `InsertChatMessagesRow`. It is structurally
identical to `ChatMessage`, so the call sites use a direct struct
conversion that stops compiling if the two ever diverge.

## Testing

Behavior tests write `created_at` values inverted against id order, so a
reader that leads with `created_at` returns the batch backwards. All
three queries were verified red by reverting the `ORDER BY` and
regenerating: the stream query returned `[3,2,1]` for `[1,2,3]`, and
`GetLastChatMessageByRole` picked id 1 instead of id 3.

`TestInsertChatMessagesOrderContract` asserts against the generated SQL,
covering what a behavior test cannot: PostgreSQL evaluates the id
default in row order anyway, so a batch still looks ordered once the
guarantee is removed.

`TestChatMessagesSequenceCacheIsOne` guards the cross-batch half of the
invariant. Ids follow chat row lock order only while the sequence hands
out one value at a time; sequence cache blocks are per session, so with
a cache above one a backend holding stale cached values can lock second
and still commit lower ids. Bumping a sequence cache is an ordinary
throughput tweak, and it would silently corrupt history order.

The index was checked on a 200k row fixture. Without it, the zero-match
lookup filters all 200,000 rows over 2763 buffers; with it, the plan is
an index scan with both `chat_id` and `role` in the index condition, no
sort node, and 3 buffers.

Note that the within-batch mapping does not depend on the cache size. It
is established by `ROW_NUMBER() OVER (ORDER BY id)` over the allocated
ids, so it holds regardless of `nextval` evaluation order.

## Note on the deleted subagent hand-sort

The subagent history reader's hand-sort stays deleted, but calling it
redundant was imprecise. It sorted by `created_at` then `id`, so it is
only equivalent to `id` ordering when the two agree. When they disagree
the old code selected a different "latest assistant". This is a
deliberate behavior change to match the new invariant, not dead-code
removal.

> Opened by Mux on behalf of Mike.
2026-07-29 07:17:42 +00:00
Zach 85984ff142 feat: add enable/disable support for user secrets (#27537)
Users can now disable a secret to stop it from being injected into
workspaces without deleting it, and re-enable it later. Disabled secrets
stay visible and editable everywhere they already appear.

An enabled secret must have at least one injection target; a secret with
no target can be stored only while disabled. Existing target-less secrets
are migrated to disabled to preserve current behavior.

Support spans the REST API, SDK, CLI, dashboard, and audit log.
2026-07-28 09:58:33 -06:00
Michael Suchacz 4ed6fcced7 refactor(coderd): stop storing chat gateway key IDs and drop the columns (#27171)
> Mux is working on behalf of Mike.

## Summary

Stop reading and writing the legacy `api_key_id` columns on chat
messages and queued messages, and drop the columns in the same PR.
Runtime AI Gateway attribution continues to use the per-user synthetic
key introduced by #27170.

With the columns gone, `sqlc` generates `database.ChatMessage` and
`database.ChatQueuedMessage` without `api_key_id`, so no transitional
query scaffolding is needed.

Migration `000548` drops the `api_key_id` columns. #27170 already
removed their foreign keys, so the down migration re-adds nullable text
columns without constraints. Previous column values cannot be restored.

Also moves the model config validation in `CreateChat` above the
message-building work so a disabled or invalid model fails fast. On main
this mattered more: the old ordering minted a synthetic API key before
rejecting the request.

Deploy note: replicas still running the previous release write
`api_key_id` on insert, so chat message inserts on old replicas fail
during the rolling window after the column drop. This was previously
split across two PRs to avoid that window; per review feedback the split
added more churn than it was worth for an experimental surface.

Depends on #27170 (merged).
2026-07-20 19:49:19 +02:00
Michael Suchacz 9f4ddea571 feat: revoke MCP server OAuth grants at the provider on disconnect (#27300)
Closes
[CODAGT-805](https://linear.app/codercom/issue/CODAGT-805/revoke-oauth-grants-at-the-source-for-mcp-servers).

The experimental MCP server OAuth2 disconnect endpoint previously
deleted only the stored token row, leaving the grant active at the OAuth
provider. This PR adds provider-side token revocation while keeping
local disconnect independent of provider availability.

## Changes

- Add `mcp_server_configs.oauth2_revocation_url` in migration `000547`.
The value can be configured manually, discovered from RFC 8414 metadata,
and managed through the MCP server settings UI. Non-admin responses
redact it with the other OAuth2 fields.
- Revoke the refresh token first through the RFC 7009 endpoint, then
fall back to the access token only for `unsupported_token_type`. Public
clients send `client_id`; confidential clients use
`client_secret_basic`.
- Delete the local token transactionally before best-effort provider
revocation. Callers without a token receive the same response for hidden
and nonexistent config IDs, and provider failures return a generic
warning without exposing provider response bodies.
- Require HTTPS revocation endpoints except for HTTP loopback URLs.
Redirects must preserve the POST and remain on the configured origin.
Redirect errors omit provider-controlled paths and query strings so
reflected token material cannot enter logs.
- Treat `200 OK` and `204 No Content` as completed revocations. `202
Accepted` remains a failure because it does not confirm completion.
- Prevent an in-flight refresh from recreating a token deleted by
disconnect. Refresh persistence now uses an optimistic update keyed by
token ID and `updated_at`; only the OAuth callback can create a token
row. Refresh conflicts reload the current row or clear in-memory auth
when disconnect deleted it.
- Return `{token_revoked, token_revocation_error}` from disconnect,
while retaining SDK compatibility with the legacy `204` response. The UI
surfaces provider revocation failures as warning toasts.
- Document revocation endpoint discovery, HTTPS requirements, and
best-effort disconnect behavior.

No token or no configured revocation URL returns `token_revoked: false`
without an error, so disconnect remains idempotent.

> Updated by Mux, an AI coding agent, on Mike's behalf.
2026-07-20 00:14:03 +02:00
Callum Styan ad29777cb2 feat: NATS mTLS pubsub implementation (#26902) 2026-07-13 11:00:02 -07:00
Danielle Maywood d66e4d794f feat: add configurable reasoning effort to Coder agents (#26974) 2026-07-09 23:35:12 +01:00
Danny Kopping ef0b5585d5 feat: record and expose terminal upstream interception errors (#26961)
Categorises the terminal error of a failed interception and persists it
on the interception record, then surfaces it on the AI Gateway API.

- Categorise into an enum (`bad_request`, `unauthorized`,
  `rate_limited`, `overloaded`, `server_error`, `unknown`), unwrapping
  the ResponseError envelope, the upstream Anthropic/OpenAI SDK errors,
  and key-pool exhaustion so blocking and streaming paths agree.
- Thread the type and raw message through the recorder dRPC into the
  `aibridge_interceptions` row (optional proto fields; NULL on success).
- Expose the error on the AI Gateway thread API from the root
  interception.

*This PR was produced by opencode (agent) using the `anthropic/claude-opus-4-8` model, under human direction and review.*
2026-07-09 15:36:56 +02:00
Danielle Maywood d51762440b feat: add custom AI provider icons and instance-based model picker grouping (#27026) 2026-07-06 23:00:09 +01:00
Cian Johnston b21e0717d5 feat: remove chat chain mode (#26980)
Removes OpenAI Responses "chain mode" from chatd. Closes CODAGT-445.

- Deletes `chatopenai/responses.go` (chain detection, activation, prompt filtering, response ID extraction) and its tests.
- Deletes the `ChainBroken` classification in `chaterror` and the chatloop retry bookkeeping that disabled chain mode mid-generation.
- Drops the `chain_broken` label from the `coderd_chatd_stream_retries_total` metric.
- Stops reading and writing `chat_messages.provider_response_id`
- Deletes the dead `ClearChatMessageProviderResponseIDsByChatID` query. Dropping the column is a follow-up migration.
- Deletes three chatloop hooks no caller sets (`ReloadMessages`, `DisableChainMode`, `PrepareMessages`), the dead `const AgentChatContextSentinelPath`, and stale chain-mode comments.

🤖 Generated by Coder Agents on behalf of @johnstcn.
2026-07-06 11:57:12 +01:00
Danny Kopping dd216b96fe feat: record provider_item_id for tool usage (#26856)
## Summary

Plumbs the Responses output item id (added as `ToolUsageRecord.ItemID` in #26855) through to the database, captured independently of the `provider_tool_call_id` correlation key. Hosted tools (`web_search_call`, etc.) only have an item id; agentic tools have both.

`provider_item_id` is specific to the OpenAI Responses API; it stays empty for chat completions and Anthropic messages, which have no separate item id.

## Changes

- Migration `000534`: nullable `provider_item_id` column on `aibridge_tool_usages`.
- Proto: `item_id` field 11 on `RecordToolUsageRequest`.
- Server handler: persists `provider_item_id` and adds it to structured logging.
- Translator: maps `ToolUsageRecord.ItemID` to the proto field.

## Tests

- `TestRecordToolUsageProviderItemID`: real-database round-trip asserting `provider_item_id` persists for both hosted and agentic tools, independently of `provider_tool_call_id`.

Stacked on #26855. Linear: AIGOV-96

---

_This PR was produced by opencode (agent) using the_ _`anthropic/claude-opus-4-8`_ _model, under human direction and review._
2026-07-06 09:29:07 +02:00
Mathias Fredriksson 047c47495b refactor: drop chat_model_configs provider column (#26877)
The provider type already lives authoritatively in ai_providers.type,
reachable on every active row through ai_provider_id, which the
chat_model_configs_ai_provider_required_when_active CHECK makes
mandatory. The stored provider string was a denormalized copy the system
kept in sync with a startup backfill and no longer needs.

Every surface now derives provider type from the linked ai_providers
row. Telemetry is the one exception: it keeps emitting provider, now
sourced from ai_providers.type via a JOIN, so the BigQuery column and the
Nexus dashboards that read it are unaffected. The experimental HTTP/SDK
response drops provider and makes ai_provider_id required, since those
endpoints return only active configs; consumers resolve provider type
from ai_provider_id and the AI providers listing.

This ships in a single release with no compatibility window: production
reads the table via SELECT *, so a pre-drop binary fails config reads the
moment the column is gone. Operators must scale to zero before upgrading,
and there is no rollback.

Closes CODAGT-599
2026-07-01 15:59:55 +03:00
Sas Swart 491a75294e feat: GET /api/v2/agent-firewall/sessions/{id} (#24814)
Add a GET endpoint at `/api/v2/agent-firewall/sessions/{id}` that
returns agent firewall session metadata (`id`, `workspace_id`,
`owner_id`, `confined_process`, `started_at`). The handler authorizes
against the `boundary_log` resource with `ActionRead` via dbauthz.

The endpoint is enterprise-only, gated behind the `FeatureBoundary`
entitlement.

The `GetBoundarySessionByID` SQL query JOINs through `workspace_agents`
→ `workspace_resources` → `workspace_builds` → `workspaces` to return
`workspace_id` and `workspace_owner_id` directly, avoiding a separate
query.

Also adds an `owner_id` column to the `boundary_logs` table (migration
000526) with a FK to `users(id)` and a backfill from
`boundary_sessions`. This enables user-scoped RBAC authorization for
`InsertBoundaryLogs` via `.WithOwner()`, ensuring workspace agents can
only insert logs for their own owner.

Depends on #24810

**RBAC behaviour:**

| Role    | Result |
|---------|--------|
| Owner   | read   |
| Auditor | read   |
| Member  | 404    |

> [!NOTE]
> This PR was authored by Coder Agents.
2026-06-18 20:50:17 +02:00
Ethan 35af54d6aa test: isolate passive chatd internal tests (#26369)
Make `newInternalTestServer` use option functions for logger, clock, and
worker startup, and make it passive by default so internal chatd tests
only opt into background execution when they need a real worker.

Use the passive server path in `TestAwaitSubagentCompletion` for the
state-driven subtests, keep `ContextCanceled` explicitly active for real
provider cancellation coverage, and keep the fail-fast default AI
provider base URL so accidental provider calls still fail immediately.

Closes CODAGT-586
Closes https://github.com/coder/internal/issues/1549
2026-06-18 00:21:09 +10:00
Yevhenii Shcherbina b6fcb9a30a feat: record cost on aibridge token usages (#26229)
Implements
https://linear.app/codercom/issue/AIGOV-286/add-interception-cost-calculation-to-aibridge-token-usages

Adds spend attribution to AI Gateway. After the upstream response, each
token-usage record now captures the user's effective group, the
per-token prices in effect at that moment, and a computed cost — so
spend is recorded as an immutable, point-in-time snapshot.

Concretely, `aibridge_token_usages` gains `effective_group_id`,
`input_price_micros`, `output_price_micros`, `cache_read_price_micros`,
`cache_write_price_micros`, and `cost_micros`. When a usage record is
written, the effective group is resolved (per-user override, else the
deployment budget policy), the `(provider, model)` price is looked up
and snapshotted onto the row, and cost is computed from the
provider-reported token counts. A model that isn't in the price table
records its tokens with a `NULL` cost; any *other* resolution failure
fails the write, so a `NULL` cost unambiguously means "model not priced"
rather than "lookup errored."

All values are stored in micro-units (1 unit = 1,000,000 micro-units;
Phase 1 assumes USD, so 1 micro-unit = $0.000001). Prices are quoted per
million tokens.

This also grants the AI Bridge RBAC subject `read` on `ai_model_prices`
(the per-interception price lookup needs it; it previously only had
`update` for the startup seeder).

## Cost precision

Cost is computed per token category as `tokens × price / 1_000_000` with
integer division, then the four categories are summed. The division is
done **per category** (not once over the summed numerator) on purpose:
it keeps the per-category line items summing exactly to the stored total
— no "the parts don't add up to the whole" in reporting).

Integer division truncates sub-micro-unit fractions. For example, a
cheap model at $0.10 per million tokens is a price of `100_000`; 9
tokens cost `9 × 100_000 / 1_000_000 = 900_000 / 1_000_000 = 0` (the
true 0.9 micro-units floors to 0). At real list prices this rarely bites
— $3/M input is a price of `3_000_000`, so even a single token is 3
micro-units. The per-record under-count is bounded below 1 micro-unit
per category, so under $0.000004 total across the four categories, which
is acceptable for list-price-based cost approximation.

## Overflow safety

`cost_micros` is a `BIGINT` (int64), and the largest intermediate value
is a single category's `tokens × price` before division. int64's ceiling
is ≈ `9.223e18`.

- At a steep $75/M model (price `75_000_000`), overflow would require
~123 billion tokens in one response: `123e9 × 75e6 = 9.225e18`, just
over the limit. `122e9` stays under at `9.15e18`.
- A realistically maxed-out Opus 4.8 response (≈1M input + 128K output
at list prices) costs about $15, with a numerator around `1.5e13` —
roughly six orders of magnitude below the ceiling.

So overflow is unreachable from real token counts.

### Multi-currency support

In the future, we may encounter issues with multi-currency support,
especially when dealing with currencies that have very large exchange
rates relative to USD, for example:

IRR: ~1,300,000 IRR ≈ 1 USD
VND: ~26,000 VND ≈ 1 USD

For currencies with such large denominations, numeric overflow is
technically possible, considering that we have only about six orders of
magnitude of headroom before reaching the limit (see above).

## `effective_group_id` has no foreign key

`effective_group_id` records the group a spend was attributed to, as an
immutable historical fact. It is intentionally **not** a foreign key, so
the record survives deletion of the group.

Alternatives were considered and rejected:

- **`ON DELETE SET NULL`** would mutate an "immutable" record — deleting
a group silently erases that interception's attribution and under-counts
the group's historical spend.
- **`RESTRICT` / `NO ACTION`** would block group deletion entirely
(groups are hard-deleted).
- **`CASCADE`** would delete spend history when a group is deleted — the
worst outcome for an audit record.

There is also no insert-time check that the group still exists: the id
comes from a budget that was just resolved, meaning it was valid at some
point.

## Open question: group name snapshotting

Should we also snapshot the group *name* onto each record? Two options:

- **Denormalize it now** — readable in historical reports even after a
group is deleted, but the snapshot can drift from the current name on
rename, raising a "show point-in-time vs. current name" question.
- **Postpone until needed** — it's a purely additive column later, and
the name is display-only (not correctness-bearing like the price). The
cost: names of groups deleted before the column is added can't be
backfilled.

Leaning toward postponing until a concrete reporting need settles the
drift question.
2026-06-16 20:09:00 +00:00
Sas Swart 2716e2181c feat: purge boundary logs past retention (#24815)
Add a periodic purge job for `boundary_logs` rows past their retention
threshold, following the same pattern as the existing audit log and
connection log purge jobs in `dbpurge`.

Expose a `--boundary-log-retention` deployment flag (env
`CODER_BOUNDARY_LOG_RETENTION`, YAML `retention.boundary_logs`). Default
is `0` (keep indefinitely). When set to a positive duration, `purgeTick`
deletes rows where `captured_at` is older than the threshold in batches
of 10,000, matching other log purge operations. The `boundary_logs`
label is added to the `records_purged_total` Prometheus counter.

Also removes the random-UUID fallback for `OwnerID` in
`dbgen.BoundarySession`. The previous fallback generated a UUID that
could never satisfy the `boundary_sessions_owner_id_fkey` FK constraint,
masking test setup bugs. Callers must now provide a valid user ID or
accept NULL (the legitimate "user deleted" state).
2026-06-16 14:32:54 +02:00
Danny Kopping a1330e3a8c refactor: rename Ai* database identifiers to AI* (AIGOV-369) (#26327)
Adds `ai` to sqlc's `gen.go.initialisms` in `coderd/database/sqlc.yaml`
so the generated DB code follows Go's initialism convention. Adds the
matching `ai` -> `AI` case to the dbgen PascalCase helper
(`scripts/dbgen/main.go`) so the corresponding `dbmem` / mock
identifiers stay in sync. `make gen` regenerates the rest; hand-written
call sites that consume DB-generated identifiers
(`enterprise/audit/table.go`, `coderd/database/modelmethods.go`,
`enterprise/coderd/aigatewaykeys.go`, `coderd/database/dbauthz/*`, etc.)
are updated to match.

Scope is deliberately limited to the database layer:

- `coderd/rbac/*` (resource and scope generators) is untouched —
`ResourceAi*` / `ScopeAi*` constants stay on main's casing.
- `codersdk/*` (Go SDK) is untouched — `codersdk.ResourceAi*` /
`codersdk.APIKeyScopeAi*` constants stay on main's casing, so external
Go SDK consumers see no source-level break.
- `Aibridge*` identifiers (one SQL token `aibridge`, not `ai_bridge`)
are out of scope.

On-the-wire values are unchanged: enum strings, RBAC resource type
strings, API key scope strings, and JSON tags all stay the same. The
HTTP/JSON surface is unaffected.

Refs:
[AIGOV-369](https://linear.app/codercom/issue/AIGOV-369/change-ai-references-in-coderddatabasemodelsgo-to-ai)

🤖 Generated with [Coder Agents](https://coder.com)
2026-06-16 09:01:43 +00:00
Sas Swart e1c7e61eb9 feat(coderd): add Agent Firewall correlation columns to aibridge_interceptions (#24817)
Add `agent_firewall_session_id` (UUID NULL) and
`agent_firewall_sequence_number` (INT NULL) to `aibridge_interceptions`
with a partial index on `agent_firewall_session_id`. No FK to
`boundary_sessions` (soft reference, resolved at query time).
`RecordInterception` reads the new fields from the proto request (merged
in #25884) via `parseOptionalUUID` / `parseOptionalInt32` helpers.

> This PR was authored by Coder Agents.
2026-06-15 12:34:16 +02:00
Hugo Dutka 4debd23cbb fix: chatd refactor (#26270)
Implements the chatd stabilization RFC.

Combines:
- https://github.com/coder/coder/pull/25908
- https://github.com/coder/coder/pull/25923
- https://github.com/coder/coder/pull/26109
- https://github.com/coder/coder/pull/26110
- https://github.com/coder/coder/pull/26111
- https://github.com/coder/coder/pull/26112
2026-06-12 13:33:12 +02:00
Steven Masley 938c2080f3 feat: configurable default org member roles (#25994)
Refs #25936. 
Adds a configurable per-org default member role set. Unioned into each member's effective roles at read time.

<sub>with Coder Agents on behalf of @Emyrk.</sub>
2026-06-05 14:33:13 -05:00