Commit Graph
10 Commits
Author SHA1 Message Date
Thomas Kosiewski f923f4a525 chore!: remove the chat automations experiment (#30412)
Removes the experimental Chat Automations feature (`chat-automations` experiment) from main. This reverts the 16-PR stack that added it and adds two forward migrations that undo its schema and credential changes.

> [!WARNING]
> **Upgrade consequences**
>
> - **Automation data is deleted.** Migration 000615 drops the `chat_automations` table and deletes every queued message an automation delivered. Chats, their messages and ordinary queued messages stay. Messages lose their automation provenance (`automation_id`, `input_id`).
> - **Automation-only credentials are revoked.** Migration 000616 deletes API keys, OAuth2 codes and OAuth2 tokens whose only scopes (or allow-list entries) were `chat_automation:*`. Mixed credentials keep their other scopes. Custom roles lose their `chat_automation` permissions. OAuth2 apps that allowed only `chat_automation` scopes get the whitespace allowlist that grants nothing, as in 000595.
> - **A coordinated restart is required.** Stop all old replicas before the new version runs the migrations. Old replicas read the dropped table and columns, so a rolling deploy is not supported.
> - **Remove `--chat-max-automations-per-owner` before upgrading.** The flag, the YAML key `chat.maxAutomationsPerOwner` and the env var `CODER_CHAT_MAX_AUTOMATIONS_PER_OWNER` are removed. A deployment that still sets the flag or the YAML key will not start (`unknown flag` / `unknown option`). The env var is ignored. This includes dogfood. No release or tag contains the flag.
> - **Down migrations restore the schema only.** 000616 down and 000615 down recreate the 000611 schema with empty tables. They do not restore deleted automations, webhook secrets, provenance, queued deliveries, revoked credentials, stripped permissions or the deleted experiment rule.

## What is removed

- The `chat-automations` experiment and its stored runtime rule (`experiment_rule:chat-automations`).
- The `--chat-max-automations-per-owner` deployment option.
- The `chat_automation` RBAC resource, its member and chatd grants, and the `chat_automation:*` API key scopes.
- The automation sqlc queries, dbauthz wrappers and audit table entries.
- The management routes under `/api/experimental/organizations/{organization}/chat-automations`, and the webhook route `POST /api/experimental/chat-automations/{automation}/events` with its rate limiter.
- The chatd management service, webhook publish, schedule loop, the `manage_automations` tool, and the chatstate admission seam and queue promotion guard that only automations used.
- The Agents Automations page, editor, nav entry, chat labels, the Manage automations toggle and the API client.
- The `archived:any` chat search value that the automations "View chats" dialog added.
- `docs/ai-coder/agents/automations.md`, its manifest entry, the glossary entry and the related rows.

Reverted PRs: #30085, #30096, #30112, #30117, #30127, #30134, #30160, #30166, #30167, #30171, #30187, #30207, #30214, #30217, #30219, #30220.

## What stays, and why

- Migrations 000609, 000610 and 000611, their fixture, and the #30297 lock fix with `migration000609_test.go` stay unchanged. Deployments such as dogfood already ran them.
- New migration **000615** reverses the schema changes of 000609 and 000611 against the current schema. It uses the same short-wait lock-and-retry block as 000609, and it rebuilds `chats_expanded` from main's current definition minus the two automation columns. New migration **000616** follows the 000595 Tasks removal for credentials and recreates `api_key_scope` without the five `chat_automation:*` values. It keeps the `chat_project_memory:*` values from 000612.
- The `chat_automation` value stays in the `resource_type` enum, and `codersdk.ResourceTypeChatAutomation` stays. Old audit log rows stay readable, as with `task` after the Tasks removal.
- The 000615 delete fires the existing queue-version trigger, so `queue_version` moves to `snapshot_version` for each affected chat. `snapshot_version` is not bumped: the coordinated restart restarts every stream loop from an empty state, and the site does not read the version fields.
- `coderd/x/chatd/ARCHITECTURE.md` drops the sections the stack added and gets back its pre-stack wording, merged with the later main edits to the same file.

## Notes for reviewers

- #30379 (chat project page) merged while this PR was in progress. It passed `manageAutomationsEnabled` in `AgentCreatePage.test.tsx` and listed an `automations` route in `AgentsPageLayout.test.tsx`. Later main commits also added automation props in `ConversationTimeline.test.tsx` and `ConversationTimelineWorkingBlocks.stories.tsx`. This PR removes all of them.
- Remaining `automation` matches under `coderd`, `codersdk`, `enterprise`, `site/src` and `docs` are intended:
  - migrations 000609 to 000611, their fixture and `migration000609_test.go`;
  - the new 000615 and 000616 migrations and `migration000615_test.go`;
  - the `chat_automation` resource type in `dump.sql`, `models.go`, `codersdk/audit.go`, the generated API docs, `swagger.json` and `typesGenerated.ts`;
  - unrelated uses of the word "automation" (docs, `chatlabels_test.go`, `ratelimit.go`, `autostop.go`, `subagent_catalog.go`, `WorkspaceSharingSection.tsx`, `AgentChatPage.stories.tsx`).

Remote UAT on an isolated dev deployment passed. It upgraded a populated pre-removal database, then checked chats, queued messages, tokens, the 404 routes, the removed page and nav entry, and the experiments list.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_

![Agents sidebar after the upgrade, with no Automations entry](https://github.com/user-attachments/assets/bd446035-7fc8-4ca4-a735-3a4f36380738)
2026-10-11 19:57:49 +02:00
Jakub Domeracki b4fb7901e6 docs: clarify that disable-password-auth also applies to owners (#30362)
The `--disable-password-auth` help text says owners can still sign in
with a password "to avoid potential lock out", and a comment in
`coderd/userauth.go` says the same. Neither is true. #6193 removed the
owner bypass on purpose, so the flag blocks password login for everyone.
The recovery advice was also wrong. It named a command that doesn't
exist (`create-admin`), and any admin that `create-admin-user` creates
while the flag is on still can't sign in.

This PR changes docs and comments only:
- Rewrites the flag description to say it applies to owners too and how
to recover access. Regenerated the CLI and config reference docs and the
golden files.
- Fixes the misleading comment in `coderd/userauth.go`.
- Adds the same warning and recovery steps to
`docs/admin/users/password-auth.md`.

> [!NOTE]
> Generated by Coder Agents on behalf of @jdomeracki-coder.
2026-10-07 19:03:46 +02:00
Thomas Kosiewski 9316d67440 feat: add chat automations management API behind chat-automations (#30112)
Adds the management API for chat automations under
`/api/experimental/organizations/{organization}/chat-automations`,
behind the new user-scoped `chat-automations` experiment. This is the
first of two PRs for RFC step 3; nothing publishes automation messages
yet.

- **Experiment:** `chat-automations` is in `ExperimentsKnown`,
`ExperimentsUserScoped`, and the ruleguard list in `scripts/rules.go`.
Every route asks `chatd.AutomationsEnabled(ctx, api.ExperimentEvaluator,
callerID)`, which fails closed on a missing subject or evaluator; the
routes stay mounted and return 404 when the experiment is off for the
caller, so a runtime rule for one user applies on the next request
without a restart. There is no `/api/v2` alias.
- **Routes:** list, create (201), get, `PATCH`, delete (204), with
Swagger annotations and audit entries. List returns the automations in
the organization the caller can read (members see their own,
organization admins see all).
- **Service** (`coderd/x/chatd/automations.go`): create validates kind
and target shape, a non-empty prompt, a five-field cron and IANA time
zone, model access, and that an `existing_chat` target is a root chat of
the owner in the same organization. It defaults `when_busy` to `skip`
for schedules and `queue` for webhooks and computes the first schedule
cursor. `PATCH` runs the same per-field validation on each changed
field, rejects `kind` and `target_mode`, bumps `schedule_revision` on
prompt, cron, or time zone changes, and recomputes the cursor on cron or
time zone changes.
- **Owner-only:** only the owner can `PATCH` an automation, even with
organization-admin or site-owner roles (403); admins can list, read, and
delete.
- **Secrets:** a webhook's secret (`coder_automation_` prefix) is
returned once in the create response. Only its SHA-256 is stored, and it
appears in no list, get, audit, or log output.
- **Cap:** new option `--chat-max-automations-per-owner` (default 50)
caps automations per owner across all organizations; creating one more
returns 409. Creates serialize on a per-owner advisory lock.
- **Cron:** `cron.Standard(spec, timeZone)` accepts any standard
five-field expression (the existing `Weekly` rejects day-of-month and
month restrictions). Responses include the next five run times of an
enabled schedule.

PR 3b (stacked on this one) adds disable and re-enable with the queue
cutoff, makes delete run disable first, and adds secret rotation and the
schedule preview route. Until then, the queue promotion guard from
#30096 discards queued rows whose automation was deleted.

Dogfood UAT ran on a real dev instance and passed:
create/list/get/patch/delete via curl, owner-only 403s for an
organization admin and the site owner, the cap (including concurrent
creates and cross-organization counting), a runtime rule turning the
experiment off for one user without a restart, and `/api/v2` 404s.

Part of CODAGT-1209 (PR 3b closes it)
Part of CODAGT-1135
Refs CODAGT-1217

Stack: PR 3 of the Automations for Coder Agents stack, on top of #30096
(which is on top of #30085).

> This PR was written by an AI agent (xum) and reviewed by the agent's
own test audit and UAT. A human has not reviewed it yet.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model:
`anthropic:claude-opus-5-5` • Thinking: `high`_

---------

Signed-off-by: Thomas Kosiewski <tk@coder.com>
2026-10-02 15:54:49 +00:00
Nick Vigilante 302edffd87 feat: add JSON output to coder server create-admin-user (#30138)
`coder server create-admin-user` is mostly used in automation but only
printed text.

It now accepts `-o text|json`.
JSON contains the created user's ID, username, email, roles, and
organization memberships.

**Behavior change:** Additive.
The default text output is unchanged and still goes to stderr.
JSON goes to stdout so it can be piped.
The password is never included in any format.

<details>
<summary>Decisions</summary>

- The command talks to the database directly, so the JSON uses a small
local response type instead of a codersdk type.
- JSON goes to stdout while text stays on stderr, matching how other
commands emit machine-readable output.

</details>

Closes [CLI-17](https://linear.app/codercom/issue/CLI-17)

_This PR was generated by Coder Agents on behalf of @nickvigilante._
2026-09-30 12:36:05 -04:00
Cian Johnston 9fa17a12ef fix: strip client actor headers unconditionally (#30037)
Strip all client-supplied headers matching `IsActorHeader()` in
`PrepareClientHeaders`, regardless of `send_actor_headers`. This
reserves the case-insensitive `X-AI-Bridge-Actor` prefix for identity
injected by the gateway. Remove the redundant deletion of the three
standard actor headers from `BuildUpstreamHeaders`.

Generated with Coder Agents on behalf of @johnstcn.
2026-09-29 20:26:17 +01:00
Cian Johnston 8742d2484c feat: add opt-in AI Gateway actor email headers (#29930)
Add opt-in forwarding of the authenticated Coder account email through
`--ai-gateway-actor-header-meta-email`,
`CODER_AI_GATEWAY_ACTOR_HEADER_META_EMAIL`, or
`ai_gateway.actor_header_meta_email`. The option defaults to empty:
enabling `send_actor_headers` alone does not forward email. Set a
destination header explicitly to opt in; accounts without an email omit
it.

Carry authenticated email through the Gateway authorization response and
strip client-supplied standard/configured email headers before applying
trusted identity. The additive protocol field joins unreleased Gateway
API v1.3 alongside workspace attribution. Includes email configuration
precedence, authorization compatibility, captured upstream request
tests, and matching documentation.

Refs https://linear.app/codercom/issue/AIGOV-601

Generated with Coder Agents on behalf of @johnstcn.
2026-09-29 20:07:49 +01:00
Cian Johnston 05d1fa8a4c feat: configure AI Gateway actor identity headers (#29929)
Allows configuring AI Gateway actor ID and username headers independently through
`--ai-gateway-actor-header-id` and
`--ai-gateway-actor-header-username` or associated environment
variables and YAML settings. 

- Defaults to existing header name, an empty string disables that attribute. 
- Existing `send_actor_headers` gate remains off by default.
- Validates header names, case-insensitive collisions, and protected
destinations for embedded and standalone gateways.
- Strips standard and configured identity destinations before injecting authenticated values.
- Adds coverage for configuration precedence, captured upstream request, and generated help/API documentation.

Refs https://linear.app/codercom/issue/AIGOV-601

Generated with Coder Agents on behalf of @johnstcn.
2026-09-29 18:22:59 +01:00
Sas Swart 813b8c151f feat: make AI Gateway content recording optional (#30031)
Add `--ai-gateway-disable-content-recording`, which stops user prompts,
tool
calls and model thoughts from being stored. Cost control reads none of
those
tables, so budgets, spend accounting and enforcement are unaffected.

A dropped record is still logged, so pairing the option with
`--ai-gateway-structured-logging-source=gateway` keeps the SIEM export
for
records that are never persisted. The gateway warns at startup when
content is
dropped while coderd is the only emitter, since those records then reach
neither
the database nor a SIEM.

Covered end to end for the embedded and the standalone gateway, and
documented
alongside cost controls.

<details>
<summary>Why the stack is ordered this way</summary>

The final design is a chain of recorder decorators:

    +---------------------+
    |    AsyncRecorder    |  all records except the interception start
    +---------------------+
    |     LogRecorder     |  emit the structured record
    +---------------------+
    | ValidatingRecorder  |  refuse malformed records
    +---------------------+
    |  caller middleware  |  record policy, e.g. drop content
    +---------------------+
    |    TraceRecorder    |
    +---------------------+
    |    DRPCRecorder     |  acquire client, send to coderd
    +---------------------+

Ordering constraints:

- The record type constants (1) are the vocabulary both emitters and the
  validator use, so they come first.
- Validation (5) rejects unset timestamps, so records must already be
stamped
where they are created (3). Stamping at creation leaves
`WrappedRecorder`
  doing nothing but client acquisition, which is why it is folded into
  `DRPCRecorder` (4) and not before.
- coderd's server-time fallback (2) is what keeps a gateway of an older
version
  from persisting zero times once the gateway owns stamping (3).
- PRs 1 to 5 change no user-visible surface. Each deployment option then
lands
with the code that reads it: the log source with gateway-side emission
(6),
content recording with the filter that drops the records (7). Pool
options are
plumbed to the serving pool in 6, where the first option that needs them
  arrives, rather than as an unused hook beforehand.

</details>

---
🤖 Raised by [Coder Agents](https://coder.com) on behalf of @SasSwart.
2026-09-29 11:48:53 +02:00
Sas Swart 23fb35d185 feat: let the AI Gateway emit the structured interception records (#30030)
coderd emitted the structured interception records as they arrived over
DRPC, so
a record the gateway never sends cannot be exported.

- Add `--ai-gateway-structured-logging-source` (`coderd`, `gateway`,
`both`) and
teach the gateway's `LogRecorder` to emit the same format where the
records
originate. Gateway lines omit `thread_parent_id` and `thread_root_id`,
which
  are resolved by a database lookup only coderd can perform.
- Reaching the serving pool required repairing the pool options: they
were
derived at construction sites and discarded, so the server always built
its
pool with the defaults. They are now derived from the deployment
configuration
in one place and passed to the pool that serves requests. The server
reports
those options, and its ownership of the `key_pool_state` collector, so a
second
  registration cannot silently reappear and panic startup.
- The standalone gateway inherits both structured logging options, since
it is
  now one of the processes that can emit.

This is the keystone of the stack: the flag, the mechanism that carries
it and
the behavior it selects land together, so
`TestAIGatewayStart_InheritedOptions`
and the gateway help output stay truthful at every commit.

<details>
<summary>Why the stack is ordered this way</summary>

The final design is a chain of recorder decorators:

    +---------------------+
    |    AsyncRecorder    |  all records except the interception start
    +---------------------+
    |     LogRecorder     |  emit the structured record
    +---------------------+
    | ValidatingRecorder  |  refuse malformed records
    +---------------------+
    |  caller middleware  |  record policy, e.g. drop content
    +---------------------+
    |    TraceRecorder    |
    +---------------------+
    |    DRPCRecorder     |  acquire client, send to coderd
    +---------------------+

Ordering constraints:

- The record type constants (1) are the vocabulary both emitters and the
  validator use, so they come first.
- Validation (5) rejects unset timestamps, so records must already be
stamped
where they are created (3). Stamping at creation leaves
`WrappedRecorder`
  doing nothing but client acquisition, which is why it is folded into
  `DRPCRecorder` (4) and not before.
- coderd's server-time fallback (2) is what keeps a gateway of an older
version
  from persisting zero times once the gateway owns stamping (3).
- PRs 1 to 5 change no user-visible surface. Each deployment option then
lands
with the code that reads it: the log source with gateway-side emission
(6),
content recording with the filter that drops the records (7). Pool
options are
plumbed to the serving pool in 6, where the first option that needs them
  arrives, rather than as an unused hook beforehand.

</details>

---
🤖 Raised by [Coder Agents](https://coder.com) on behalf of @SasSwart.
2026-09-29 11:48:52 +02:00
Nick Vigilante d40e24cf7d docs: nest the CLI reference by subcommand (#29956)
## Summary

Nests the CLI reference by subcommand so the Command Line sidebar
collapses from 156 flat entries to 49 top-level verbs.
Each page's URL follows the invocation, its page title is the full
command as inline code, and its sidebar label is only its own verb as
inline code.

| | Before | After |
| --- | --- | --- |
| File | `docs/reference/cli/oauth2-provider_dcr_disable.md` |
`docs/reference/cli/oauth2-provider/dcr/disable.md` |
| URL | `/reference/cli/oauth2-provider_dcr_disable` |
`/reference/cli/oauth2-provider/dcr/disable` |
| Page title (front matter) | `oauth2-provider dcr disable` | `` `coder
oauth2-provider dcr disable` `` |
| Sidebar label (manifest) | `oauth2-provider dcr disable` | ``
`disable` `` |

Resolves DOCS-140 (coder/coder half). Also fixes the Makefile part of
DOCS-1073 / DOCS-912, which this change depends on.

## Changes

- `scripts/clidocgen`: pages nest by command path. A command with
visible subcommands becomes its directory's `index.md`, so
`oauth2-provider/index.md` and `oauth2-provider/dcr/index.md` sit above
`oauth2-provider/dcr/disable.md`. Subcommand table links are relative to
the parent page.
- `scripts/clidocgen`: the manifest "Command Line" section is built from
the command tree (nested, sorted by name, hidden commands skipped)
instead of a flat list. The redundant `coder` child that pointed back at
`reference/cli/index.md` is gone.
- `Makefile`: the CLI rule now replaces `docs/reference/cli/` wholesale,
so stale pages get removed. The manifest rule now runs `clidocgen
-manifest-only` on top of the API staging manifest, so the CLI nav is
generated instead of edited by hand. As a result, the nine pages that
had no nav entry (`ai-gateway*`, `secret enable/disable`, `server
fix-oidc-links`) now have one.
- `scripts/configdocgen` and 48 prose pages: links point at the new
paths (for example `reference/cli/server.md` is now
`reference/cli/server/index.md`).

## How to review/QA this PR

- `docs/reference/cli/**/*` (approximately 190 pages): Each of these
pages are generated automatically. Some pages appear deleted, and some
pages appear new. Every deleted page corresponds to a new page somewhere
else, so no information is being deleted. The docs in this section
should adhere to the following, so a test in the build preview for each
of these items should be satisfactory:
- [ ] Each subcommand of a command gets nested as a child instead of
being displayed as a flat list.
  - [ ] Each parent command is stored in an `index.md` file.
- [ ] The title of each page inside the page should now feature the full
`coder XYZ` command.
- [ ] The title of each page inside the page should now be formatted as
inline code.
- [ ] The title of each page in the sidebar should be the most granular
subcommand of the whole command string (if the page's title is `coder
ai-gateway keys create`, the sidebar label is `create`).
- [ ] The title of each page in the sidebar should now be formatted as
inline code.
  - [ ] The page slug for each page is now the most granular subcommand.
  - [ ] All links now point to the correct page.
- `docs/manifest.json`: The changes here reflect the updated structure
of the `docs/reference/cli` section.
- `docs/**/*.md` outside of `docs/reference/cli/**/*` (approximately 50
pages): The extent of the changes in these pages are just updating the
paths to point to the new page locations.
- [ ] The links on the page all point to the correct new path in
`docs/reference/cli`.
- `scripts/clidocgen` (4 files): This is the Go code responsible for
generating everything inside of `docs/reference/cli/**/*`. A typical
code review here should be satisfactory.
- `scripts/configdocgen` (2 files): these two files just have updates to
the raw `.md` path used by `docs/admin/setup/configuration-reference.md`
and updates the tests to reflect that.
- `Makefile`: This updates the `make` targets for generating the CLI
docs to fit the new structure.

## Merge ordering

Without the renderer PRs deployed, both docs sites show literal
backticks in titles and sidebar labels. Without the redirects, the old
flat URLs return 404.

1. Deploy before merging this PR: coder/docs-engine#176 (beta-docs
renders these titles as inline code) and coder/coder.com#1111 (classic
docs renders them as inline code). Both are safe to ship first, because
titles without backticks render as before.
2. Merge this PR.
3. Merge coder/coder.com#1112 (redirects from the old flat URLs) only
once this content reaches the mainline release branch that serves
unversioned `/docs`.

## Testing

- `go test ./scripts/clidocgen/ ./scripts/configdocgen/
./scripts/docshtmlcheck/`: pass. New tests cover page paths, subcommand
links, the nested manifest, and front matter quoting.
- `make docs/reference/cli/index.md docs/manifest.json
docs/admin/setup/configuration-reference.md`, then a forced rerun: the
second run produced no diff.
- Checked the manifest: all 164 CLI entries resolve to a file, no page
is missing a nav entry, and everything outside "Command Line" is
byte-identical.
- Relative `.md` links across `docs/`: 0 broken, same as `main`.
- `make pre-commit` and `make pre-push`: pass. On earlier pre-push
attempts, different unrelated vitest files timed out under host load.
Those files pass when run alone, and the final push passed with
`PARALLEL_JOBS=1`.

<details>
<summary>Implementation plan and decision log</summary>

**Goal (from DOCS-140 and the requester):** URLs follow the command
hierarchy. Pages are titled with the full command, and sidebar entries
with the last verb only. Both are formatted as inline code.

**Contract with the renderers:** a title wrapped entirely in one pair of
backticks (``^`[^`]+`$``) renders as inline code. Plain-text consumers
(`<title>`, meta, search, llms corpus) strip the backticks. Page title
comes from front matter `title`, and the sidebar label from the manifest
node `title`.

**Decisions**

- A parent command is `dir/index.md` rather than a sibling `dir.md`. The
root CLI page is already `index.md`, and both docs-engine and the legacy
coder.com renderer map `x/index.md` to `/x`.
- The manifest is built from the command tree, not from the written
files, so ordering is deterministic and the nesting matches the pages
exactly.
- The manifest discard bug (DOCS-1073 item 1) is fixed here, because
without that fix the generated nested manifest would never reach the
tree. DOCS-1073 items 2 through 4 (a prose fix and a hidden-command
policy) are out of scope.
- `-manifest-only` keeps the existing parallel `make gen` layout: the
API rule still owns `_gen/manifest-staging.json`, and CLI regeneration
doesn't retrigger swag.
- The renderer and redirect work lives in the companion PRs in
coder/docs-engine and coder/coder.com. It is linked above and not
duplicated here.

</details>

---

> [!NOTE]
> Generated by Coder Agents on behalf of @nickvigilante.
2026-09-28 15:44:10 -04:00