Files
coder/docs/admin/monitoring/notifications/index.md
Nick VigilanteandBen Potter 1baff57cb0 fix: resolve docs and code conflicts found in the main docs audit (#32158)
## Summary

Addresses 106 of the 113 conflicts between the docs and the code that
the November audit of `main` found: 102 are fixed and 4 have an interim
docs fix. Every finding was re-checked against the code before it was
fixed. Most fixes are docs-only. Three are small code fixes where the
docs described the intended behavior and the code was wrong, each in its
own commits so it can be split out. No change in this PR alters template
build or agent behavior.

## Changes

Fixes by area (finding counts): templates 28, install 13, integrations
and monitoring 13, auth and security 12, networking 11, Coder Agents 9,
AI governance 7, reference 7, workspaces 6.

Code fixes. Each needs a look from the code owner. If an owner prefers
not to take one, drop its commits and I will change the docs to describe
the current behavior instead.

| Commits | Change | Risk to check |
|---------|--------|---------------|
| `fix: keep built-in listener defaults in packaged coder.env`, and
three follow-up commits | `coder.env` comments out `CODER_HTTP_ADDRESS=`
and the three `CODER_TLS_*` lines, so an untouched package install
listens on `127.0.0.1:3000` as the docs say. Only the
`CODER_HTTP_ADDRESS=` line changed behavior. The TLS options default to
empty, so commenting them out is for consistency. The Lima examples
rewrite the commented line and fail if the line is not found. A test
compares every option that the file sets against the built-in defaults.
| The deb and RPM packages were not built or installed. Every released
package through v2.39.0 ships the empty line, and upgrades keep an
existing file (`config\|noreplace`), so the upgrade pages now say how to
fix it. |
| `fix(cli/clilog): accept log sinks that are all disabled`, and three
follow-up commits | `--log-human=/dev/null` with no other log location
no longer fails with "no loggers provided". It is the documented way to
disable logging. An empty location still fails with that error when no
other sink is set, as before. | Behavior change: with `/dev/null`, the
server and the provisioner daemon can start with no log output. |
| `fix(cli): detect CODER_PROC_PRIO_MGMT by presence`, and a comment
follow-up | Log line only. With the variable set to an empty value, the
agent now logs that process priority management is enabled, which is
what `agentexec` already does. No detection or behavior change. | Low. |

Two option descriptions in `codersdk/deployment.go` are corrected, with
the CLI reference, configuration reference, and golden files
regenerated: the default quiet hours schedule (the max deadline is the
quiet hours window on the next day that the template's autostop
requirement applies) and `--experiments` (`*` opts in to the experiments
marked safe, also in `enterprise/cli/provisionerdaemonstart.go`).

### Code fixes that were reverted after review

Two code fixes from the first version of this PR are reverted in the
branch. The net diff has no change under `agent/` or `coderd/`. The docs
now describe what the code does, and both issues stay open for the
owner.

- **`error` merge strategy for `coder_env` (DOCS-1669).** The fix made
`merge_strategy = "error"` fail whenever a name was set twice, whatever
the resource order. Existing template versions that pair an `error`
entry with a later `replace` or `append` entry would start to fail every
build after `terraform apply`. That is a breaking change that needs its
own PR and release note. The fix and its tests are in `dff8e7c6ce` and
can be cherry-picked. `environment-variables.md` now says the check runs
only when the `error` resource is processed.
- **Nice value before the capability drop (DOCS-1687).** The fix let
`coder agent-exec` apply a negative nice value by calling `setpriority`
before it drops effective capabilities. The current order is deliberate:
with the fix, any local process could run the agent binary with
`--coder-nice=-20` when the binary has the `CAP_SYS_NICE` file
capability. `process-priority.md` now says a nice value below the
agent's own is not applied, and no longer tells admins to grant
`CAP_SYS_NICE`.

## Not included

Seven findings are left out.

- DOCS-1629: covered by #31639.
- DOCS-1703 and DOCS-1724: covered by #31980.
- DOCS-1584, DOCS-1683, DOCS-1685, DOCS-1686: each needs an owner
decision, and no neutral docs wording exists until that decision is
made. The validation comment on each issue names the decision.

Four fixes are interim, so their issues stay open:

- DOCS-1600: the AI Gateway pages now agree that the legacy metric names
are deprecated and "will be removed in a future release". The code still
emits them, and its TODO names v2.37, which has shipped. The owner still
needs to set the removal release.
- DOCS-1689: the page now says **New Template** opens the builder only
for users with the Owner role, because the page gates on read access to
the deployment config. Whether Template Admins should get the builder is
still open.
- DOCS-1669 and DOCS-1687: docs-only for now. Refer to [Code fixes that
were reverted after
review](#code-fixes-that-were-reverted-after-review).

## Review notes

- Claims that describe product intent and need an owner's yes: the
Premium qualifier on per-organization workspace sharing (the API route
has no license check, the dashboard needs multiple organizations),
`organization` port sharing in OSS, the telemetry field list, and the
`1-5` cron range in the prebuilds schedule example.
- Not run end to end: the PostgreSQL TLS procedure on a cluster, the
OpenShift and air-gapped examples, the BSD `date` and `tar` forms on
macOS, the Windows command on the multiple Kubernetes clusters page, and
the keep-alive script in `docs/reference/index.md` (only its comparison
logic was run, against sample values). Nothing was checked on a live
deployment.
- Likely merge conflicts with open PRs: #31990 (`mcp-server.md`,
`feature-stages.md`, `docker-in-workspaces.md`), #31982
(`workspace-management.md`), #31980 (`health-check.md`), #31977
(`manage-provisioner-jobs.md`), #31869 (`airgap.md`), #30486
(`add-envbuilder.md`), and #31326 and #31531 (`codersdk/deployment.go`
and generated files).

## Testing

- `make pre-commit` (gen, fmt, lint, slim build), before the PR opened
and again after the review changes.
- `go test` for `./cli/clilog/...`, `./codersdk/`, `./cli/`
(`TestServer`, `TestAgent`, and golden files), and `./enterprise/cli/`
(provisioner, server, and help golden files). `./agent/agentexec/...`
and `./coderd/provisionerdserver/...` pass after the reverts. Linux
only. CI covers macOS and Windows.
- The `coder.env` test fails against the file on `main` and passes
against the new one. It reads the file with `godotenv`, which accepts
CRLF line endings.
- An empty `CODER_HTTP_ADDRESS=` was confirmed end to end with a full
build of the server (output below).
- The new `sed` and `grep` lines in the Lima examples were run with GNU
tools against the old and the new form of `coder.env`. The Lima VMs were
not started.
- `--log-human=/dev/null` and `CODER_LOGGING_HUMAN=` were confirmed with
the same build (output below).
- `coder agent-exec --coder-nice=-5 -- echo` from a full build, as a
non-root user: `coder-agent: failed to adjust niceness to -5 ...:
permission denied` on stderr, the command output on stdout, exit code 0.
- `postgres-ssl.md` rendered with the GitHub Markdown API: two ordered
lists with three and five items, where `main` has six lists.
- markdownlint, the table formatter, `make lint/emdash`, `make
lint/docs-html`, `make lint/docs-manifest`, `make lint/docs-redirects`,
`make lint/style-claims`, and Vale on the changed lines.
- A doc-check self-review of the docs diff by four independent
reviewers. It found six problems, which are fixed. The accuracy and
coverage lenses ran on every area. The language and structure lenses ran
in full on only part of the diff.

<details>
<summary>Test and terminal output for the code fixes (Linux)</summary>

```console
$ go test ./codersdk/ -run TestPackagedCoderEnvKeepsDefaults -count=1 -v
--- PASS: TestPackagedCoderEnvKeepsDefaults (0.00s)
ok  	github.com/coder/coder/v2/codersdk	0.071s

$ go test ./cli/clilog/ -run TestBuilder -count=1 -v
--- PASS: TestBuilder (0.00s)
    --- PASS: TestBuilder/NoSinks (0.00s)
        --- PASS: TestBuilder/NoSinks/JSONDevNull (0.00s)
        --- PASS: TestBuilder/NoSinks/StackdriverDevNull (0.00s)
        --- PASS: TestBuilder/NoSinks/NoConfiguration (0.00s)
        --- PASS: TestBuilder/NoSinks/HumanDevNull (0.00s)
    --- PASS: TestBuilder/WithHuman (0.00s)
    --- PASS: TestBuilder/WithFilter (0.00s)
    --- PASS: TestBuilder/WithJSON (0.00s)
    --- PASS: TestBuilder/Verbose (0.00s)
    --- PASS: TestBuilder/FromDeploymentValues (0.01s)
        --- PASS: TestBuilder/FromDeploymentValues/Defaults (0.00s)
        --- PASS: TestBuilder/FromDeploymentValues/EmptyHumanEnv (0.00s)
        --- PASS: TestBuilder/FromDeploymentValues/Override (0.00s)
ok  	github.com/coder/coder/v2/cli/clilog	0.171s

$ TMPDIR=$HOME/.cache/agentexec-tmp go test ./agent/agentexec/... ./coderd/provisionerdserver/... -count=1
ok  	github.com/coder/coder/v2/agent/agentexec	4.236s
ok  	github.com/coder/coder/v2/coderd/provisionerdserver	2.404s

# Full build of the server (go build ./cmd/coder). An empty value stops it before it connects to the database.
$ CODER_HTTP_ADDRESS= coder server --postgres-url 'postgres://127.0.0.1:1/x?sslmode=disable'
Encountered an error running "coder server", see "coder server --help" for more information
error: TLS is disabled. Enable with --tls-enable or specify a HTTP address

# With the variable unset, the server passes that check and goes on to the database.
$ coder server --postgres-url 'postgres://127.0.0.1:1/x?sslmode=disable'
2026-11-13 02:19:41.901 [warn]  ping postgres: retrying  error="dial tcp 127.0.0.1:1: connect: connection refused"  try=11
Encountered an error running "coder server", see "coder server --help" for more information
error: connect to postgres: connect to postgres: unable to connect after 11 tries; last error: dial tcp 127.0.0.1:1: connect: connection refused

# An empty log location with no other sink still stops the server.
$ CODER_LOGGING_HUMAN= coder server --postgres-url 'postgres://127.0.0.1:1/x?sslmode=disable'
Encountered an error running "coder server", see "coder server --help" for more information
error: make logger: no loggers provided, use /dev/null to disable logging

# /dev/null passes the logger check. The server prints no log lines and goes on to the database.
$ coder server --log-human=/dev/null --postgres-url 'postgres://127.0.0.1:1/x?sslmode=disable'
Encountered an error running "coder server", see "coder server --help" for more information
error: connect to postgres: connect to postgres: unable to connect after 11 tries; last error: dial tcp 127.0.0.1:1: connect: connection refused
```

</details>

## Linear

Fixes DOCS-1583, DOCS-1585, DOCS-1586, DOCS-1587, DOCS-1588, DOCS-1589,
DOCS-1590, DOCS-1591, DOCS-1592, DOCS-1593, DOCS-1594, DOCS-1595,
DOCS-1596, DOCS-1597, DOCS-1598, DOCS-1599, DOCS-1601, DOCS-1602,
DOCS-1603, DOCS-1604, DOCS-1605, DOCS-1606, DOCS-1607, DOCS-1608,
DOCS-1609, DOCS-1614, DOCS-1618, DOCS-1619, DOCS-1620, DOCS-1624,
DOCS-1635, DOCS-1639, DOCS-1644, DOCS-1651, DOCS-1652, DOCS-1653,
DOCS-1654, DOCS-1655, DOCS-1656, DOCS-1657, DOCS-1658, DOCS-1659,
DOCS-1660, DOCS-1661, DOCS-1662, DOCS-1663, DOCS-1664, DOCS-1665,
DOCS-1666, DOCS-1667, DOCS-1668, DOCS-1670, DOCS-1671, DOCS-1672,
DOCS-1673, DOCS-1674, DOCS-1675, DOCS-1676, DOCS-1677, DOCS-1678,
DOCS-1679, DOCS-1680, DOCS-1681, DOCS-1682, DOCS-1684, DOCS-1688,
DOCS-1690, DOCS-1691, DOCS-1692, DOCS-1693, DOCS-1694, DOCS-1695,
DOCS-1696, DOCS-1697, DOCS-1698, DOCS-1699, DOCS-1700, DOCS-1701,
DOCS-1702, DOCS-1704, DOCS-1705, DOCS-1706, DOCS-1707, DOCS-1708,
DOCS-1709, DOCS-1710, DOCS-1711, DOCS-1712, DOCS-1713, DOCS-1714,
DOCS-1715, DOCS-1716, DOCS-1717, DOCS-1718, DOCS-1719, DOCS-1720,
DOCS-1721, DOCS-1722, DOCS-1723, DOCS-1725, DOCS-1726, DOCS-1727

Part of DOCS-1600, DOCS-1689, DOCS-1669, DOCS-1687

<details>
<summary>Decision log</summary>

### Scope

- One PR for all findings, as agreed with the reviewer. The code fixes
are separate commits so an owner can ask for any of them to move to its
own PR. Nothing in the PR is a breaking change, so the title has no `!`.
- A finding was skipped only when an open PR already covers it, or when
every available wording would assert an undecided product behavior.
- Each validation comment on the Linear issues records the evidence, the
corrections to the original finding, and what was not verified.

### Docs or code

- The default was a docs-only fix that describes what the code does.
- A code fix was chosen when the docs describe a behavior the code
plainly intended and the code change is small and testable: the packaged
env file, the log sink check, and the log line for the environment
variable presence check.
- The `error` merge strategy and the nice value ordering started as code
fixes and went back to docs-only after review. The first breaks existing
template versions. The second weakens a deliberate security ordering.
- The SCIM `userName` finding (DOCS-1609) took the docs path. The
handler is unchanged.

### Wording choices worth a second look

- `feature-stages.md` says `--experiments=*` enables only the
experiments marked safe, and points to the "Get safe experiments" API.
It does not list the other experiments, because the list would drift and
those experiments are hidden on purpose. The `--experiments` flag help
now says the same.
- `agent-metadata.md` replaces the writes-per-second formula with a
description of the batcher (each server replica writes one statement
every 5 seconds, or sooner at 500 entries).
- `extending-templates/index.md` no longer calls `startup_script`
deprecated. The provider documents it as an alias for a `coder_script`.
- `postgres-ssl.md` separates client verification from server
verification, so the server verification steps work alone.
- `rpm.md`, `upgrade.md`, and the 2.29 to 2.34 ESR upgrade page explain
the empty `CODER_HTTP_ADDRESS=` line that older packages shipped,
because upgrades keep the old file. The behavior changed in v2.34.0 with
the serpent upgrade in #24926.
- The keep-alive script in `docs/reference/index.md` reads the current
deadline and skips the request when it is already later. The comparison
is on text, so it applies only when the API returns a UTC timestamp,
which needs the database to use the UTC time zone. Otherwise the script
sets the deadline on every pass, as it did before, and the page says so.
- `external-auth.md` anchors the new GitHub Enterprise regex example.
The older unanchored examples on that page and on `git-providers.md` are
not changed here.

### Self-review findings that were rejected

- "The builder opens for Template Admins": the Templates page needs a
successful deployment config read, which the Template Admin role does
not have.
- "Organization Admins can set AI model prices": the price write
authorizes against a site-level object, which organization-scoped grants
do not match.
- "The provisioner chart sets no default resources":
`helm/libcoder/templates/_coder.yaml` applies 2000m and 4096Mi when
`coder.resources` is empty.

### Automated review, round 1

The `coder-agents-review` app left 45 findings on `8853332`. Fixed: the
two reverts above, the anchored regex, the `coder.env` follow-ups (Lima
examples, upgrade notes, a comment in the file, and a test that covers
every option in the file), the clilog change, the two option
descriptions, and the docs findings (CORS behavior, token lifetime
example, keep-alive script, agent metadata, refresh token errors,
`switch` form type, `dbcrypt` skip list, and a stray line in a YAML
example).

Declined, with the reason on each thread:

- Refactoring the `agentexec` logging and renaming its log field: not
related to a docs conflict.
- Rewording the "process priority management not enabled (linux-only)"
log message: the docs quote the message, and the wording predates this
PR.
- Problems in code and docs that this PR does not change (the
`Capabilities` subtest, the Helm values comment, the AWS provider path
in `airgap.md`, and others in the review's out-of-scope list).

### Automated review, round 2

The app left 14 new findings on `04925bd`, and accepted the three
declined threads on `cli/agent.go`. All 14 are fixed. The main one: the
round 1 change that removed the "no loggers provided" error went too
far, because an empty log location would then start a server with no
logs. The error is back for every case except `/dev/null`.

Reported by the app as out of scope and not changed: the swagger
annotation for `GET /api/v2/experiments/available` declares an array,
but the handler returns an object with a `safe` array. Three Storybook
fixtures keep the old `--experiments` description.

### Automated review, round 3

The app confirmed that all round 2 findings are fixed, and left 10 small
findings on `f55cb6d` (8 P4, 2 Nit). All are fixed: the TLS condition
and the release range in the `coder.env` notes, one place for the
`/dev/null` rule in `clilog`, two test cases, a clearer test failure,
and the CPU example in `agent-metadata.md`.

### Automated review, rounds 4 and 5

Round 4 left one P4 on `8fffb38` (name the `--trace` option in the
logging note), which is fixed. Round 5 on `2ce754e` reports no new
findings and says every earlier finding is fixed or closed. The app did
not post an approval: it leaves its out-of-scope items and the code
fixes to a human reviewer. Two threads on `cli/agent.go` stay open on
purpose for the code owner.

### Doc-check review

The doc-check workflow left 12 comments on `2ce754e` (10 blocking, 2
suggestions). All are fixed in `c888885`: "refer to" or "visit" in place
of "see" for cross-references on the lines this PR added, the **New
template** label as the dashboard shows it, one instruction per step in
the server verification procedure of `postgres-ssl.md`, and two wording
changes in `process-priority.md`. The `CAP_SYS_NICE` sentence keeps a
short reason, which differs from the suggested text.

The `coder-agents-review` app then reviewed that commit (rounds 6 and
7). Round 6 left two P4 findings, fixed in `30b2c79`: the `failed to
adjust niceness` message is described once, in the troubleshooting
entry, with where it appears, and the client verification steps in
`postgres-ssl.md` render as one numbered list. Round 7 reports no new
findings. `2a6035c` then softens one sentence from that fix, because the
agent logs the stderr of devcontainer CLI commands. The app was not run
again on that commit.

### Adjacent gaps found and not fixed here

- `add-envbuilder.md` has the same `zip` command that
`template-from-scratch.md` had.
- `template-from-scratch.md` leaves the reader in `build/` after `cd
$_`, before a step that runs from `template-tour`.

</details>

---

_Generated by Coder Agents on behalf of @nickvigilante._

---------

Co-authored-by: Ben Potter <me@bpmct.net>
2026-11-13 21:34:59 +00:00

18 KiB
Raw Permalink Blame History

title
title
Notifications

Notifications are sent by Coder in response to specific internal events, such as a workspace being deleted or a user being created.

Available events might differ between versions. For a list of all events, visit your Coder deployment's https://coder.example.com/deployment/notifications.

Event Types

Notifications are sent in response to internal events, to alert the affected user(s) of the event.

Coder supports the following list of events:

Template Events

These notifications are sent to users with template admin roles:

  • Report: Workspace builds failed for template
    • This notification is delivered as part of a weekly cron job and summarizes the failed builds for a given template.
  • Template deleted
  • Template deprecated

User Events

These notifications are sent to users with owner and user admin roles:

  • User account activated
  • User account created
  • User account deleted
  • User account suspended

These account notifications cover both user and service accounts.

These notifications are sent to users themselves:

  • User account suspended
  • User account activated
  • User password reset (One-time passcode)

Workspace Events

These notifications are sent to the workspace owner:

  • Workspace automatic build failure
  • Workspace autostop reminder
  • Workspace created
  • Workspace deleted
  • Workspace manual build failure
  • Workspace manually updated
  • Workspace marked as dormant
  • Workspace marked for deletion
  • Out of memory (OOM) / Out of disk (OOD)
  • Workspace automatically updated

Delivery Methods

Notifications can be delivered through the Coder dashboard Inbox and by Simple Mail Transfer Protocol (SMTP) or webhook. OOM/OOD notifications can be delivered to users in VS Code.

You can configure:

Premium customers can configure which method to use for each of the supported Events. See the Preferences section for more details.

Configuration

You can modify the notification delivery behavior in your Coder deployment's https://coder.example.com/deployment/notifications?tab=settings, or with the following server flags:

Required CLI Env Type Description Default
✔️ --notifications-dispatch-timeout CODER_NOTIFICATIONS_DISPATCH_TIMEOUT duration How long to wait while a notification is being sent before giving up. 1m
✔️ --notifications-method CODER_NOTIFICATIONS_METHOD string Which delivery method to use (available options: 'smtp', 'webhook'). See Delivery Methods below. smtp
-️ --notifications-max-send-attempts CODER_NOTIFICATIONS_MAX_SEND_ATTEMPTS int The upper limit of attempts to send a notification. 5
-️ --notifications-inbox-enabled CODER_NOTIFICATIONS_INBOX_ENABLED bool Enable or disable inbox notifications in the Coder dashboard. true

Configure OOM/OOD notifications

You can monitor out of memory (OOM) and out of disk (OOD) errors and alert users when they overutilize memory and disk.

This can help prevent agent disconnects due to OOM/OOD issues.

To enable OOM/OOD notifications on a template, follow the steps in the resource monitoring guide.

SMTP (Email)

Use the smtp method to deliver notifications by email to your users. Coder does not ship with an SMTP server, so you will need to configure Coder to use an existing one.

Server Settings:

Required CLI Env Type Description Default
✔️ --email-from CODER_EMAIL_FROM string The sender's address to use (e.g. "Coder <coder@example.com>").
✔️ --email-smarthost CODER_EMAIL_SMARTHOST string The SMTP relay to send messages (format: hostname:port)
✔️ --email-hello CODER_EMAIL_HELLO string The hostname identifying the SMTP server. localhost

Authentication Settings:

Required CLI Env Type Description
- --email-auth-username CODER_EMAIL_AUTH_USERNAME string Username to use with PLAIN/LOGIN authentication.
- --email-auth-password CODER_EMAIL_AUTH_PASSWORD string Password to use with PLAIN/LOGIN authentication.
- --email-auth-password-file CODER_EMAIL_AUTH_PASSWORD_FILE string File from which to load password for use with PLAIN/LOGIN authentication.
- --email-auth-identity CODER_EMAIL_AUTH_IDENTITY string Identity to use with PLAIN authentication.

TLS Settings:

Required CLI Env Type Description Default
- --email-force-tls CODER_EMAIL_FORCE_TLS bool Force a TLS connection to the configured SMTP smarthost. If port 465 is used, TLS will be forced. See https://datatracker.ietf.org/doc/html/rfc8314#section-3.3. false
- --email-tls-starttls CODER_EMAIL_TLS_STARTTLS bool Enable STARTTLS to upgrade insecure SMTP connections using TLS. Ignored if CODER_EMAIL_FORCE_TLS is set. false
- --email-tls-skip-verify CODER_EMAIL_TLS_SKIPVERIFY bool Skip verification of the target server's certificate (insecure). false
- --email-tls-server-name CODER_EMAIL_TLS_SERVERNAME string Server name to verify against the target certificate.
- --email-tls-cert-file CODER_EMAIL_TLS_CERTFILE string Certificate file to use.
- --email-tls-cert-key-file CODER_EMAIL_TLS_CERTKEYFILE string Certificate key file to use.

NOTE: you MUST use CODER_EMAIL_FORCE_TLS if your smarthost supports TLS on a port other than 465.

Send emails using G-Suite

After setting the required fields above:

  1. Create an App Password using the account you wish to send from.

  2. Set the following configuration options:

    CODER_EMAIL_SMARTHOST=smtp.gmail.com:465
    CODER_EMAIL_AUTH_USERNAME=<user>@<domain>
    CODER_EMAIL_AUTH_PASSWORD="<app password created above (no spaces)>"
    

    Note: The CODER_EMAIL_AUTH_PASSWORD must be entered without spaces.

See this help article from Google for more options.

Send emails using Outlook.com

After setting the required fields above:

  1. Set up an account on Microsoft 365 or outlook.com

  2. Set the following configuration options:

    CODER_EMAIL_SMARTHOST=smtp-mail.outlook.com:587
    CODER_EMAIL_TLS_STARTTLS=true
    CODER_EMAIL_AUTH_USERNAME=<user>@<domain>
    CODER_EMAIL_AUTH_PASSWORD="<account password>"
    

See this help article from Microsoft for more options.

Webhook

The webhook delivery method sends an HTTP POST request to the defined endpoint. The purpose of webhook notifications is to enable integrations with other systems.

Settings:

Required CLI Env Type Description
✔️ --notifications-webhook-endpoint CODER_NOTIFICATIONS_WEBHOOK_ENDPOINT url The endpoint to which to send webhooks.

Here is an example payload for Coder's webhook notification:

{
    "_version": "1.1",
    "msg_id": "88750cad-77d4-4663-8bc0-f46855f5019b",
    "payload": {
        "_version": "1.2",
        "notification_name": "Workspace Deleted",
        "notification_template_id": "f517da0b-cdc9-410f-ab89-a86107c420ed",
        "user_id": "4ac34fcb-8155-44d5-8301-e3cd46e88b35",
        "user_email": "danny@coder.com",
        "user_name": "danny",
        "user_username": "danny",
        "actions": [
            {
                "label": "View workspaces",
                "url": "https://et23ntkhpueak.pit-1.try.coder.app/workspaces"
            },
            {
                "label": "View templates",
                "url": "https://et23ntkhpueak.pit-1.try.coder.app/templates"
            }
        ],
        "labels": {
            "initiator": "danny",
            "name": "my-workspace",
            "reason": "initiated by user"
        },
        "data": null,
        "targets": null
    },
    "title": "Workspace \"my-workspace\" deleted",
    "title_markdown": "Workspace \"my-workspace\" deleted",
    "body": "Hi danny\n\nYour workspace my-workspace was deleted.\nThe specified reason was \"initiated by user (danny)\".",
    "body_markdown": "Hi danny\n\nYour workspace **my-workspace** was deleted.\nThe specified reason was \"initiated by user (danny)\"."
}

The top-level object has these keys:

  • _version: describes the version of this schema and follows semantic versioning
  • msg_id: the universally unique identifier (UUID) of the notification (matches the ID in the notification_messages table)
  • payload: contains the specific details of the notification (described below)
  • title: the title of the notification message (equivalent to a subject in SMTP delivery)
  • title_markdown: the title of the notification message with Markdown formatting
  • body: the body of the notification message (equivalent to the message body in SMTP delivery)
  • body_markdown: the body of the notification message with Markdown formatting

The payload object has these keys:

  • _version: describes the version of this inner schema and follows semantic versioning
  • notification_name: name of the event which triggered the notification
  • notification_template_id: ID of the notification template that the event used (UUID)
  • user_id: Coder internal user identifier of the target user (UUID)
  • user_email: email address of the target user
  • user_name: name of the target user
  • user_username: username of the target user
  • actions: a list of calls to action (CTAs). These are mainly relevant for SMTP delivery in which they're shown as buttons
  • labels: dynamic map of zero or more string key-value pairs. These vary from event to event
  • data: structured data that the event passed with the notification, or null when the event passed none
  • targets: list of UUIDs that the event passed with the notification, or null when the event passed none

User Preferences

All users have the option to opt-out of any notifications. Go to Account -> Notifications to turn notifications on or off. The delivery method for each notification is indicated on the right hand side of this table.

User Notification Preferences

Delivery Preferences

Note

Delivery preferences is a Premium feature. Learn more.

Administrators can configure which delivery methods are used for each different event type.

preferences

You can find this page under https://$CODER_ACCESS_URL/deployment/notifications?tab=events.

Custom notifications

Custom notifications let you send an ad‑hoc notification to yourself using the Coder CLI. These are useful for surfacing the result of long-running tasks or important state changes. At this time, custom notifications can only be sent to the user making the request.

To send a custom notification, execute coder notifications custom <title> <message>.

Note: The recipient is always the requesting user as targeting other users or groups isn’t supported yet.

Examples

  • Send yourself a quick update:
coder templates push -y && coder notifications custom "Template push complete" "Template version uploaded."
  • Use in a script after a long-running task:
#!/usr/bin/env bash
set -o pipefail

if make test 2>&1 | tee test_output.log; then
  coder notifications custom "Tests Succeeded" $'Test results:\n  • ✅ success'
else
  failures=$(grep -Po '\d+(?=\s+failures)' test_output.log | tail -n1 || echo 0)
  coder notifications custom "Tests Failed" $'Test results:\n  • ❌ failed ('"$failures"' tests failed)'
  exit 1
fi

Stop sending notifications

Administrators might wish to stop all notifications across the deployment. We support a killswitch in the CLI for these cases.

To pause sending notifications, execute coder notifications pause.

To resume sending notifications, execute coder notifications resume.

Troubleshooting

If notifications are not being delivered, use the following methods to troubleshoot:

  1. Make sure notifications are being added to the notification_messages table.

  2. Review any available error messages in the status_reason column

  3. Review the logs. Search for the term notifications for diagnostic information.

    • If you do not see any relevant logs, set CODER_LOG_FILTER=".*notifications.*" to filter for notification-related logs.
  4. If you are on version 2.15.x, notifications must be enabled using the notifications experiment.

    Notifications are enabled by default in Coder v2.16.0 and later.

Internals

The notification system is built to operate concurrently in a single- or multi-replica Coder deployment, and has a built-in retry mechanism. It uses the configured Postgres database to store notifications in a queue and facilitate concurrency.

All messages are stored in the notification_messages table.

Messages older than seven days are deleted.

Message States

states

A notifier here refers to a Coder replica which is responsible for dispatching the notification. All running replicas act as notifiers to process pending messages.

  • a message begins in pending state
  • transitions to leased when a Coder replica acquires new messages from the database
    • new messages are checked for every CODER_NOTIFICATIONS_FETCH_INTERVAL (default: 15s)
  • if a message is delivered successfully, it transitions to sent state
  • if a message encounters a non-retryable error (for example, misconfiguration), it transitions to permanent_failure
  • if a message encounters a retryable error (for example, temporary server outage), it transitions to temporary_failure
    • this message will be retried up to CODER_NOTIFICATIONS_MAX_SEND_ATTEMPTS (default: 5)
    • this message will transition back to pending state after CODER_NOTIFICATIONS_RETRY_INTERVAL (default: 5m) and be retried
    • after CODER_NOTIFICATIONS_MAX_SEND_ATTEMPTS is exceeded, it transitions to permanent_failure

See Troubleshooting above for more details.