Files
coder/scripts/docsredirectscheck
Nick Vigilante c95b908705 ci: make docs redirects relative and block removed routes (#30805)
## Summary

Moves `docs/redirects.json` to relative routes and makes
`lint/docs-redirects` fail when a PR removes a live docs route without a
redirect.
coder.com and docs-engine build each version's redirects from that
branch's file, so this file becomes the only place contributors add docs
redirects.

Part of DOCS-1290; tracked in DOCS-1296.

## Changes

- New rule format: `{"source": "path/to/page", "destination":
"path/to/another-page"}`, relative to the docs root of the branch's
version.
Trailing `/*` is the only wildcard, `/` is the version's docs root, one
`#fragment` is allowed on exact destinations, and external destinations
are limited to `https://github.com/coder/` and `https://coder.com/`.
The old `/docs/...` form is rejected with the converted rule in the
message.
- Removed-route check now blocks.
Every route in the base manifest, the live page list for the target
branch (`coder.com/beta-docs/<version>/llms.txt`), and the base file's
sources must be a page or resolve through a rule.
The failure names the old file and prints a paste-ready rule, suggesting
a renamed sibling, a page with the same last segment, or `/`.
  An unreadable live list exits 2 in CI; `-offline` skips it locally.
Links under `/docs` count as well, so the check keeps working if the
list URL redirects there after the move to `/docs`.
- `docs/redirects.json` on `main`: 304 rules.
It carries the coder.com docs rules that 404 under `/docs/@main` today,
plus the 144 `@main`-only rules coder.com holds, with three sibling sets
condensed into wildcards (`networking/*`, `admin/notifications/*`,
`guides/*`).
- README, content guidelines, and agent guidance describe the new
format.

## Testing

- `go test ./scripts/docsredirectscheck/...` (also `-race`),
`golangci-lint`, and the pre-commit hook.
- `go run ./scripts/docsredirectscheck -base origin/main -branch main`
online, after merging current `main`: 304 rules OK, 497 routes resolve.
- Renaming `admin/integrations/istio.md` without a rule fails with
`{"source": "admin/integrations/istio", "destination":
"admin/integrations/istio-mesh"}`; adding that rule passes.
- coder.com's `generateRedirects` with this file and `main`'s manifest:
0 dropped; 336 of 336 sampled old URLs resolve to the expected page with
Next's matcher.
- `curl` on every destination under `coder.com/docs/@main`: all 200.
- Live baseline before merge: 185 of 336 sampled `/docs/@main` URLs
return 404.
- The check runs on Linux in `ci.yaml`'s `lint` job; the package's unit
tests also run on macOS and Windows in `test-go-pg`.

## Merge order

Merge after the coder.com generator (coder/coder.com,
`vigilante/docs-redirects-relative`) is deployed, otherwise coder.com
drops these rules with warnings until it is.
The docs-engine change (coder/docs-engine,
`vigilante/version-redirects`) can land before or after.
Then merge the `release/2.38` backport.

<details>
<summary>Contract shared by coder/coder, coder.com, and
docs-engine</summary>

# Docs redirects contract (coder/coder owns the data; coder.com and
docs-engine consume it)

## Ownership

- Every redirect for a moved, renamed, or removed docs page lives in
`coder/coder` `docs/redirects.json`, one file per branch (`main`,
`release/X.Y`).
- A branch's file applies only to that branch's version of the docs.
`main`'s file applies to `@main`; `release/X.Y`'s file applies to vX.Y,
and to unversioned `/docs` while X.Y is mainline.
- URL-scheme rules (`/docs/v1`, `/docs/v2/...`, `/docs/coder-oss/...`,
`/docs/coder/...`, `/docs/code-server/...`, classic `/@v<line>` forms)
are not content moves. They stay in the consumer that owns the URL
scheme, as code or a static engine-owned file.
- Nobody adds content redirects in coder.com or docs-engine. Existing
ones there are removed only after the matching coder/coder rules are
verified live for that version (later phase, not this change).

## File format

`docs/redirects.json` is a JSON array. Each entry is an object with
exactly these fields:

| Field | Required | Meaning |

|---------------|----------|-------------------------------------------------------------------------|
| `source` | yes | A route relative to the docs root of this version |
| `destination` | yes | A route relative to the docs root, `/` for the
docs root, or an allowed `https://` URL |
| `permanent` | no | Boolean, default `true`. `true` answers 308,
`false` answers 307 |

Unknown fields are an error.

### Routes

- A route is the URL path below `/docs` with no version, no leading
slash, no trailing slash, and no `.md`:
`admin/templates/extending-templates`.
- A route is one or more segments joined by `/`. Each segment matches
`[A-Za-z0-9._-]+`. Nothing else is allowed (no `@`, `%`, `:`, `(`, `?`,
`#`, spaces, empty segments, `.` or `..` segments).
- Routes are derived from `docs/manifest.json` the way the sites do
today: strip a leading `./`, then strip `README.md`, `index.md`, `.md`
in that order as plain text, trim slashes. The empty result is the docs
root.

### Wildcards

- A `source` may end in `/*`: `ai-coder/ai-bridge/*`. It matches the
prefix itself and every route beneath it.
- A `destination` may end in `/*` only when the source does. The part
the source's `*` matched (possibly empty) replaces the destination's
`*`; an empty match means the destination prefix itself
(`ai-coder/ai-gateway/*` with an empty match is `ai-coder/ai-gateway`).
- A wildcard source with a non-wildcard destination sends everything
under the prefix to that one page.
- No other pattern syntax exists. Consumers translate `*` into their own
matcher. For Next.js that is `:path([A-Za-z0-9._-]+)*`, never `(.*)`,
`.+`, or a bare `:name*`.

### Destinations

- `/` is the docs root of the same version (`/docs`, `/docs/@main`,
`/beta-docs/v2.37`, and so on). Pages with no suitable home redirect to
`/`.
- An internal destination always stays in the same version as the
request.
- A non-wildcard internal destination may end in one `#fragment`, where
the fragment matches `[A-Za-z0-9._-]+`:
`user-guides/workspace-access#web-ides`. The route part must still be a
live route. A wildcard destination and `/` never carry a fragment.
- An external destination is an `https://` URL whose host and path start
with an allowed prefix: `https://github.com/coder/` or
`https://coder.com/`. It contains no `:` after the scheme and no
parameter. A `*` is not allowed in an external destination.

### Order and precedence

- Rules apply in file order; the first match wins.
- A page always wins over a redirect: a rule only fires for a request
that is not a live page of that version. The checker rejects any rule
whose source matches a live route, so consumers that check redirects
before pages (Next.js static redirects) behave the same as those that
check on a miss.
- Matching ignores one trailing slash on the request.
- Consumers follow at most 3 chained hops and refuse loops. The checker
rejects chains so a valid file never needs this.

### Legacy entries

- During migration, consumers also accept a `source` or `destination`
that starts with `/docs/` (or equals `/docs`) and treat it as the route
after `/docs/` (or `/` for `/docs`). A legacy `:name*` or `:name(.*)`
trailing parameter is treated as `/*`. The coder/coder checker accepts
only the new form.
- A missing file and an empty array both mean no rules. A file that is
not a JSON array is an error.

## CI check (coder/coder `scripts/docsredirectscheck`)

Structural checks as today, adapted to the new form, plus a blocking
removed-route check:

1. **Inventory.** The set of routes that must keep working is the union
of:
   - the base branch's `docs/manifest.json` routes,
- every route in the live production page list for the PR's target
branch, read from `https://coder.com/beta-docs/<segment>/llms.txt` where
the segment is `@main` for `main` and `vX.Y` for `release/X.Y`, keeping
only links under that segment,
- every `source` in the base branch's `docs/redirects.json` (exact
sources as-is; wildcard sources contribute the prefix).
2. **Resolution.** Each inventory route must be a live route in the head
manifest, or be resolved by the head rules (with wildcard substitution)
to a live head route, `/`, or an allowed external URL.
3. **Failure message.** Name the route, the file it came from on the
base branch when known, and a ready-to-paste rule, for example
`{"source": "path/to/page", "destination": "path/to/another-page"}`,
pointing at `/` when no candidate is obvious.
4. **Live list failures.** A network error, non-200, or a list with no
routes is an infrastructure failure (exit 2) in CI, not a pass. Locally,
`-offline` skips the live list and says so.
5. **Known engine difference.** beta-docs currently serves dotted
segments with hyphens (`esr-2.24-2.29-upgrade` is served as
`esr-2-24-2-29-upgrade`). Until docs-engine fixes it, a live route that
equals a head route with `.` replaced by `-` in each segment counts as
present.

## Consumers

- **coder.com classic** (`scripts/generate-docs-redirects.ts`): reads
`main` (scoped to `/docs/@main/...`) and the mainline release branch
(unversioned `/docs/...`) as today, in the new format, and emits Next.js
redirects. Version coverage beyond those two is out of scope.
- **docs-engine beta**: each version's publish reads
`docs/redirects.json` from the commit it builds, validates it, and
stores the compiled rules with that version's archive. The router
applies a version's own rules on a miss inside that version. Versions
published before this change keep today's behavior.

</details>

---

🤖 Built with AI assistance.
2026-11-05 14:04:17 -05:00
..

docsredirectscheck

docsredirectscheck validates docs/redirects.json, the list of redirects the documentation websites apply for pages that have moved or been removed. It runs as make lint/docs-redirects (part of make lint and make lint-light), so a bad or missing redirect is caught in the pull request that introduces it.

Each branch has its own docs/redirects.json. The file on main applies to @main, and the file on release/X.Y applies to vX.Y and, while X.Y is the mainline release, to the unversioned docs. A missing docs/redirects.json passes.

The rule file

docs/redirects.json is a JSON array of rules:

[
  {
    "source": "about/architecture",
    "destination": "/"
  },
  {
    "source": "ai-coder/ai-gateway/*",
    "destination": "ai-governance/ai-gateway/*",
    "permanent": false
  }
]
  • source and destination are routes relative to the docs root. A route is the URL path below /docs with no version, no leading or trailing slash, and no .md, such as templates/extending-templates. Each segment matches [A-Za-z0-9._-]+.
  • permanent is optional and defaults to true. true answers 308 and false answers 307.
  • A source is an exact route, or a route ending in /*. A wildcard matches the prefix itself and every route beneath it.
  • A destination is an exact route, / for the docs root, or an https:// URL that starts with https://github.com/coder/ or https://coder.com/. An external destination has no : after the scheme, no *, no \, and no . or .. path segment, also when percent-encoded.
  • An exact route destination may end in one #fragment that matches [A-Za-z0-9._-]+, such as workspaces/workspace-access#other-web-ides. The route part must be a live route. / and wildcard destinations never carry a fragment.
  • A destination ends in /* only when the source does. The part the source's * matched replaces the destination's *. An empty match means the destination prefix itself, so ai-governance/ai-gateway/* with an empty match is ai-governance/ai-gateway. A wildcard source with an exact destination sends everything under the prefix to that one page.
  • Rules apply in file order and the first match wins, so they must run from most specific to least. Put an exact rule above any wildcard that would also match it.
  • Unknown fields are an error.

The old absolute form, such as /docs/admin/old or /docs/x/:path*, is rejected. The error shows the converted rule.

What it catches

These fail the check:

  • A malformed file. Not valid JSON, not an array, an entry that is not an object, a missing or empty source or destination, a non-boolean permanent, or an unknown field.
  • Unsupported syntax. A route outside the grammar above, the old /docs/... form, or an external destination outside the allowlist.
  • A destination that is not a page. An internal destination must be a route in docs/manifest.json. A wildcard destination must point into a section that has pages.
  • A source that hides a live page. A page always wins over a redirect, and some sites apply redirects first. A rule whose source matches a page in the manifest is therefore rejected.
  • Duplicates, chains, and loops. Two rules with the same source, a rule whose destination is another rule's source (point it at the final page instead), and cycles.
  • A rule that can never apply. An earlier rule whose source matches every route a later rule would. A wildcard above a rule inside its subtree is the usual cause. An exact rule above a wildcard on the same prefix is allowed, because it claims the bare prefix and leaves the subtree to the wildcard.
  • A removed route. See the next section.

Removed routes

A route that worked before the change must still work after it. Each route in the inventory must be a live route in the head manifest, or be resolved by the head rules to a live route, to /, or to an allowed external URL. The inventory is the union of:

  • The routes in docs/manifest.json on the base branch.
  • The routes in the live production page list for the target branch. The list is https://coder.com/beta-docs/@main/llms.txt for main and https://coder.com/beta-docs/vX.Y/llms.txt for release/X.Y. Only links under that version count. The version served at the unversioned /beta-docs path links its pages without a version segment, so when a list has no links under its own segment, the unversioned links count instead. When the list redirects to /docs, links under /docs count the same way.
  • The source of every rule in docs/redirects.json on the base branch. A wildcard source contributes its prefix. A base file in the old absolute form is read as well.

A failure names the route, where it came from, and a rule to paste, for example {"source": "path/to/page", "destination": "/"}. The destination is the one live page with the same last segment. Otherwise it is the one page this change added in the same parent section, which catches a rename such as admin/integrations/istio to admin/integrations/istio-mesh. Otherwise it is /. Change the destination when a better page exists. In GitHub Actions each failure is also an error annotation.

Both ends of the check can be missing, and the check says so and skips that part:

  • Base branch. In GitHub Actions on a pull request the base is the first parent of the merge commit, which is the tip of the branch the change merges into. Elsewhere pass -base <rev>. A shallow checkout that lacks the revision skips the comparison with a note.
  • Live page list. The target branch is -branch, then GITHUB_BASE_REF on a pull request, then GITHUB_REF_NAME on a push. A branch other than main or release/X.Y, or no known branch, skips the list with a note. A list that answers 404 skips the list with a note, because a new release/X.Y branch has no published docs version yet. When docs/manifest.json and docs/redirects.json are the same as on the base branch, the list is skipped with a note, because the change cannot affect the result. On a pull request, a coder.com outage only blocks changes to those two files. Pass -offline to skip the list on purpose, for example without network access.

A live page list that cannot be read for any other reason is an infrastructure failure and exits 2, not a pass. The request has a descriptive User-Agent, because the site answers 403 to some default clients, and it retries a few times before it gives up. An empty list is also a failure.

A base docs/redirects.json that does not exist means the base branch had no rules. Any other failure to read it skips the base redirect sources with a note.

Dotted segments

beta-docs currently serves a dotted segment with hyphens, so esr-2.24-2.29-upgrade is served as esr-2-24-2-29-upgrade. Until the docs engine fixes it, a live route that equals a head route with . replaced by - counts as present.

How it decides which pages exist

It derives routes from docs/manifest.json as the docs websites do: strip a leading ./, then strip the suffixes README.md, index.md, and .md one after another, and trim slashes. That includes the websites' quirk of matching the suffixes as plain text rather than per path segment, so a file named my-index.md becomes the route .../my-. Every manifest entry with a path counts as a page, whether or not it appears in the sidebar. The empty route is the docs root, and it always exists.

Usage

go run ./scripts/docsredirectscheck [-manifest path] [-redirects path] [-base rev] [-branch name] [-offline]

Exit status is 0 when the file is valid or absent, 1 when it has problems, and 2 when the check itself could not run. Examples are an unreadable manifest or a live page list that cannot be read.

Limitations

  • Chains through a wildcard destination are not checked, because where such a destination lands depends on the route being redirected. The removed-route check does follow rules, up to 3 hops, whatever the last destination is.
  • Only the two source forms above are understood. Regular-expression groups, version ranges, and mid-path parameters are rejected rather than guessed at.
  • The check does not know whether an external destination is reachable.