mirror of
https://github.com/coder/coder.git
synced 2026-11-22 11:28:40 +08:00
feea5ab959f59cd2135443ed2a0c19bf7643db8f
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
d187dea338 |
docs: end every page description with a period and lint for it (#30863)
## Summary Page descriptions in the docs were inconsistent about ending with a period. This makes every page description end with one and adds a lint so it stays that way. A page description can come from three places, and the docs site reads front matter first and falls back to the manifest: - the hand-written YAML sidebar sources under `docs/manifest/` (including `children_from` overlays), - the generated fragments (`docs/manifest/generated/cli.json`, `rest-api.json`), compiled into `docs/manifest.json`, - a page's own front matter (today only the generated CLI and API reference pages carry one). The check covers all three. For the CLI, the description comes from each command's `Short`, which stays unpunctuated by CLI-help convention, so `clidocgen` adds the period when it writes the docs description instead of changing 120 `Short` strings and the CLI golden files. ## What the lint checks `make lint/docs-manifest` (`docsmanifestgen check`, already part of `make lint` in CI) now fails when a non-empty description doesn't end with `.`. No description in the tree ends with `?` or `!`, so the rule allows only a period. Errors name the source file, the sidebar breadcrumb, and the description, for example: ``` install/plan/sizing.yml: Install > Plan your deployment > Coder Validated Architecture > Up to 1,000 users: description must end with a period: "Hardware specifications and architecture guidance for Coder deployments that support up to 1,000 users" generated/api.json: General: description must end with a period: "General endpoints" reference/cli/ping.md: Reference > Command Line > `ping`: front matter description must end with a period: "Ping a workspace" ``` The style guide's page-description rule now lists the requirement and cites the checker, and `styleclaims` recognises `scripts/docsmanifestgen` so the coverage table counts it as tool-checked. ## Count of descriptions fixed 147 descriptions in `docs/manifest.json`: - 27 hand-written descriptions in 15 sidebar source files, - 120 generated CLI descriptions, which also update the front matter of 120 CLI reference pages. The REST API descriptions already ended with a period. No description is otherwise reworded. ## Testing - New table cases in `scripts/docgenenv` (section route, child route, overlay, generated fragment, front matter, and a `?` ending) and `scripts/clidocgen`, written first and failing before the change. - `docsmanifestgen check` on the unfixed tree reported the 27 source descriptions; after the fixes and regeneration it passes. - `go test ./scripts/docgenenv/ ./scripts/clidocgen/ ./scripts/docsmanifestgen/ ./scripts/styleclaims/ ./scripts/apidocgen/...` - `golangci-lint run` on the changed packages, `pnpm run check-docs`, `go run ./scripts/styleclaims`, `scripts/check_emdash.sh`, and `docsmanifestgen fmt` (no changes). Linear: [DOCS-1304](https://linear.app/codercom/issue/DOCS-1304) --- 🤖 Built with AI assistance. |
||
|
|
12caf883fc |
test(scripts/docgenenv): match missing include by fs.ErrNotExist (#30285)
## Summary `TestBuildManifestErrors/IncludeMissingFile` fails deterministically on Windows because it matches the POSIX error text `no such file or directory`. Windows reports the same condition as `The system cannot find the file specified.` This asserts `fs.ErrNotExist` in the error chain instead, plus the included file name. Fixes [ENG-3439](https://linear.app/codercom/issue/ENG-3439). ## Changes - Add an optional `wantErrIs` field to the `TestBuildManifestErrors` table, checked with `require.ErrorIs`. - `IncludeMissingFile` now expects `fs.ErrNotExist` and the text `nope.yml` (not `ref/nope.yml`, since Windows renders the path with escaped backslashes). No production code changes: `DecodeSourceFile` already wraps the `os.ReadFile` error with `%w`, and `errors.Join` preserves it. ## Testing - `go test ./scripts/docgenenv/... -count=1` passes on Linux. - `GOOS=windows go test -c ./scripts/docgenenv` compiles. - `golangci-lint run ./scripts/docgenenv/...` reports 0 issues; pre-commit hook passed. - Not run on Windows locally; the `test-go-pg (windows-2022)` job is the real check. --- _Generated by Coder Agents on behalf of @nickvigilante._ |
||
|
|
f1ed84ffab |
docs: author the sidebar in per-section YAML compiled to manifest.json (#30165)
## Summary The docs sidebar is now written as YAML files under `docs/manifest/`, one route and its direct children per file, and a new tool, `scripts/docsmanifestgen`, compiles them into `docs/manifest.json`. Moving two top-level sections is now a 2-line change in `docs/manifest/index.yml` instead of a diff of hundreds of lines in a 2,600-line JSON file. The compiled `docs/manifest.json` is byte-identical to `main`, so coder.com, docs-preview, deploy-docs, and offlinedocs need no changes. Refs [DOCS-1131](https://linear.app/codercom/issue/DOCS-1131/author-the-docs-sidebar-in-per-section-yaml-compiled-to) ## Changes - **Sources:** `docs/manifest/index.yml` holds `versions` and the section files in sidebar order. Each section file (`about.yml`, `admin.yml`, and so on) holds one top-level route. - **One level per file:** a file holds one route and its direct children. A child with children of its own, or with `children_from`, lives in its own file in a directory named after the parent file, and the parent lists it as `- include: admin/users.yml`. For example, `get-started.yml` includes `get-started/customize-your-template.yml`, which lists that page's children. The 8 section files split into 53 source files. - **`scripts/docsmanifestgen`:** `build` compiles the sources into `docs/manifest.json`, `fmt` formats them, and `check` validates them. - Keys must be known (a typo like `titel:` fails with a line number) and appear in a fixed order in both the YAML and the JSON. - Every `path` must exist, and every file under `docs/manifest/` must be referenced from `index.yml`, an `include`, or a `children_from`. - A grandchild in the same file fails with a message to move the child into its own file, and an `include` entry can't set other keys or name a file that is already used. - A comment on the same line as a `title` or `description` is rejected, because an unquoted ` #` would silently cut off the value. - **Generated fragments:** `clidocgen` and `apidocgen` no longer rewrite the manifest. They write the Command Line and REST API children to `docs/manifest/generated/*.json`, which the sources pull in with `children_from`. The tool accepts fragments as YAML or JSON. - The hand-written REST API descriptions and states moved into the YAML, attached to generated routes by title. A YAML entry that matches no generated route, or disagrees with one, fails the build instead of being dropped silently. - **Shared code:** loading, validation, and fragment merging live in `scripts/docgenenv/sources.go`, so the generators and the tool share one loader. `docgenenv.Route` gains YAML tags and the source-only `ChildrenFrom` and `Include` fields (`json:"-"`). `LoadManifest` is removed because nothing reads the compiled JSON anymore. - **Makefile:** - `docs/manifest.json` is built from the sources and fragments, which replaces the `_gen/manifest-staging.json` handoff. - Each fragment has its own rule, and the fragments are added to the `.PRECIOUS`, `GEN_FILES`, and `gen/mark-fresh` lists. - `DOCS_MANIFEST_SOURCES` finds the source files recursively. - New targets: `fmt/docs-manifest` (part of `fmt`), `lint/docs-manifest` (part of `lint`), and `gen/docs-manifest`, the quick path after a sidebar-only edit that skips regenerating the CLI and API fragments. - **`.gitattributes`:** marks `docs/manifest.json` and `docs/manifest/generated/*` as generated so GitHub collapses them in diffs. - **Guidance:** the content guidelines, the style guide, `DOCS_STYLE_GUIDE.md`, the `write-docs` and `doc-check` skills, `docs/README.md`, and `docs/tutorials/example-guide.md` now point to the YAML sources instead of `docs/manifest.json`. ## Testing - Split today's `docs/manifest.json` into the sources, rebuilt it, and confirmed the output is byte-identical to `main`. After splitting the sources to one level per file, the rebuilt `docs/manifest.json` and a fresh `clidocgen` CLI fragment were still byte-identical, and `LoadManifestSources` still finds the REST API and Command Line routes through their includes. After rebasing onto `main`, both commits rebuild `main`'s `docs/manifest.json` byte for byte. - Hand-edited `docs/manifest.json` and a YAML source, then rebuilt: the YAML change persisted and the JSON edit was overwritten. CI's `make -B gen` plus `check_unstaged.sh` fails on a committed hand edit to the JSON. - Deleted both fragments and ran `make docs/manifest.json` in a fresh worktree, which regenerated them through swag and `clidocgen`. `docs/manifest.json`, the fragments, and every generated CLI and API page had no diff. - Swapped User Guides and Administration in `index.yml`: 2 changed lines in the source, 166 generated lines in `docs/manifest.json`. Swapping back restored the file exactly. - `go test` for `scripts/docgenenv`, `scripts/docsmanifestgen`, `scripts/clidocgen`, and `scripts/apidocgen/postprocess`. The new tests cover merging, includes, every validation error, idempotent formatting that keeps comments, and a check that the key order matches the struct field order. - `golangci-lint` on the changed packages, and the full `make pre-commit` hook (gen, fmt, lint including `lint/docs-manifest`, and the slim build). ## Notes - Open PRs that edit `docs/manifest.json` will need to redo their sidebar changes in the YAML sources after this merges. - #30161 (merged) stopped escaping `&`, `<`, and `>` in `docs/manifest.json`. This PR carries that over: all manifest JSON escaping is in `docgenenv.MarshalJSON`, which sets `enc.SetEscapeHTML(false)`, and `Manifest.Marshal` from #30161 is gone because the generators no longer write the manifest. - #30129 (merged) flattened `admin/external-auth/index.md` and `ai-coder/agents/tools/index.md` to single files, and the YAML sources use the new paths. - The CLI index and REST API page front matter still update only on a full `make gen` when their sidebar metadata changes, as it did before. That gap and the overall cost of `make gen` are tracked in [DOCS-1141](https://linear.app/codercom/issue/DOCS-1141/reduce-the-compute-cost-of-make-gen). <details> <summary>Decision log</summary> ### Goal Make the sidebar easy to reorder and hard to break, with strict key order and YAML sources, while keeping `docs/manifest.json` as the file downstream systems read. ### Jsonnet vs. a custom tool Jsonnet was evaluated first and rejected, based on tests against go-jsonnet v0.22.0: - Jsonnet always sorts object keys alphabetically, and there's no option to keep them in order (`preserve_order` isn't a parameter of `manifestJsonEx` or `objectFieldsEx`). Passing today's manifest through Jsonnet and then biome changed 2,114 of 2,478 lines. - Unknown keys pass through silently, so a `titel:` typo ends up in the output. - The sources would be `.jsonnet` files, not YAML. Reading YAML takes `std.parseYaml(importstr ...)`, and the keys still come out sorted. - Restoring key order would need a Go step after Jsonnet anyway, and that step is most of this tool. Decoding YAML into Go structs with yaml.v3 (already a dependency) gives key order from the struct, rejects unknown keys, keeps comments, and reproduced the current manifest byte for byte. ### Design choices - **Ordered by `index.yml`:** reordering sections touches only the index. - **One route and its direct children per file:** every file stays short and flat, so a subsection can be found, reviewed, and moved as one file. Files nest by parent file name rather than by docs path, so the directory tree matches the sidebar tree. Include paths are relative to `docs/manifest/`, like `children_from`, so every reference to a file is spelled the same way. A child with `children_from` also gets its own file, so the rule has no exceptions. - **Generated subtrees in their own fragments:** the generators can't safely rewrite hand-edited YAML, so they own separate files. Hand-written metadata for generated routes lives in the YAML, attached by title, so descriptions never live in a generated file. - **Key order enforced in both sources and output:** `fmt` rewrites the sources, `check` fails on unformatted sources, and the output order comes from the `docgenenv.Route` field order. - **`&` left unescaped:** the output matches `main` after #30161, which stopped escaping `&` downstream. - **`gen/docs-manifest` separate from `fmt`:** keeping manifest assembly out of `fmt` means `make -j gen fmt` never has two jobs writing `docs/manifest.json` at once. - **YAML pitfalls:** unquoted `no` and `123` stay strings because the fields are typed as strings, an unquoted `: ` is a parse error, and the silent ` #` truncation is rejected by the comment check. ### Out of scope - Replacing `&` with "and" in titles (a style question). - Reducing the cost of `make gen` (DOCS-1141). </details> --- _This PR was generated by Coder Agents on behalf of @nickvigilante._ |
||
|
|
0ce91ffc18 |
fix: stop escaping ampersands in docs manifest titles (#30161)
Closes DOCS-1139.
Go's `json.MarshalIndent` escapes `&`, `<` and `>` for HTML safety, so
both docs generators wrote titles like "Licensing & usage" into
`docs/manifest.json` as `Licensing & usage`. The two forms decode
identically (the offline docs site reads the file with `JSON.parse`),
but the escaped form reads like a bug in review, and a hand-typed `&`
flipped back on the next `make gen`.
Add `Manifest.Marshal` in `scripts/docgenenv`, which disables HTML
escaping, and use it from `clidocgen` and `apidocgen/postprocess`. The
two titles in `docs/manifest.json` that still carried the escape are
updated to match. A unit test covers the unescaped output and the round
trip through `LoadManifest`.
Testing: `go test ./scripts/docgenenv ./scripts/clidocgen
./scripts/apidocgen/...` passes. I ran `clidocgen -manifest-only`
against a copy of the manifest and confirmed the output has no escapes
and parses to the same content as the committed file. I did not run the
full `make docs/manifest.json` (it needs the site dependencies and
Biome), so the committed file keeps its existing Biome formatting.
cc @ianjevans for awareness.
---
🤖 Built with AI assistance.
|
||
|
|
0a7bb80a1e |
feat: make CLI/API doc generators emit front-matter metadata (Phase 2) (#27246)
## Summary
Phase 2 of the H1 → front-matter migration (`DOCS-483`; parent
`DOCS-477`). Makes the two reference-doc generators emit per-page
metadata as YAML front matter instead of a leading `# H1`, so generated
pages are self-describing and `make gen` stops reverting migrated pages
(Phase 3).
Phase 1 (`DOCS-482`) made the coder.com renderers prefer a front-matter
`title` (manifest fallback).
> [!NOTE]
> Rebased onto `main` and fully regenerated, and updated across two
rounds of Coder Agents Review — see **Review follow-ups** below.
## Changes
- **`scripts/clidocgen/command.tpl` + `gen.go` + `main.go`** — front
matter now carries `title` (from `fullName`) and `description` (from the
command's `Short`), and the leading `# H1` is dropped. The CLI index
page's front matter is taken from the manifest `Command Line` route
(title/description/icon_path).
- **`scripts/apidocgen/postprocess/main.go`** — reads the manifest and,
at write time, injects front matter carrying each section's `title` plus
any curated `description`, `state`, and `icon_path`. The API index
page's front matter is taken from the manifest `REST API` route.
- **`scripts/docgenenv`** (new shared code) — one `YAMLScalar`
front-matter escaper, one `Route`/`Manifest` schema +
`LoadManifest`/`FindRoute`, and one `FrontMatter(Route)` emitter, all
imported by both generators (no duplicated helpers, types, or emitters).
- Regenerated all **166 CLI + 31 API** reference pages.
### Metadata → front matter, and what stays in the manifest
Every *per-page* manifest field is mirrored into the page's front
matter: `title`, `description`, `state`, `icon_path`. The **structural**
fields stay in `manifest.json`:
- `children` — the nav tree (explicitly out of scope).
- `path` — the manifest's pointer to the file; a page carrying its own
path is redundant/error-prone, so it's treated like `children`.
The fields are **duplicated** into front matter and **`manifest.json` is
left unchanged**, so this is a **no-op for rendering today** (coder.com
strips front matter for `llms`, and Algolia + the renderer read only
`title`). Removing the fields from the manifest is the natural
follow-up, gated on the renderer reading them from front matter first.
### Why the API side changes the postprocessor, not the `.dot` templates
The issue text suggested editing
`scripts/apidocgen/markdown-template/*`. I deliberately did **not**,
because the postprocessor derives each page's **filename, section title,
and manifest route** from the leading `# {name}` line
(`extractSectionName`). Emitting front matter from the template would
break that extraction. Instead the widdershins templates still emit `#
{name}`, the postprocessor reads it (and now verifies it), and then
swaps the heading for a front-matter block as each section is written.
## Review follow-ups (Coder Agents Review)
### Round 1 — addressed in `e53d5e03` (all threads resolved)
- **CRF-1 / CRF-4** — de-duplicated the escaper and the
`route`/`manifest` schema + traversal into `scripts/docgenenv` (shared
by both generators).
- **CRF-2** — `YAMLScalar` now quotes YAML-reserved scalars
(`true/false/null/…`, numbers); no current value is affected.
- **CRF-3** — added unit tests: a `YAMLScalar` round-trip, `FindRoute`,
and `prependFrontMatter`.
- **CRF-5** — the CLI and API **index** pages now mirror their manifest
route's title/description/icon_path instead of a hardcoded
`coder`/`API`, fixing a rendered-heading regression (`REST API`/`Command
Line` were being overwritten).
- **CRF-6** — dropped the dead `#login` anchor in
`docs/support/support-bundle.md` (the migrated `login.md` no longer
mints that heading anchor).
- **CRF-7 / CRF-8 / CRF-11** — renamed to `prependFrontMatter`, switched
to `bytes.Cut`, and it now strips the first line only when it is the `#
{name}` heading (`extractSectionName` errors otherwise).
- **CRF-9** — removed the orphan `docs/reference/api/chat.md` (not in
the manifest, not linked; the real page is `chats.md`).
- **CRF-10** — the metadata read and the manifest rewrite now share one
`FindRoute` traversal.
- **CRF-13** — moot under squash-merge; this branch is a single
scopeless commit.
- **CRF-15** — the pre-existing `sort.Slice`/`slices.IsSorted`
comparator is left as-is per the review (out of scope; safe today
because section names are unique).
### Round 2 — addressed in `ee796e7107` (all threads resolved)
- **CRF-16** (P1) — removed three em-dashes from new doc comments (the
only `make lint` failure on the prior head); the emdash gate is green.
- **CRF-17 / CRF-18** — unified front-matter emission into one shared
`docgenenv.FrontMatter(Route)`, used by the API postprocessor directly
and by `command.tpl` via a `frontMatter` template func. This retires the
hand-written template YAML and the
`indexTitle`/`indexDescription`/`indexIconPath` closures, so a new
front-matter field is wired in one place, and it gives the CLI index the
`state` arm it previously lacked. Verified byte-identical: a full CLI +
API regen produces zero page changes.
- **CRF-19** — CLI child sort switched to `slices.SortFunc` +
`cmp.Compare` (typed comparator).
- **CRF-20** — reworded the `prependFrontMatter` comment:
`extractSectionName`'s fail-fast is the load-bearing guard; the prefix
check is a defensive backstop.
- **CRF-21** — added `icon_path`/`state` coverage in `docgenenv`'s
`TestFrontMatter/AllFields` (the branch the index page relies on,
previously at 0%).
- **CRF-22** — `YAMLScalar` no longer emits a trailing-space value as a
bare scalar (YAML strips it on read, so it would not round-trip); added
test coverage.
- **CRF-24** — the shared emitter removed the duplicated `cliIndexRoute`
doc comment; the rationale now lives in one place.
- **CRF-23** (Phase 3, out of scope here) — noted: the API generator
wipes and regenerates `reference/api/` from the manifest, so removing
curated metadata from the manifest in Phase 3 needs another source first
(a generator that preserves existing front matter, or metadata carried
alongside the swagger annotations).
- **Process (Mafu-san)** — the verification set below now leads with
`make lint`, the mandatory CI gate that the earlier list omitted.
## Cross-repo dependency
**Resolved — this PR no longer has a hard merge-ordering gate** (CRF-14
was right; the earlier "must merge after #968" note was stale).
The coder.com surfaces that would otherwise leak raw front matter from
`coder/coder` `main` are already front-matter-aware on merged PRs:
- **coder.com#964** (`DOCS-554`, llms-full.txt corpus + Algolia) —
**merged**.
- **coder.com#974** (`DOCS-574`, the `.md` proxy twin + `llms.txt` index
titles) — **merged**.
coder.com#968 (`DOCS-577`) was re-scoped to only the renderer
route-metadata generalization; it's a no-op on today's corpus and its
own description confirms the "deploy before the generators" constraint
no longer applies (that was driven by the llms corpus, now in #964).
Worth a final confirmation that #964/#974 are **deployed** before merge,
but there's no branch/PR ordering blocker left.
## Verification & evidence
AI was the primary author of this PR (see disclosure below); per the [AI
Contribution
Guidelines](https://coder.com/docs/about/contributing/AI_CONTRIBUTING)
here is manual verification.
- `make lint` (golangci-lint + the emdash gate) passes; `go build` / `go
vet` / `go test` are clean for the generators + `scripts/docgenenv`;
`pnpm check-docs` passes.
- `swagger.json`, `docs.go`, and `manifest.json` are **unchanged** —
metadata is duplicated into front matter; command/section names and
routes did not move.
- The diff is purely additive front matter
(`title`/`description`/`state`/`icon_path`) + the leading H1 removal; no
body reflow. A full CLI + API regen produces **zero** page changes
beyond the two index pages.
<details>
<summary>Terminal evidence</summary>
CLI `description` from the command's `Short` (`YAMLScalar` quotes when
needed, e.g. a `Short` with a colon):
```md
---
title: server
description: Start a Coder server
---
```
API pages inherit curated manifest metadata (only Agents/Chats have any
today):
```md
---
title: Chats
description: "REST endpoints for Coder Agents Chats API (programmatic agent sessions)."
state:
- early access
---
```
Diff scope + "no body changes" proof (uses an explicit `base..HEAD`
range, so it actually tests the claim):
```
$ git diff --shortstat origin/main
210 files changed, 1447 insertions(+), 344 deletions(-)
# = 166 CLI + 31 API reference pages + generators + scripts/docgenenv
# swagger.json / docs.go / manifest.json: NOT modified
# Every removed line under docs/reference is a leading "# H1"; nothing else:
$ git diff origin/main..HEAD -- docs/reference/ | grep '^-' | grep -v '^---' | grep -v '^-# '
(empty)
$ pnpm check-docs
Summary: 0 error(s)
```
</details>
Linear: DOCS-483
> This PR was created with AI assistance (Coder Agents).
|
||
|
|
1cc230b43a |
refactor: extract docgen env prep into a shared package (#26827)
## What `clidocgen` and the new `configdocgen` (coder/coder#26824) both carried a byte-identical `prepareEnv()` that unsets `CODER_*` and pins `CLIDOCGEN_*` / `TMPDIR` so generated docs don't embed the generating host's home directory. This extracts it to `scripts/docgenenv.Prepare()` and migrates `clidocgen`. ## Why Duplication flagged during review of #26824. `configdocgen` adopts the shared helper in that PR, removing its copy. ## Risk Behavior-preserving: regenerating the CLI reference (`make docs/reference/cli/index.md`) yields no diff, and `make pre-commit` passes (`lint/go`, `lint/ts`, `build`). A focused unit test pins the `Prepare()` contract, and `_test.go` files are excluded from `CLIDOCGEN_INPUTS` so test edits don't mark the generated docs stale. <details> <summary>CI status — blocked by an unrelated <code>main</code> breakage (#24993)</summary> All red checks on this PR are inherited from `main`, not caused by these changes. This PR touches only `Makefile` and `scripts/{clidocgen,docgenenv}`; it does not touch Helm. `main` went red at `d0f68cb9b0` ("feat: add listenerset", #24993, merged ~18:26 UTC). The committed `helm/coder/tests/testdata/listenerset*.golden` files don't match what `helm template` renders, so: - **`gen`** regenerates those goldens, and the unstaged-files check fails. - **`test-go-pg` (ubuntu-latest, pg-17) and `test-go-race-pg`** fail only on `TestRenderChart/{coder,default}/listenerset[_redirect]` (golden mismatch; the test prints "Run with -update to update golden files"). The same `test-go-pg` job passes on macOS and Windows, where the Helm render test is skipped, and `scripts/docgenenv` reports `ok` on the failing runners. Base commit `14a61041d9` was green; `main` is red from `d0f68cb9b0` onward. These checks clear once `main` is fixed and this branch is updated. `fmt`, `lint`, `Storybook`, `check-build`, and `test-e2e` are green. </details> --- 🤖 Opened by Coder Agents on behalf of @nickvigilante. --------- Co-authored-by: Cian Johnston <cian@coder.com> |