mirror of
https://github.com/coder/coder.git
synced 2026-11-22 03:18:31 +08:00
25af21b5d5d86ab7d9253e1e72d33b2da170e4f8
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7e0f62c204 | feat(coderd/templatebuilder): add base agent metadata and per-module agent targeting (#28794) | ||
|
|
c73f34c35d |
feat: honor the deployment registry mirror in template builder bases (#28727)
## Summary Threads the deployment's configured module registry (`CODER_TEMPLATE_BUILDER_REGISTRY_URL`) into template-builder **base** rendering, so a base that embeds a catalog module (e.g. the quickstart base's `git-clone`) resolves its `source` through a registry mirror the same way wizard-composed modules already do. Previously base-embedded module sources hardcoded `registry.coder.com`, so on a mirror/private-registry deployment a base-embedded module reached the public registry while wizard modules used the mirror. Implements [DOCS-610](https://linear.app/codercom/issue/DOCS-610). ## What changed * `BaseRenderContext.RegistryBase` (new field), mirroring `ModuleRenderContext.RegistryBase`. `Compose` threads the deployment registry through `renderBase` into the base render context, and the four hardcoded base module sources (`quickstart` git-clone, `azure-linux`/`gcp-linux`/`gcp-windows` region modules) now use `{{ .RegistryBase }}`. Because the value defaults back to `registry.coder.com`, rendered sources stay byte-identical; the only golden change is the removal of the now-stale quickstart "follow-up" note. * **Boot-time validation.** `codersdk.ValidateTemplateBuilderRegistryURL` runs in `DeploymentValues.Validate()`, gated on the builder being enabled, matching the chat-hook precedent. A malformed value fails at server start naming the option, instead of surfacing as a per-request error to a template author who cannot fix it. It uses `net/url` in authority form to reject a scheme, path, query, fragment, trailing slash, or credentials and accept a bare `host[:port]`; it does not canonicalize the host. * `Compose` **trusts the validated value**, defaulting an unset one and defensively rejecting a malformed one for direct callers that bypass boot. ## Tests * `codersdk`: validator accept/reject matrix (incl. credentials-without-echo) and a `DeploymentValues.Validate` boundary test with a disabled-skips-validation case. * `templatebuilder`: a base honors the mirror and falls back to the default when unset, plus a defensive-reject case; existing compose tests updated to bare-host registry values. <details><summary>Design decision log</summary> This PR is a deliberately minimal reimplementation of the [DOCS-610](https://linear.app/codercom/issue/DOCS-610/template-builder-base-embedded-modules-should-honor-the-deployment) scope, started fresh from `main` rather than carrying forward an earlier, larger draft (coder/coder#28480). Key decisions: * **Scope kept to the ticket.** The ticket's fix is three bullets: add a registry field to `BaseRenderContext`, thread the deployment `RegistryURL` through `Compose` -> `renderBase`, and use it in base module sources. Independent hardening explored in the earlier draft was dropped as out of scope: `terraform-svchost`-based IDN/punycode/port canonicalization, a rendered-HCL backstop with a 500 mapping, and HCL module-source extraction helpers. Telemetry misclassification of mirror modules is tracked separately ([DOCS-712](https://linear.app/codercom/issue/DOCS-712/telemetry-misclassifies-registry-mirror-modules-as-non-coder)) and is not addressed here. * **Default lives in the render layer.** `templatebuilder.DefaultRegistryBase` holds the fallback host; `codersdk` keeps its own option-default literal. An earlier version exported the constant from `codersdk`, which only emitted a dead `typesGenerated.ts` entry that nothing in the frontend consumed, so it was moved out. The `codersdk` validator is a function, so it generates no TS type. * **Validate at the config boundary, fail fast.** The repo convention is to hard-fail structurally invalid config in `DeploymentValues.Validate()` (access URL, OAuth lifetimes, chat hook URL) and to warn only for functional-but-risky settings. A malformed registry host is structurally invalid, so it fails at boot. This also puts the error in front of the admin who set it, rather than a template author hitting a per-request error they cannot fix. * `net/url`**, not a regex.** The codebase validates identifiers/names with regexes but host/URL-shaped config with `url.Parse` (chat hook URL, OAuth redirect URIs, AI provider base URL). This value is host-shaped, so it follows that convention. `url.Parse("//" + v)` with `u.Host == v` rejects a scheme/path/query/fragment/trailing-slash in one check. </details> > This PR was created with AI assistance (Coder Agents). |
||
|
|
c78dca5141 |
feat: add Coder Quickstart base to the template builder (#27247)
## Summary Adds the **Coder Quickstart** template as a selectable base in the [template builder](coderd/templatebuilder) guided wizard. It is a Docker-based starter that lets a user pick languages and optionally clone a repo. ## What changed - **New base package** `coderd/templatebuilder/bases/quickstart/`: - `base.json`: `id: "quickstart"`, `display_name: "Coder Quickstart"`, `os: "linux"`. - `main.tf.tmpl`: a Docker workspace with a language selector, a single language-install script, an optional Git clone, and workspace presets. Editors are added via the builder's module step rather than baked into the base. - `install-languages.sh.tftpl` and `README.md` (with prerequisites markers so the builder can extract the prerequisites section). - **Placement**: the bases endpoint returns a plain list sorted by display name (with ID as a deterministic tiebreak). The wizard's base-infra select step prioritizes the Quickstart and Docker starters to the front on the client, mirroring the existing client-side module prioritization in `ModuleSelectStep` (shared `sortByPriority` helper). - **Base/module collision guard**: a base's bundled catalog modules are derived from its rendered Terraform (`ExtractModuleNames`, cached with `sync.OnceValues` and filtered to catalog IDs). `validateModules` seeds its seen-set from that derived set, so a wizard-selected module the base already renders is rejected with a clear error, and the modules endpoint omits it so the wizard never offers a colliding module. The base's Terraform is the single source of truth, so there is no separate manifest list that can drift. ## Testing - `go test ./coderd/` (template builder handler: bases ordering, the `baseSpec` table, the modules-endpoint base filter): pass. - `go test ./coderd/templatebuilder/` (all-bases render/snapshot, the collision guard, `TestBaseIncludedModules`, the selector/install-script drift test): pass. - `gofmt`, `go vet`, `golangci-lint`: clean. - Frontend `tsc`, `biome`, and `knip`: clean. ## Review decisions - **Template scope**: trimmed to infrastructure, a language selector, and an optional Git clone. The IDE selector was removed; editors are added via the builder's module step. - **Placement**: base ordering is presentational, so it lives on the client base-infra select step rather than in the API. - **git-clone in the base**: kept for the build-time clone affordance. The base renders its module source verbatim, so it currently pins the public registry and does not yet honor a deployment's registry mirror; that fix is tracked in DOCS-610 and delivered as a stacked follow-up PR (an in-code note documents the current behavior). - **Container image**: left hardcoded (`codercom/enterprise-base:ubuntu`). The quickstart base is the opinionated path; the Docker base covers custom images. Revisiting a `container_image` variable is tracked in DOCS-611. ## Review updates (@jeremyruppel) - **Ordering moved to the client**: deleted the server-side Quickstart-before-Docker grouping (and its position test); the base-infra select step now prioritizes bases client-side via a shared `sortByPriority` helper, following the `ModuleSelectStep` precedent. The API just returns a stable, name-sorted list. - **Derive included modules from render**: dropped the `included_modules` manifest field and the `IncludedModules` struct field; `BaseIncludedModules` now derives from the rendered base (single source of truth). Removed `TestBaseIncludedModulesMatchRendered` and added a direct `TestBaseIncludedModules`. - **Kept the PR scoped to "add quickstart base"**: reverted `ExtractAgentResourceName`/`ExtractModuleNames` to the pre-`hclsyntax` regex extraction and removed the option/preset HCL extractor family; the selector↔install-script drift test now uses a small regex. The broader templatebuilder HCL migration and the preset-language subset guard are tracked in DOCS-735. - Branch rebased onto `origin/main` (no content changes beyond the above). --- Linear: [DOCS-558](https://linear.app/codercom/issue/DOCS-558/add-coder-quickstart-template-to-template-builder) > This PR was created with AI assistance (Coder Agents). |
||
|
|
d0982e3cc7 |
fix(coderd/templatebuilder): prompt for DigitalOcean base variables (#27268)
## Summary The DigitalOcean template builder base declared Terraform `variable` blocks for `project_uuid` and `ssh_key_id` that the template builder never filled. `project_uuid` was required with no default, so the build broke with no way to supply a value from the wizard (DEVEX-591). This brings the DigitalOcean base to parity with the GCP bases fixed in #27015: - Declare `project_uuid` (required) and `ssh_key_id` (optional, default `0`) in `base.json` so the wizard prompts for them on the first step. - Inject the entered values via `default = {{ .Variables.* }}` in `main.tf.tmpl`, keeping the existing `variable` blocks and validation. - Drop the `sensitive` flags. The variable-injection path (`mergeBaseVariables`, `DefaultBaseRenderContext`, and the snapshot test helper) skips sensitive variables, so a sensitive base variable renders empty. A project UUID / SSH key ID are not secrets. - Update the README now that the values are prompted rather than manually edited. - Regenerate the `digitalocean-linux.tf.golden` snapshot. ## Testing - `go test ./coderd/templatebuilder/` <img width="1048" height="616" alt="Screenshot 2026-07-15 at 12 44 42 PM" src="https://github.com/user-attachments/assets/89e334ca-e904-4387-9264-6ed1614a40ba" /> <details> <summary>Audit of all template builder bases for unfilled variables</summary> | Base | Variable status | Verdict | |------|-----------------|---------| | aws-linux | no HCL `variable` blocks; provider env auth | OK | | aws-windows | same | OK | | azure-linux | same | OK | | **digitalocean-linux** | `project_uuid` (required, no default) + `ssh_key_id`; absent from `base.json` | **Fixed here** | | docker | `docker_socket` has `default = ""`; `container_image` via `{{ .Variables }}` + declared | OK | | gcp-linux | fixed in #27015 | OK | | gcp-windows | fixed in #27015 | OK | | kubernetes | `namespace` (required), `use_kubeconfig`, `container_image` all via `{{ .Variables }}` + declared | OK | | scratch | no variables | OK | DigitalOcean was the only broken base; all others either have safe defaults or already declare their variables. **Mechanism note:** `base.json` `variables[]` drives the first-step prompts and values are injected as HCL literals via `{{ .Variables.<name> }}` (strings quoted, numbers/bools raw; supported types: string, number, bool). Sensitive/computed variables are intentionally skipped everywhere the injection map is built, so they cannot currently be injected. That is why the `sensitive` flags were removed here. </details> --- *This PR was generated by Coder Agents on behalf of @jeremyruppel.* |
||
|
|
581f906947 |
fix(coderd): declare project_id variable in GCP template builder bases (#27015)
## Summary
GCP base templates (`gcp-linux`, `gcp-windows`) in the Template Builder
had a Terraform `variable "project_id"` with no default, but their
`base.json` manifests didn't declare it. The UI never prompted for it,
so the provisioner import always failed with:
```
required template variables need values: project_id
```
## Changes
- Add `project_id` as a required variable in both GCP `base.json`
manifests
- Convert templates from raw Terraform variable blocks to Go template
injection (`{{ .Variables.project_id }}`), matching the existing
kubernetes pattern
- Fix `DefaultBaseRenderContext` to supply a `"REQUIRED"` placeholder
for required variables without defaults (previously rendered as `<no
value>`)
- Replace duplicate test subtests with proper GCP coverage including a
missing-variable error case
<details>
<summary>Implementation plan</summary>
### Root cause
The GCP base templates contained `variable "project_id" {}` (no default
= required) in their `.tf.tmpl` files, but the `base.json` manifests had
an empty `variables` array. The Template Builder UI
(`BaseTemplateParametersStep`) is data-driven from `base.json`, so it
never showed a field for `project_id`. The composed Terraform output
still contained the required variable, causing the provisioner import to
fail.
### Fix approach
Follow the pattern established by the kubernetes base template:
1. Declare variables in `base.json` so the UI prompts for them
2. Use Go template syntax (`{{ .Variables.project_id }}`) to inject
values at compose time
3. Remove raw Terraform `variable` blocks from the template since the
value is now baked in
### Files changed
| File | Change |
|---|---|
| `bases/gcp-linux/base.json` | Added `project_id` as a required
variable |
| `bases/gcp-windows/base.json` | Added `project_id` as a required
variable |
| `bases/gcp-linux/main.tf.tmpl` | Removed Terraform variable block, use
Go template injection |
| `bases/gcp-windows/main.tf.tmpl` | Same |
| `bases.go` | `DefaultBaseRenderContext` supplies placeholder for
required vars without defaults |
| `compose_test.go` | Replaced duplicate subtests with proper GCP tests
|
| `templatebuilder_handler_test.go` | Updated `gcp-windows` spec to
expect `project_id` variable |
| Golden files | Regenerated |
</details>
> 🤖 Generated by Coder Agents on behalf of @jeremyruppel
|
||
|
|
7b19ec3933 |
feat: improve the image management experience with template builder (#27018)
Makes it easier to pick the right workspace image, both in the template builder and in the docs. - Template builder: the Docker and Kubernetes bases now expose a `container_image` variable in the wizard (freeform text, defaults to `codercom/example-base:ubuntu`), and their prerequisites explain why image choice matters, with tradeoffs between `codercom/example-base:ubuntu` (minimal) and `codercom/example-universal:ubuntu` (catch-all), plus pointers to [coder/images](https://github.com/coder/images) and the image management docs. - Docs: reworked [image management](https://coder.com/docs/@ben%2Fdevrel-201-image-guidance-prereqs/admin/templates/managing-templates/image-management) into a clearer maturity ladder (minimal → golden → project-specific → developer customization), with pullable image references in every example, `codercom/oss-dogfood` as a project-specific example, and Dev Containers + [mise](https://mise.jdx.dev/) as ways to customize without new images. Companion PR for the starter templates: coder/registry#943 Part of DEVREL-201. 🤖 Generated with Coder Agents using Claude, on behalf of @bpmct (wizard variable by @jeremyruppel in #27024) --------- Co-authored-by: Jeremy Ruppel <jeremyruppel@users.noreply.github.com> |
||
|
|
c40b3b9bcb |
feat: add base templates for all major cloud providers (#26634)
Add 6 new base templates to the template builder, covering all major cloud providers and platforms: - **scratch**: Minimal starter template with only `coder_agent` and metadata - **aws-windows**: AWS EC2 Windows instances with PowerShell user_data - **azure-linux**: Azure VMs with cloud-init and managed disk persistence - **gcp-linux**: Google Compute Engine Linux instances with persistent disk - **gcp-windows**: Google Compute Engine Windows instances - **digitalocean-linux**: DigitalOcean Linux droplets with persistent volumes Each base includes `base.json`, `main.tf.tmpl`, `README.md` (with prerequisite markers), and any static files (cloud-init configs). Tests verify all 9 bases load, render without error, and produce valid single-agent declarations. `azure-windows` is deferred; it needs to be registered in the `examples` package first. Depends on #26633 > [!NOTE] > This PR was authored by Coder Agents on behalf of @jeremyruppel. --- NB: This is very much an agent-generated PR and draws completely from base templates that exist in `examples/templates/`. The base.json files are new, so review those, but don't spend any brain tokens on the correctness of the terraform and supporting files: any issues there are issues with the upstream example template |
||
|
|
10717572ac |
feat: show template prerequisites in builder UI (#26523)
Surface base template prerequisites to admins before they create a
template in the Template Builder wizard.
Today, template prerequisites (Docker socket setup, Kubernetes auth, AWS
IAM policies) are only visible in the registry README after import.
Admins hit opaque provisioner errors and have to hunt for docs. This
change extracts the prerequisites from the README and serves them via
the API so the frontend can display them inline.
## How it works
Each base template README uses HTML comment markers (`<!--
prerequisites:start -->` / `<!-- prerequisites:end -->`) to delimit the
prerequisites section. At boot time, the base catalog loader reads the
README, extracts the content between markers via `strings.Index`, and
caches both the full README and the prerequisites string.
The prerequisites are served via a new `prerequisites` field on `GET
/api/v2/templatebuilder/bases`. The full README is included in the
composed template tar bundle and stored as the template version readme.
## Changes
- Add `README.md` with prerequisite markers to
`coderd/templatebuilder/bases/{docker,kubernetes,aws-linux}/`
- New `ExtractPrerequisites()` in `prerequisites.go` using literal
string matching
- `bases.go`: load README at boot, fail loudly if missing, extract
prerequisites
- `compose.go`: include README in `ComposeResult` and tar bundle
- `codersdk`: add `Prerequisites` field to `TemplateBuilderBase`
- Handler: populate prerequisites in bases response, set readme on
template version
<details>
<summary>Implementation notes</summary>
- Prerequisites extraction uses `strings.Index` for exact literal marker
matching; no regex or AST parser needed since we control the markers.
- YAML frontmatter is deliberately retained in the stored README. The
frontend `TemplateDocsPage` already strips it at render time via
`front-matter`.
- The prerequisite markers are HTML comments, invisible in rendered
markdown.
- The `RejectsMissingReadme` test enforces that every base template must
include a README.
- AWS Linux prerequisites span two H2 sections (`## Prerequisites` and
`## Required permissions / policy`), which is why heading-based parsing
was rejected in favor of explicit markers.
*Generated with the assistance of an AI coding agent. Reviewed by
@jeremyruppel.*
</details>
Relates to https://linear.app/codercom/issue/DEVEX-446
|
||
|
|
7ea5d48296 |
fix: render base variables into templates instead of tfvars (DEVEX-287) (#26436)
Part of the Template Builder wizard PR stack.
## Problem
The kubernetes base template used Terraform `variable` blocks and
`var.*` references for `use_kubeconfig` and `namespace`, but the
composed tar bundle never included a `.tfvars` file. This caused
`terraform plan` to fail with "required template variables need values:
namespace".
## Fix
Base templates now use Go template variables (`{{ .Variables.* }}`) just
like module templates do. Values are validated, HCL-quoted, and rendered
directly into the output HCL.
Also adds explicit "variable is required" validation to both
`mergeBaseVariables` and `mergeModuleVariables`, replacing the previous
reliance on `missingkey=error` at render time for clearer error
messages.
---
> [!NOTE]
> Generated by Coder Agents on behalf of @jeremyruppel
|
||
|
|
87de6dc23e | feat: add base template variables to API (#26425) | ||
|
|
71957a2bc0 |
feat(coderd/templatebuilder): add exampleID->OS map and base template .tf.tmpl files (#26115)
Add the bundled `exampleID -> OS` Go map for Docker, Kubernetes, and AWS
EC2 Linux base templates. Create `.tf.tmpl` Go template files for each
within `coderd/templatebuilder/bases/`, along with `BaseRenderContext`
and `RenderBaseTemplate` rendering helpers.
The `.tf.tmpl` files are independent copies of the example templates
with module blocks (code-server, jetbrains) removed, since the template
builder composes modules separately into `modules.tf`. When
`ImageOptions` is provided, the container image field references the
Terraform parameter; otherwise it uses the hardcoded value via Go
template whitespace control (`{{-`).
Golden file snapshot tests verify rendered output stability with an
`-update` flag for regeneration.
Depends on #25909
> [!NOTE]
> This PR was authored by Coder Agents on behalf of @jeremyruppel.
|