Commit Graph
8 Commits
Author SHA1 Message Date
Jeremy Ruppel c73f34c35d feat: honor the deployment registry mirror in template builder bases (#28727)
## Summary

Threads the deployment's configured module registry
(`CODER_TEMPLATE_BUILDER_REGISTRY_URL`) into template-builder **base**
rendering, so a base that embeds a catalog module (e.g. the quickstart
base's `git-clone`) resolves its `source` through a registry mirror the
same way wizard-composed modules already do. Previously base-embedded
module sources hardcoded `registry.coder.com`, so on a
mirror/private-registry deployment a base-embedded module reached the
public registry while wizard modules used the mirror.

Implements [DOCS-610](https://linear.app/codercom/issue/DOCS-610).

## What changed

* `BaseRenderContext.RegistryBase` (new field), mirroring
`ModuleRenderContext.RegistryBase`. `Compose` threads the deployment
registry through `renderBase` into the base render context, and the four
hardcoded base module sources (`quickstart` git-clone,
`azure-linux`/`gcp-linux`/`gcp-windows` region modules) now use `{{
.RegistryBase }}`. Because the value defaults back to
`registry.coder.com`, rendered sources stay byte-identical; the only
golden change is the removal of the now-stale quickstart "follow-up"
note.
* **Boot-time validation.**
`codersdk.ValidateTemplateBuilderRegistryURL` runs in
`DeploymentValues.Validate()`, gated on the builder being enabled,
matching the chat-hook precedent. A malformed value fails at server
start naming the option, instead of surfacing as a per-request error to
a template author who cannot fix it. It uses `net/url` in authority form
to reject a scheme, path, query, fragment, trailing slash, or
credentials and accept a bare `host[:port]`; it does not canonicalize
the host.
* `Compose` **trusts the validated value**, defaulting an unset one and
defensively rejecting a malformed one for direct callers that bypass
boot.

## Tests

* `codersdk`: validator accept/reject matrix (incl.
credentials-without-echo) and a `DeploymentValues.Validate` boundary
test with a disabled-skips-validation case.
* `templatebuilder`: a base honors the mirror and falls back to the
default when unset, plus a defensive-reject case; existing compose tests
updated to bare-host registry values.

<details><summary>Design decision log</summary>

This PR is a deliberately minimal reimplementation of the
[DOCS-610](https://linear.app/codercom/issue/DOCS-610/template-builder-base-embedded-modules-should-honor-the-deployment)
scope, started fresh from `main` rather than carrying forward an
earlier, larger draft (coder/coder#28480). Key decisions:

* **Scope kept to the ticket.** The ticket's fix is three bullets: add a
registry field to `BaseRenderContext`, thread the deployment
`RegistryURL` through `Compose` -> `renderBase`, and use it in base
module sources. Independent hardening explored in the earlier draft was
dropped as out of scope: `terraform-svchost`-based IDN/punycode/port
canonicalization, a rendered-HCL backstop with a 500 mapping, and HCL
module-source extraction helpers. Telemetry misclassification of mirror
modules is tracked separately
([DOCS-712](https://linear.app/codercom/issue/DOCS-712/telemetry-misclassifies-registry-mirror-modules-as-non-coder))
and is not addressed here.
* **Default lives in the render layer.**
`templatebuilder.DefaultRegistryBase` holds the fallback host;
`codersdk` keeps its own option-default literal. An earlier version
exported the constant from `codersdk`, which only emitted a dead
`typesGenerated.ts` entry that nothing in the frontend consumed, so it
was moved out. The `codersdk` validator is a function, so it generates
no TS type.
* **Validate at the config boundary, fail fast.** The repo convention is
to hard-fail structurally invalid config in
`DeploymentValues.Validate()` (access URL, OAuth lifetimes, chat hook
URL) and to warn only for functional-but-risky settings. A malformed
registry host is structurally invalid, so it fails at boot. This also
puts the error in front of the admin who set it, rather than a template
author hitting a per-request error they cannot fix.
* `net/url`**, not a regex.** The codebase validates identifiers/names
with regexes but host/URL-shaped config with `url.Parse` (chat hook URL,
OAuth redirect URIs, AI provider base URL). This value is host-shaped,
so it follows that convention. `url.Parse("//" + v)` with `u.Host == v`
rejects a scheme/path/query/fragment/trailing-slash in one check.

</details>

> This PR was created with AI assistance (Coder Agents).
2026-08-31 09:32:24 -04:00
Nick Vigilante c78dca5141 feat: add Coder Quickstart base to the template builder (#27247)
## Summary

Adds the **Coder Quickstart** template as a selectable base in the
[template builder](coderd/templatebuilder) guided wizard. It is a
Docker-based starter that lets a user pick languages and optionally
clone a repo.

## What changed

- **New base package** `coderd/templatebuilder/bases/quickstart/`:
- `base.json`: `id: "quickstart"`, `display_name: "Coder Quickstart"`,
`os: "linux"`.
- `main.tf.tmpl`: a Docker workspace with a language selector, a single
language-install script, an optional Git clone, and workspace presets.
Editors are added via the builder's module step rather than baked into
the base.
- `install-languages.sh.tftpl` and `README.md` (with prerequisites
markers so the builder can extract the prerequisites section).
- **Placement**: the bases endpoint returns a plain list sorted by
display name (with ID as a deterministic tiebreak). The wizard's
base-infra select step prioritizes the Quickstart and Docker starters to
the front on the client, mirroring the existing client-side module
prioritization in `ModuleSelectStep` (shared `sortByPriority` helper).
- **Base/module collision guard**: a base's bundled catalog modules are
derived from its rendered Terraform (`ExtractModuleNames`, cached with
`sync.OnceValues` and filtered to catalog IDs). `validateModules` seeds
its seen-set from that derived set, so a wizard-selected module the base
already renders is rejected with a clear error, and the modules endpoint
omits it so the wizard never offers a colliding module. The base's
Terraform is the single source of truth, so there is no separate
manifest list that can drift.

## Testing

- `go test ./coderd/` (template builder handler: bases ordering, the
`baseSpec` table, the modules-endpoint base filter): pass.
- `go test ./coderd/templatebuilder/` (all-bases render/snapshot, the
collision guard, `TestBaseIncludedModules`, the selector/install-script
drift test): pass.
- `gofmt`, `go vet`, `golangci-lint`: clean.
- Frontend `tsc`, `biome`, and `knip`: clean.

## Review decisions

- **Template scope**: trimmed to infrastructure, a language selector,
and an optional Git clone. The IDE selector was removed; editors are
added via the builder's module step.
- **Placement**: base ordering is presentational, so it lives on the
client base-infra select step rather than in the API.
- **git-clone in the base**: kept for the build-time clone affordance.
The base renders its module source verbatim, so it currently pins the
public registry and does not yet honor a deployment's registry mirror;
that fix is tracked in DOCS-610 and delivered as a stacked follow-up PR
(an in-code note documents the current behavior).
- **Container image**: left hardcoded
(`codercom/enterprise-base:ubuntu`). The quickstart base is the
opinionated path; the Docker base covers custom images. Revisiting a
`container_image` variable is tracked in DOCS-611.

## Review updates (@jeremyruppel)

- **Ordering moved to the client**: deleted the server-side
Quickstart-before-Docker grouping (and its position test); the
base-infra select step now prioritizes bases client-side via a shared
`sortByPriority` helper, following the `ModuleSelectStep` precedent. The
API just returns a stable, name-sorted list.
- **Derive included modules from render**: dropped the
`included_modules` manifest field and the `IncludedModules` struct
field; `BaseIncludedModules` now derives from the rendered base (single
source of truth). Removed `TestBaseIncludedModulesMatchRendered` and
added a direct `TestBaseIncludedModules`.
- **Kept the PR scoped to "add quickstart base"**: reverted
`ExtractAgentResourceName`/`ExtractModuleNames` to the pre-`hclsyntax`
regex extraction and removed the option/preset HCL extractor family; the
selector↔install-script drift test now uses a small regex. The broader
templatebuilder HCL migration and the preset-language subset guard are
tracked in DOCS-735.
- Branch rebased onto `origin/main` (no content changes beyond the
above).

---

Linear:
[DOCS-558](https://linear.app/codercom/issue/DOCS-558/add-coder-quickstart-template-to-template-builder)

> This PR was created with AI assistance (Coder Agents).
2026-08-26 20:29:20 -04:00
Jeremy Ruppel d0982e3cc7 fix(coderd/templatebuilder): prompt for DigitalOcean base variables (#27268)
## Summary

The DigitalOcean template builder base declared Terraform `variable`
blocks for `project_uuid` and `ssh_key_id` that the template builder
never filled. `project_uuid` was required with no default, so the build
broke with no way to supply a value from the wizard (DEVEX-591).

This brings the DigitalOcean base to parity with the GCP bases fixed in
#27015:

- Declare `project_uuid` (required) and `ssh_key_id` (optional, default
`0`) in `base.json` so the wizard prompts for them on the first step.
- Inject the entered values via `default = {{ .Variables.* }}` in
`main.tf.tmpl`, keeping the existing `variable` blocks and validation.
- Drop the `sensitive` flags. The variable-injection path
(`mergeBaseVariables`, `DefaultBaseRenderContext`, and the snapshot test
helper) skips sensitive variables, so a sensitive base variable renders
empty. A project UUID / SSH key ID are not secrets.
- Update the README now that the values are prompted rather than
manually edited.
- Regenerate the `digitalocean-linux.tf.golden` snapshot.

## Testing

- `go test ./coderd/templatebuilder/`

<img width="1048" height="616" alt="Screenshot 2026-07-15 at 12 44
42 PM"
src="https://github.com/user-attachments/assets/89e334ca-e904-4387-9264-6ed1614a40ba"
/>

<details>
<summary>Audit of all template builder bases for unfilled
variables</summary>

| Base | Variable status | Verdict |
|------|-----------------|---------|
| aws-linux | no HCL `variable` blocks; provider env auth | OK |
| aws-windows | same | OK |
| azure-linux | same | OK |
| **digitalocean-linux** | `project_uuid` (required, no default) +
`ssh_key_id`; absent from `base.json` | **Fixed here** |
| docker | `docker_socket` has `default = ""`; `container_image` via `{{
.Variables }}` + declared | OK |
| gcp-linux | fixed in #27015 | OK |
| gcp-windows | fixed in #27015 | OK |
| kubernetes | `namespace` (required), `use_kubeconfig`,
`container_image` all via `{{ .Variables }}` + declared | OK |
| scratch | no variables | OK |

DigitalOcean was the only broken base; all others either have safe
defaults or already declare their variables.

**Mechanism note:** `base.json` `variables[]` drives the first-step
prompts and values are injected as HCL literals via `{{
.Variables.<name> }}` (strings quoted, numbers/bools raw; supported
types: string, number, bool). Sensitive/computed variables are
intentionally skipped everywhere the injection map is built, so they
cannot currently be injected. That is why the `sensitive` flags were
removed here.
</details>

---
*This PR was generated by Coder Agents on behalf of @jeremyruppel.*
2026-07-15 14:05:18 -04:00
Jeremy Ruppel 581f906947 fix(coderd): declare project_id variable in GCP template builder bases (#27015)
## Summary

GCP base templates (`gcp-linux`, `gcp-windows`) in the Template Builder
had a Terraform `variable "project_id"` with no default, but their
`base.json` manifests didn't declare it. The UI never prompted for it,
so the provisioner import always failed with:

```
required template variables need values: project_id
```

## Changes

- Add `project_id` as a required variable in both GCP `base.json`
manifests
- Convert templates from raw Terraform variable blocks to Go template
injection (`{{ .Variables.project_id }}`), matching the existing
kubernetes pattern
- Fix `DefaultBaseRenderContext` to supply a `"REQUIRED"` placeholder
for required variables without defaults (previously rendered as `<no
value>`)
- Replace duplicate test subtests with proper GCP coverage including a
missing-variable error case

<details>
<summary>Implementation plan</summary>

### Root cause

The GCP base templates contained `variable "project_id" {}` (no default
= required) in their `.tf.tmpl` files, but the `base.json` manifests had
an empty `variables` array. The Template Builder UI
(`BaseTemplateParametersStep`) is data-driven from `base.json`, so it
never showed a field for `project_id`. The composed Terraform output
still contained the required variable, causing the provisioner import to
fail.

### Fix approach

Follow the pattern established by the kubernetes base template:
1. Declare variables in `base.json` so the UI prompts for them
2. Use Go template syntax (`{{ .Variables.project_id }}`) to inject
values at compose time
3. Remove raw Terraform `variable` blocks from the template since the
value is now baked in

### Files changed

| File | Change |
|---|---|
| `bases/gcp-linux/base.json` | Added `project_id` as a required
variable |
| `bases/gcp-windows/base.json` | Added `project_id` as a required
variable |
| `bases/gcp-linux/main.tf.tmpl` | Removed Terraform variable block, use
Go template injection |
| `bases/gcp-windows/main.tf.tmpl` | Same |
| `bases.go` | `DefaultBaseRenderContext` supplies placeholder for
required vars without defaults |
| `compose_test.go` | Replaced duplicate subtests with proper GCP tests
|
| `templatebuilder_handler_test.go` | Updated `gcp-windows` spec to
expect `project_id` variable |
| Golden files | Regenerated |

</details>

> 🤖 Generated by Coder Agents on behalf of @jeremyruppel
2026-07-06 16:48:00 -04:00
Ben PotterandJeremy Ruppel 7b19ec3933 feat: improve the image management experience with template builder (#27018)
Makes it easier to pick the right workspace image, both in the template
builder and in the docs.

- Template builder: the Docker and Kubernetes bases now expose a
`container_image` variable in the wizard (freeform text, defaults to
`codercom/example-base:ubuntu`), and their prerequisites explain why
image choice matters, with tradeoffs between
`codercom/example-base:ubuntu` (minimal) and
`codercom/example-universal:ubuntu` (catch-all), plus pointers to
[coder/images](https://github.com/coder/images) and the image management
docs.
- Docs: reworked [image
management](https://coder.com/docs/@ben%2Fdevrel-201-image-guidance-prereqs/admin/templates/managing-templates/image-management)
into a clearer maturity ladder (minimal → golden → project-specific →
developer customization), with pullable image references in every
example, `codercom/oss-dogfood` as a project-specific example, and Dev
Containers + [mise](https://mise.jdx.dev/) as ways to customize without
new images.

Companion PR for the starter templates: coder/registry#943

Part of DEVREL-201.

🤖 Generated with Coder Agents using Claude, on behalf of @bpmct (wizard
variable by @jeremyruppel in #27024)

---------

Co-authored-by: Jeremy Ruppel <jeremyruppel@users.noreply.github.com>
2026-07-06 20:37:34 +00:00
Jeremy Ruppel c40b3b9bcb feat: add base templates for all major cloud providers (#26634)
Add 6 new base templates to the template builder, covering all major
cloud providers and platforms:

- **scratch**: Minimal starter template with only `coder_agent` and
metadata
- **aws-windows**: AWS EC2 Windows instances with PowerShell user_data
- **azure-linux**: Azure VMs with cloud-init and managed disk
persistence
- **gcp-linux**: Google Compute Engine Linux instances with persistent
disk
- **gcp-windows**: Google Compute Engine Windows instances
- **digitalocean-linux**: DigitalOcean Linux droplets with persistent
volumes

Each base includes `base.json`, `main.tf.tmpl`, `README.md` (with
prerequisite markers), and any static files (cloud-init configs). Tests
verify all 9 bases load, render without error, and produce valid
single-agent declarations.

`azure-windows` is deferred; it needs to be registered in the `examples`
package first.

Depends on #26633

> [!NOTE]
> This PR was authored by Coder Agents on behalf of @jeremyruppel.

---

NB: This is very much an agent-generated PR and draws completely from
base templates that exist in `examples/templates/`. The base.json files
are new, so review those, but don't spend any brain tokens on the
correctness of the terraform and supporting files: any issues there are
issues with the upstream example template
2026-06-30 18:26:31 -04:00
Jeremy Ruppel 7ea5d48296 fix: render base variables into templates instead of tfvars (DEVEX-287) (#26436)
Part of the Template Builder wizard PR stack.

## Problem

The kubernetes base template used Terraform `variable` blocks and
`var.*` references for `use_kubeconfig` and `namespace`, but the
composed tar bundle never included a `.tfvars` file. This caused
`terraform plan` to fail with "required template variables need values:
namespace".

## Fix

Base templates now use Go template variables (`{{ .Variables.* }}`) just
like module templates do. Values are validated, HCL-quoted, and rendered
directly into the output HCL.

Also adds explicit "variable is required" validation to both
`mergeBaseVariables` and `mergeModuleVariables`, replacing the previous
reliance on `missingkey=error` at render time for clearer error
messages.

---
> [!NOTE]
> Generated by Coder Agents on behalf of @jeremyruppel
2026-06-23 11:26:38 -04:00
Jeremy Ruppel 71957a2bc0 feat(coderd/templatebuilder): add exampleID->OS map and base template .tf.tmpl files (#26115)
Add the bundled `exampleID -> OS` Go map for Docker, Kubernetes, and AWS
EC2 Linux base templates. Create `.tf.tmpl` Go template files for each
within `coderd/templatebuilder/bases/`, along with `BaseRenderContext`
and `RenderBaseTemplate` rendering helpers.

The `.tf.tmpl` files are independent copies of the example templates
with module blocks (code-server, jetbrains) removed, since the template
builder composes modules separately into `modules.tf`. When
`ImageOptions` is provided, the container image field references the
Terraform parameter; otherwise it uses the hardcoded value via Go
template whitespace control (`{{-`).

Golden file snapshot tests verify rendered output stability with an
`-update` flag for regeneration.

Depends on #25909

> [!NOTE]
> This PR was authored by Coder Agents on behalf of @jeremyruppel.
2026-06-12 17:12:17 -04:00