mirror of
https://github.com/coder/coder.git
synced 2026-11-22 03:18:31 +08:00
25af21b5d5d86ab7d9253e1e72d33b2da170e4f8
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
a93b2258cf |
refactor(coderd/templatebuilder): parse rendered HCL with hclparse and support multiple agents (#28787)
Part of [DEVEX-556](https://linear.app/codercom/issue/DEVEX-556). ## Summary Replaces the Template Builder's regex-based extraction of information from base-template Terraform with an `hclsyntax` parse of the rendered output, and removes the single-`coder_agent` assumption from the compose core. ## What changed - `render.go`: parse rendered Terraform via `hclsyntax.ParseConfig` instead of regex. - Added `ExtractedAgent{Name, Reference}` and `ExtractAgentResourceNames`, which enumerates every `coder_agent` in declaration order and reads `count`/`for_each` from each block body. This is robust to nested blocks (`env {}`, `metadata {}`), fixing a case the old `[^}]*` regex missed. - Kept singular `ExtractAgentResourceName` as a compatibility wrapper that still errors unless exactly one agent exists. - `ExtractModuleNames` now walks parsed `module` blocks rather than line-anchored regex. - `compose.go`: `Compose` wires modules to the base's first/default agent instead of erroring when a base declares multiple agents. Single-agent output is byte-identical. - `render_test.go`: added `TestExtractAgentResourceNames` (multiple-in-order, `count` after a nested block, no agents). ## Scope Parsing/compose layer only. Per-module agent **selection** (SDK/UI) is intentionally out of scope and left as follow-up. ## Testing - `go test ./coderd/templatebuilder/` passes (golden snapshots unchanged). - `golangci-lint run ./coderd/templatebuilder/` reports 0 issues. - `go build ./coderd/...` succeeds. No new dependencies (`hashicorp/hcl/v2` already vendored). <details> <summary>Design context</summary> - The `.tf.tmpl` files are Go templates and only become valid HCL after rendering, so parsing happens on rendered output (which the extractors already operated on). - `base.json` stays authoritative; no manifest fields are inferred or removed here. A survey found `os` is mostly inferable from `coder_agent.os` (except `scratch`, which uses a dynamic `data.coder_provisioner.me.os`), and that `variables` cannot be safely inferred because base variables are Go-template inputs, not Terraform `variable` blocks. These were left out to keep the change minimal. - Multiple agents are supported by wiring modules to the first agent in document order; single-agent bases are unaffected because first == only. </details> --- 🤖 Generated with Coder Agents on behalf of @jeremyruppel. --------- Co-authored-by: Coder Agents <agents@coder.com> |
||
|
|
c73f34c35d |
feat: honor the deployment registry mirror in template builder bases (#28727)
## Summary Threads the deployment's configured module registry (`CODER_TEMPLATE_BUILDER_REGISTRY_URL`) into template-builder **base** rendering, so a base that embeds a catalog module (e.g. the quickstart base's `git-clone`) resolves its `source` through a registry mirror the same way wizard-composed modules already do. Previously base-embedded module sources hardcoded `registry.coder.com`, so on a mirror/private-registry deployment a base-embedded module reached the public registry while wizard modules used the mirror. Implements [DOCS-610](https://linear.app/codercom/issue/DOCS-610). ## What changed * `BaseRenderContext.RegistryBase` (new field), mirroring `ModuleRenderContext.RegistryBase`. `Compose` threads the deployment registry through `renderBase` into the base render context, and the four hardcoded base module sources (`quickstart` git-clone, `azure-linux`/`gcp-linux`/`gcp-windows` region modules) now use `{{ .RegistryBase }}`. Because the value defaults back to `registry.coder.com`, rendered sources stay byte-identical; the only golden change is the removal of the now-stale quickstart "follow-up" note. * **Boot-time validation.** `codersdk.ValidateTemplateBuilderRegistryURL` runs in `DeploymentValues.Validate()`, gated on the builder being enabled, matching the chat-hook precedent. A malformed value fails at server start naming the option, instead of surfacing as a per-request error to a template author who cannot fix it. It uses `net/url` in authority form to reject a scheme, path, query, fragment, trailing slash, or credentials and accept a bare `host[:port]`; it does not canonicalize the host. * `Compose` **trusts the validated value**, defaulting an unset one and defensively rejecting a malformed one for direct callers that bypass boot. ## Tests * `codersdk`: validator accept/reject matrix (incl. credentials-without-echo) and a `DeploymentValues.Validate` boundary test with a disabled-skips-validation case. * `templatebuilder`: a base honors the mirror and falls back to the default when unset, plus a defensive-reject case; existing compose tests updated to bare-host registry values. <details><summary>Design decision log</summary> This PR is a deliberately minimal reimplementation of the [DOCS-610](https://linear.app/codercom/issue/DOCS-610/template-builder-base-embedded-modules-should-honor-the-deployment) scope, started fresh from `main` rather than carrying forward an earlier, larger draft (coder/coder#28480). Key decisions: * **Scope kept to the ticket.** The ticket's fix is three bullets: add a registry field to `BaseRenderContext`, thread the deployment `RegistryURL` through `Compose` -> `renderBase`, and use it in base module sources. Independent hardening explored in the earlier draft was dropped as out of scope: `terraform-svchost`-based IDN/punycode/port canonicalization, a rendered-HCL backstop with a 500 mapping, and HCL module-source extraction helpers. Telemetry misclassification of mirror modules is tracked separately ([DOCS-712](https://linear.app/codercom/issue/DOCS-712/telemetry-misclassifies-registry-mirror-modules-as-non-coder)) and is not addressed here. * **Default lives in the render layer.** `templatebuilder.DefaultRegistryBase` holds the fallback host; `codersdk` keeps its own option-default literal. An earlier version exported the constant from `codersdk`, which only emitted a dead `typesGenerated.ts` entry that nothing in the frontend consumed, so it was moved out. The `codersdk` validator is a function, so it generates no TS type. * **Validate at the config boundary, fail fast.** The repo convention is to hard-fail structurally invalid config in `DeploymentValues.Validate()` (access URL, OAuth lifetimes, chat hook URL) and to warn only for functional-but-risky settings. A malformed registry host is structurally invalid, so it fails at boot. This also puts the error in front of the admin who set it, rather than a template author hitting a per-request error they cannot fix. * `net/url`**, not a regex.** The codebase validates identifiers/names with regexes but host/URL-shaped config with `url.Parse` (chat hook URL, OAuth redirect URIs, AI provider base URL). This value is host-shaped, so it follows that convention. `url.Parse("//" + v)` with `u.Host == v` rejects a scheme/path/query/fragment/trailing-slash in one check. </details> > This PR was created with AI assistance (Coder Agents). |
||
|
|
c78dca5141 |
feat: add Coder Quickstart base to the template builder (#27247)
## Summary Adds the **Coder Quickstart** template as a selectable base in the [template builder](coderd/templatebuilder) guided wizard. It is a Docker-based starter that lets a user pick languages and optionally clone a repo. ## What changed - **New base package** `coderd/templatebuilder/bases/quickstart/`: - `base.json`: `id: "quickstart"`, `display_name: "Coder Quickstart"`, `os: "linux"`. - `main.tf.tmpl`: a Docker workspace with a language selector, a single language-install script, an optional Git clone, and workspace presets. Editors are added via the builder's module step rather than baked into the base. - `install-languages.sh.tftpl` and `README.md` (with prerequisites markers so the builder can extract the prerequisites section). - **Placement**: the bases endpoint returns a plain list sorted by display name (with ID as a deterministic tiebreak). The wizard's base-infra select step prioritizes the Quickstart and Docker starters to the front on the client, mirroring the existing client-side module prioritization in `ModuleSelectStep` (shared `sortByPriority` helper). - **Base/module collision guard**: a base's bundled catalog modules are derived from its rendered Terraform (`ExtractModuleNames`, cached with `sync.OnceValues` and filtered to catalog IDs). `validateModules` seeds its seen-set from that derived set, so a wizard-selected module the base already renders is rejected with a clear error, and the modules endpoint omits it so the wizard never offers a colliding module. The base's Terraform is the single source of truth, so there is no separate manifest list that can drift. ## Testing - `go test ./coderd/` (template builder handler: bases ordering, the `baseSpec` table, the modules-endpoint base filter): pass. - `go test ./coderd/templatebuilder/` (all-bases render/snapshot, the collision guard, `TestBaseIncludedModules`, the selector/install-script drift test): pass. - `gofmt`, `go vet`, `golangci-lint`: clean. - Frontend `tsc`, `biome`, and `knip`: clean. ## Review decisions - **Template scope**: trimmed to infrastructure, a language selector, and an optional Git clone. The IDE selector was removed; editors are added via the builder's module step. - **Placement**: base ordering is presentational, so it lives on the client base-infra select step rather than in the API. - **git-clone in the base**: kept for the build-time clone affordance. The base renders its module source verbatim, so it currently pins the public registry and does not yet honor a deployment's registry mirror; that fix is tracked in DOCS-610 and delivered as a stacked follow-up PR (an in-code note documents the current behavior). - **Container image**: left hardcoded (`codercom/enterprise-base:ubuntu`). The quickstart base is the opinionated path; the Docker base covers custom images. Revisiting a `container_image` variable is tracked in DOCS-611. ## Review updates (@jeremyruppel) - **Ordering moved to the client**: deleted the server-side Quickstart-before-Docker grouping (and its position test); the base-infra select step now prioritizes bases client-side via a shared `sortByPriority` helper, following the `ModuleSelectStep` precedent. The API just returns a stable, name-sorted list. - **Derive included modules from render**: dropped the `included_modules` manifest field and the `IncludedModules` struct field; `BaseIncludedModules` now derives from the rendered base (single source of truth). Removed `TestBaseIncludedModulesMatchRendered` and added a direct `TestBaseIncludedModules`. - **Kept the PR scoped to "add quickstart base"**: reverted `ExtractAgentResourceName`/`ExtractModuleNames` to the pre-`hclsyntax` regex extraction and removed the option/preset HCL extractor family; the selector↔install-script drift test now uses a small regex. The broader templatebuilder HCL migration and the preset-language subset guard are tracked in DOCS-735. - Branch rebased onto `origin/main` (no content changes beyond the above). --- Linear: [DOCS-558](https://linear.app/codercom/issue/DOCS-558/add-coder-quickstart-template-to-template-builder) > This PR was created with AI assistance (Coder Agents). |
||
|
|
581f906947 |
fix(coderd): declare project_id variable in GCP template builder bases (#27015)
## Summary
GCP base templates (`gcp-linux`, `gcp-windows`) in the Template Builder
had a Terraform `variable "project_id"` with no default, but their
`base.json` manifests didn't declare it. The UI never prompted for it,
so the provisioner import always failed with:
```
required template variables need values: project_id
```
## Changes
- Add `project_id` as a required variable in both GCP `base.json`
manifests
- Convert templates from raw Terraform variable blocks to Go template
injection (`{{ .Variables.project_id }}`), matching the existing
kubernetes pattern
- Fix `DefaultBaseRenderContext` to supply a `"REQUIRED"` placeholder
for required variables without defaults (previously rendered as `<no
value>`)
- Replace duplicate test subtests with proper GCP coverage including a
missing-variable error case
<details>
<summary>Implementation plan</summary>
### Root cause
The GCP base templates contained `variable "project_id" {}` (no default
= required) in their `.tf.tmpl` files, but the `base.json` manifests had
an empty `variables` array. The Template Builder UI
(`BaseTemplateParametersStep`) is data-driven from `base.json`, so it
never showed a field for `project_id`. The composed Terraform output
still contained the required variable, causing the provisioner import to
fail.
### Fix approach
Follow the pattern established by the kubernetes base template:
1. Declare variables in `base.json` so the UI prompts for them
2. Use Go template syntax (`{{ .Variables.project_id }}`) to inject
values at compose time
3. Remove raw Terraform `variable` blocks from the template since the
value is now baked in
### Files changed
| File | Change |
|---|---|
| `bases/gcp-linux/base.json` | Added `project_id` as a required
variable |
| `bases/gcp-windows/base.json` | Added `project_id` as a required
variable |
| `bases/gcp-linux/main.tf.tmpl` | Removed Terraform variable block, use
Go template injection |
| `bases/gcp-windows/main.tf.tmpl` | Same |
| `bases.go` | `DefaultBaseRenderContext` supplies placeholder for
required vars without defaults |
| `compose_test.go` | Replaced duplicate subtests with proper GCP tests
|
| `templatebuilder_handler_test.go` | Updated `gcp-windows` spec to
expect `project_id` variable |
| Golden files | Regenerated |
</details>
> 🤖 Generated by Coder Agents on behalf of @jeremyruppel
|
||
|
|
3d966d48b5 |
fix(coderd/templatebuilder): fix archive bundling for nested static files and counted agents (#26901)
Fixes two template builder bugs that caused AWS EC2 (Linux) template imports to fail: 1. **Missing directory entries in tar archive**: `BundleTar` wrote static files with nested paths (e.g. `cloud-init/cloud-config.yaml.tftpl`) without emitting `TypeDir` entries for parent directories. The provisioner's archive extractor requires explicit directory entries and failed with "no such file or directory". 2. **Incorrect agent reference for counted resources**: `ExtractAgentResourceName` returned `dev` for the AWS Linux base template, but the agent uses `count = data.coder_workspace.me.start_count`, so module templates need `coder_agent.dev[0].id`. The function now detects `count`/`for_each` and appends `[0]`. > [!NOTE] > Generated by Coder Agents (on behalf of @jeremyruppel) |
||
|
|
c40b3b9bcb |
feat: add base templates for all major cloud providers (#26634)
Add 6 new base templates to the template builder, covering all major cloud providers and platforms: - **scratch**: Minimal starter template with only `coder_agent` and metadata - **aws-windows**: AWS EC2 Windows instances with PowerShell user_data - **azure-linux**: Azure VMs with cloud-init and managed disk persistence - **gcp-linux**: Google Compute Engine Linux instances with persistent disk - **gcp-windows**: Google Compute Engine Windows instances - **digitalocean-linux**: DigitalOcean Linux droplets with persistent volumes Each base includes `base.json`, `main.tf.tmpl`, `README.md` (with prerequisite markers), and any static files (cloud-init configs). Tests verify all 9 bases load, render without error, and produce valid single-agent declarations. `azure-windows` is deferred; it needs to be registered in the `examples` package first. Depends on #26633 > [!NOTE] > This PR was authored by Coder Agents on behalf of @jeremyruppel. --- NB: This is very much an agent-generated PR and draws completely from base templates that exist in `examples/templates/`. The base.json files are new, so review those, but don't spend any brain tokens on the correctness of the terraform and supporting files: any issues there are issues with the upstream example template |
||
|
|
7ea5d48296 |
fix: render base variables into templates instead of tfvars (DEVEX-287) (#26436)
Part of the Template Builder wizard PR stack.
## Problem
The kubernetes base template used Terraform `variable` blocks and
`var.*` references for `use_kubeconfig` and `namespace`, but the
composed tar bundle never included a `.tfvars` file. This caused
`terraform plan` to fail with "required template variables need values:
namespace".
## Fix
Base templates now use Go template variables (`{{ .Variables.* }}`) just
like module templates do. Values are validated, HCL-quoted, and rendered
directly into the output HCL.
Also adds explicit "variable is required" validation to both
`mergeBaseVariables` and `mergeModuleVariables`, replacing the previous
reliance on `missingkey=error` at render time for clearer error
messages.
---
> [!NOTE]
> Generated by Coder Agents on behalf of @jeremyruppel
|
||
|
|
6b890116aa |
feat(coderd/templatebuilder): add module rendering and agent name extraction (#26347)
> [!NOTE] > This PR was authored by Coder Agents on behalf of @jeremyruppel. Part 1 of DEVEX-277 (POST /api/v2/templatebuilder/compose). Adds module rendering support and agent resource name extraction to the template builder, preparing for the compose endpoint. - `ModuleRenderContext` and `RenderModuleTemplate` for rendering module `.tf.tmpl` files with registry URL, pinned version, agent resource name, and variable values. Nil-guards the Variables map to prevent panics. - Extract shared `renderTemplate` with `missingkey=error` so missing variable keys fail loudly instead of producing `<no value>` in rendered HCL. - `ExtractAgentResourceName` uses a regex to find the `coder_agent` resource name from rendered base HCL. Errors unless exactly one agent is found. - `ModuleTemplateFS` exposes module template files from the embedded catalog, with validation that the expected `.tf.tmpl` file exists (`fs.Sub` on `embed.FS` silently succeeds for nonexistent paths). |
||
|
|
71957a2bc0 |
feat(coderd/templatebuilder): add exampleID->OS map and base template .tf.tmpl files (#26115)
Add the bundled `exampleID -> OS` Go map for Docker, Kubernetes, and AWS
EC2 Linux base templates. Create `.tf.tmpl` Go template files for each
within `coderd/templatebuilder/bases/`, along with `BaseRenderContext`
and `RenderBaseTemplate` rendering helpers.
The `.tf.tmpl` files are independent copies of the example templates
with module blocks (code-server, jetbrains) removed, since the template
builder composes modules separately into `modules.tf`. When
`ImageOptions` is provided, the container image field references the
Terraform parameter; otherwise it uses the hardcoded value via Go
template whitespace control (`{{-`).
Golden file snapshot tests verify rendered output stability with an
`-update` flag for regeneration.
Depends on #25909
> [!NOTE]
> This PR was authored by Coder Agents on behalf of @jeremyruppel.
|