## Summary Addresses 106 of the 113 conflicts between the docs and the code that the November audit of `main` found: 102 are fixed and 4 have an interim docs fix. Every finding was re-checked against the code before it was fixed. Most fixes are docs-only. Three are small code fixes where the docs described the intended behavior and the code was wrong, each in its own commits so it can be split out. No change in this PR alters template build or agent behavior. ## Changes Fixes by area (finding counts): templates 28, install 13, integrations and monitoring 13, auth and security 12, networking 11, Coder Agents 9, AI governance 7, reference 7, workspaces 6. Code fixes. Each needs a look from the code owner. If an owner prefers not to take one, drop its commits and I will change the docs to describe the current behavior instead. | Commits | Change | Risk to check | |---------|--------|---------------| | `fix: keep built-in listener defaults in packaged coder.env`, and three follow-up commits | `coder.env` comments out `CODER_HTTP_ADDRESS=` and the three `CODER_TLS_*` lines, so an untouched package install listens on `127.0.0.1:3000` as the docs say. Only the `CODER_HTTP_ADDRESS=` line changed behavior. The TLS options default to empty, so commenting them out is for consistency. The Lima examples rewrite the commented line and fail if the line is not found. A test compares every option that the file sets against the built-in defaults. | The deb and RPM packages were not built or installed. Every released package through v2.39.0 ships the empty line, and upgrades keep an existing file (`config\|noreplace`), so the upgrade pages now say how to fix it. | | `fix(cli/clilog): accept log sinks that are all disabled`, and three follow-up commits | `--log-human=/dev/null` with no other log location no longer fails with "no loggers provided". It is the documented way to disable logging. An empty location still fails with that error when no other sink is set, as before. | Behavior change: with `/dev/null`, the server and the provisioner daemon can start with no log output. | | `fix(cli): detect CODER_PROC_PRIO_MGMT by presence`, and a comment follow-up | Log line only. With the variable set to an empty value, the agent now logs that process priority management is enabled, which is what `agentexec` already does. No detection or behavior change. | Low. | Two option descriptions in `codersdk/deployment.go` are corrected, with the CLI reference, configuration reference, and golden files regenerated: the default quiet hours schedule (the max deadline is the quiet hours window on the next day that the template's autostop requirement applies) and `--experiments` (`*` opts in to the experiments marked safe, also in `enterprise/cli/provisionerdaemonstart.go`). ### Code fixes that were reverted after review Two code fixes from the first version of this PR are reverted in the branch. The net diff has no change under `agent/` or `coderd/`. The docs now describe what the code does, and both issues stay open for the owner. - **`error` merge strategy for `coder_env` (DOCS-1669).** The fix made `merge_strategy = "error"` fail whenever a name was set twice, whatever the resource order. Existing template versions that pair an `error` entry with a later `replace` or `append` entry would start to fail every build after `terraform apply`. That is a breaking change that needs its own PR and release note. The fix and its tests are in `dff8e7c6ce` and can be cherry-picked. `environment-variables.md` now says the check runs only when the `error` resource is processed. - **Nice value before the capability drop (DOCS-1687).** The fix let `coder agent-exec` apply a negative nice value by calling `setpriority` before it drops effective capabilities. The current order is deliberate: with the fix, any local process could run the agent binary with `--coder-nice=-20` when the binary has the `CAP_SYS_NICE` file capability. `process-priority.md` now says a nice value below the agent's own is not applied, and no longer tells admins to grant `CAP_SYS_NICE`. ## Not included Seven findings are left out. - DOCS-1629: covered by #31639. - DOCS-1703 and DOCS-1724: covered by #31980. - DOCS-1584, DOCS-1683, DOCS-1685, DOCS-1686: each needs an owner decision, and no neutral docs wording exists until that decision is made. The validation comment on each issue names the decision. Four fixes are interim, so their issues stay open: - DOCS-1600: the AI Gateway pages now agree that the legacy metric names are deprecated and "will be removed in a future release". The code still emits them, and its TODO names v2.37, which has shipped. The owner still needs to set the removal release. - DOCS-1689: the page now says **New Template** opens the builder only for users with the Owner role, because the page gates on read access to the deployment config. Whether Template Admins should get the builder is still open. - DOCS-1669 and DOCS-1687: docs-only for now. Refer to [Code fixes that were reverted after review](#code-fixes-that-were-reverted-after-review). ## Review notes - Claims that describe product intent and need an owner's yes: the Premium qualifier on per-organization workspace sharing (the API route has no license check, the dashboard needs multiple organizations), `organization` port sharing in OSS, the telemetry field list, and the `1-5` cron range in the prebuilds schedule example. - Not run end to end: the PostgreSQL TLS procedure on a cluster, the OpenShift and air-gapped examples, the BSD `date` and `tar` forms on macOS, the Windows command on the multiple Kubernetes clusters page, and the keep-alive script in `docs/reference/index.md` (only its comparison logic was run, against sample values). Nothing was checked on a live deployment. - Likely merge conflicts with open PRs: #31990 (`mcp-server.md`, `feature-stages.md`, `docker-in-workspaces.md`), #31982 (`workspace-management.md`), #31980 (`health-check.md`), #31977 (`manage-provisioner-jobs.md`), #31869 (`airgap.md`), #30486 (`add-envbuilder.md`), and #31326 and #31531 (`codersdk/deployment.go` and generated files). ## Testing - `make pre-commit` (gen, fmt, lint, slim build), before the PR opened and again after the review changes. - `go test` for `./cli/clilog/...`, `./codersdk/`, `./cli/` (`TestServer`, `TestAgent`, and golden files), and `./enterprise/cli/` (provisioner, server, and help golden files). `./agent/agentexec/...` and `./coderd/provisionerdserver/...` pass after the reverts. Linux only. CI covers macOS and Windows. - The `coder.env` test fails against the file on `main` and passes against the new one. It reads the file with `godotenv`, which accepts CRLF line endings. - An empty `CODER_HTTP_ADDRESS=` was confirmed end to end with a full build of the server (output below). - The new `sed` and `grep` lines in the Lima examples were run with GNU tools against the old and the new form of `coder.env`. The Lima VMs were not started. - `--log-human=/dev/null` and `CODER_LOGGING_HUMAN=` were confirmed with the same build (output below). - `coder agent-exec --coder-nice=-5 -- echo` from a full build, as a non-root user: `coder-agent: failed to adjust niceness to -5 ...: permission denied` on stderr, the command output on stdout, exit code 0. - `postgres-ssl.md` rendered with the GitHub Markdown API: two ordered lists with three and five items, where `main` has six lists. - markdownlint, the table formatter, `make lint/emdash`, `make lint/docs-html`, `make lint/docs-manifest`, `make lint/docs-redirects`, `make lint/style-claims`, and Vale on the changed lines. - A doc-check self-review of the docs diff by four independent reviewers. It found six problems, which are fixed. The accuracy and coverage lenses ran on every area. The language and structure lenses ran in full on only part of the diff. <details> <summary>Test and terminal output for the code fixes (Linux)</summary> ```console $ go test ./codersdk/ -run TestPackagedCoderEnvKeepsDefaults -count=1 -v --- PASS: TestPackagedCoderEnvKeepsDefaults (0.00s) ok github.com/coder/coder/v2/codersdk 0.071s $ go test ./cli/clilog/ -run TestBuilder -count=1 -v --- PASS: TestBuilder (0.00s) --- PASS: TestBuilder/NoSinks (0.00s) --- PASS: TestBuilder/NoSinks/JSONDevNull (0.00s) --- PASS: TestBuilder/NoSinks/StackdriverDevNull (0.00s) --- PASS: TestBuilder/NoSinks/NoConfiguration (0.00s) --- PASS: TestBuilder/NoSinks/HumanDevNull (0.00s) --- PASS: TestBuilder/WithHuman (0.00s) --- PASS: TestBuilder/WithFilter (0.00s) --- PASS: TestBuilder/WithJSON (0.00s) --- PASS: TestBuilder/Verbose (0.00s) --- PASS: TestBuilder/FromDeploymentValues (0.01s) --- PASS: TestBuilder/FromDeploymentValues/Defaults (0.00s) --- PASS: TestBuilder/FromDeploymentValues/EmptyHumanEnv (0.00s) --- PASS: TestBuilder/FromDeploymentValues/Override (0.00s) ok github.com/coder/coder/v2/cli/clilog 0.171s $ TMPDIR=$HOME/.cache/agentexec-tmp go test ./agent/agentexec/... ./coderd/provisionerdserver/... -count=1 ok github.com/coder/coder/v2/agent/agentexec 4.236s ok github.com/coder/coder/v2/coderd/provisionerdserver 2.404s # Full build of the server (go build ./cmd/coder). An empty value stops it before it connects to the database. $ CODER_HTTP_ADDRESS= coder server --postgres-url 'postgres://127.0.0.1:1/x?sslmode=disable' Encountered an error running "coder server", see "coder server --help" for more information error: TLS is disabled. Enable with --tls-enable or specify a HTTP address # With the variable unset, the server passes that check and goes on to the database. $ coder server --postgres-url 'postgres://127.0.0.1:1/x?sslmode=disable' 2026-11-13 02:19:41.901 [warn] ping postgres: retrying error="dial tcp 127.0.0.1:1: connect: connection refused" try=11 Encountered an error running "coder server", see "coder server --help" for more information error: connect to postgres: connect to postgres: unable to connect after 11 tries; last error: dial tcp 127.0.0.1:1: connect: connection refused # An empty log location with no other sink still stops the server. $ CODER_LOGGING_HUMAN= coder server --postgres-url 'postgres://127.0.0.1:1/x?sslmode=disable' Encountered an error running "coder server", see "coder server --help" for more information error: make logger: no loggers provided, use /dev/null to disable logging # /dev/null passes the logger check. The server prints no log lines and goes on to the database. $ coder server --log-human=/dev/null --postgres-url 'postgres://127.0.0.1:1/x?sslmode=disable' Encountered an error running "coder server", see "coder server --help" for more information error: connect to postgres: connect to postgres: unable to connect after 11 tries; last error: dial tcp 127.0.0.1:1: connect: connection refused ``` </details> ## Linear Fixes DOCS-1583, DOCS-1585, DOCS-1586, DOCS-1587, DOCS-1588, DOCS-1589, DOCS-1590, DOCS-1591, DOCS-1592, DOCS-1593, DOCS-1594, DOCS-1595, DOCS-1596, DOCS-1597, DOCS-1598, DOCS-1599, DOCS-1601, DOCS-1602, DOCS-1603, DOCS-1604, DOCS-1605, DOCS-1606, DOCS-1607, DOCS-1608, DOCS-1609, DOCS-1614, DOCS-1618, DOCS-1619, DOCS-1620, DOCS-1624, DOCS-1635, DOCS-1639, DOCS-1644, DOCS-1651, DOCS-1652, DOCS-1653, DOCS-1654, DOCS-1655, DOCS-1656, DOCS-1657, DOCS-1658, DOCS-1659, DOCS-1660, DOCS-1661, DOCS-1662, DOCS-1663, DOCS-1664, DOCS-1665, DOCS-1666, DOCS-1667, DOCS-1668, DOCS-1670, DOCS-1671, DOCS-1672, DOCS-1673, DOCS-1674, DOCS-1675, DOCS-1676, DOCS-1677, DOCS-1678, DOCS-1679, DOCS-1680, DOCS-1681, DOCS-1682, DOCS-1684, DOCS-1688, DOCS-1690, DOCS-1691, DOCS-1692, DOCS-1693, DOCS-1694, DOCS-1695, DOCS-1696, DOCS-1697, DOCS-1698, DOCS-1699, DOCS-1700, DOCS-1701, DOCS-1702, DOCS-1704, DOCS-1705, DOCS-1706, DOCS-1707, DOCS-1708, DOCS-1709, DOCS-1710, DOCS-1711, DOCS-1712, DOCS-1713, DOCS-1714, DOCS-1715, DOCS-1716, DOCS-1717, DOCS-1718, DOCS-1719, DOCS-1720, DOCS-1721, DOCS-1722, DOCS-1723, DOCS-1725, DOCS-1726, DOCS-1727 Part of DOCS-1600, DOCS-1689, DOCS-1669, DOCS-1687 <details> <summary>Decision log</summary> ### Scope - One PR for all findings, as agreed with the reviewer. The code fixes are separate commits so an owner can ask for any of them to move to its own PR. Nothing in the PR is a breaking change, so the title has no `!`. - A finding was skipped only when an open PR already covers it, or when every available wording would assert an undecided product behavior. - Each validation comment on the Linear issues records the evidence, the corrections to the original finding, and what was not verified. ### Docs or code - The default was a docs-only fix that describes what the code does. - A code fix was chosen when the docs describe a behavior the code plainly intended and the code change is small and testable: the packaged env file, the log sink check, and the log line for the environment variable presence check. - The `error` merge strategy and the nice value ordering started as code fixes and went back to docs-only after review. The first breaks existing template versions. The second weakens a deliberate security ordering. - The SCIM `userName` finding (DOCS-1609) took the docs path. The handler is unchanged. ### Wording choices worth a second look - `feature-stages.md` says `--experiments=*` enables only the experiments marked safe, and points to the "Get safe experiments" API. It does not list the other experiments, because the list would drift and those experiments are hidden on purpose. The `--experiments` flag help now says the same. - `agent-metadata.md` replaces the writes-per-second formula with a description of the batcher (each server replica writes one statement every 5 seconds, or sooner at 500 entries). - `extending-templates/index.md` no longer calls `startup_script` deprecated. The provider documents it as an alias for a `coder_script`. - `postgres-ssl.md` separates client verification from server verification, so the server verification steps work alone. - `rpm.md`, `upgrade.md`, and the 2.29 to 2.34 ESR upgrade page explain the empty `CODER_HTTP_ADDRESS=` line that older packages shipped, because upgrades keep the old file. The behavior changed in v2.34.0 with the serpent upgrade in #24926. - The keep-alive script in `docs/reference/index.md` reads the current deadline and skips the request when it is already later. The comparison is on text, so it applies only when the API returns a UTC timestamp, which needs the database to use the UTC time zone. Otherwise the script sets the deadline on every pass, as it did before, and the page says so. - `external-auth.md` anchors the new GitHub Enterprise regex example. The older unanchored examples on that page and on `git-providers.md` are not changed here. ### Self-review findings that were rejected - "The builder opens for Template Admins": the Templates page needs a successful deployment config read, which the Template Admin role does not have. - "Organization Admins can set AI model prices": the price write authorizes against a site-level object, which organization-scoped grants do not match. - "The provisioner chart sets no default resources": `helm/libcoder/templates/_coder.yaml` applies 2000m and 4096Mi when `coder.resources` is empty. ### Automated review, round 1 The `coder-agents-review` app left 45 findings on `8853332`. Fixed: the two reverts above, the anchored regex, the `coder.env` follow-ups (Lima examples, upgrade notes, a comment in the file, and a test that covers every option in the file), the clilog change, the two option descriptions, and the docs findings (CORS behavior, token lifetime example, keep-alive script, agent metadata, refresh token errors, `switch` form type, `dbcrypt` skip list, and a stray line in a YAML example). Declined, with the reason on each thread: - Refactoring the `agentexec` logging and renaming its log field: not related to a docs conflict. - Rewording the "process priority management not enabled (linux-only)" log message: the docs quote the message, and the wording predates this PR. - Problems in code and docs that this PR does not change (the `Capabilities` subtest, the Helm values comment, the AWS provider path in `airgap.md`, and others in the review's out-of-scope list). ### Automated review, round 2 The app left 14 new findings on `04925bd`, and accepted the three declined threads on `cli/agent.go`. All 14 are fixed. The main one: the round 1 change that removed the "no loggers provided" error went too far, because an empty log location would then start a server with no logs. The error is back for every case except `/dev/null`. Reported by the app as out of scope and not changed: the swagger annotation for `GET /api/v2/experiments/available` declares an array, but the handler returns an object with a `safe` array. Three Storybook fixtures keep the old `--experiments` description. ### Automated review, round 3 The app confirmed that all round 2 findings are fixed, and left 10 small findings on `f55cb6d` (8 P4, 2 Nit). All are fixed: the TLS condition and the release range in the `coder.env` notes, one place for the `/dev/null` rule in `clilog`, two test cases, a clearer test failure, and the CPU example in `agent-metadata.md`. ### Automated review, rounds 4 and 5 Round 4 left one P4 on `8fffb38` (name the `--trace` option in the logging note), which is fixed. Round 5 on `2ce754e` reports no new findings and says every earlier finding is fixed or closed. The app did not post an approval: it leaves its out-of-scope items and the code fixes to a human reviewer. Two threads on `cli/agent.go` stay open on purpose for the code owner. ### Doc-check review The doc-check workflow left 12 comments on `2ce754e` (10 blocking, 2 suggestions). All are fixed in `c888885`: "refer to" or "visit" in place of "see" for cross-references on the lines this PR added, the **New template** label as the dashboard shows it, one instruction per step in the server verification procedure of `postgres-ssl.md`, and two wording changes in `process-priority.md`. The `CAP_SYS_NICE` sentence keeps a short reason, which differs from the suggested text. The `coder-agents-review` app then reviewed that commit (rounds 6 and 7). Round 6 left two P4 findings, fixed in `30b2c79`: the `failed to adjust niceness` message is described once, in the troubleshooting entry, with where it appears, and the client verification steps in `postgres-ssl.md` render as one numbered list. Round 7 reports no new findings. `2a6035c` then softens one sentence from that fix, because the agent logs the stderr of devcontainer CLI commands. The app was not run again on that commit. ### Adjacent gaps found and not fixed here - `add-envbuilder.md` has the same `zip` command that `template-from-scratch.md` had. - `template-from-scratch.md` leaves the reader in `build/` after `cd $_`, before a step that runs from `template-tour`. </details> --- _Generated by Coder Agents on behalf of @nickvigilante._ --------- Co-authored-by: Ben Potter <me@bpmct.net>
685 lines
23 KiB
Go
685 lines
23 KiB
Go
package cli
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"io"
|
|
"net"
|
|
"net/http"
|
|
"net/http/pprof"
|
|
"net/url"
|
|
"os"
|
|
"os/signal"
|
|
"path/filepath"
|
|
"runtime"
|
|
"slices"
|
|
"strconv"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/google/uuid"
|
|
"github.com/prometheus/client_golang/prometheus"
|
|
"golang.org/x/xerrors"
|
|
"gopkg.in/natefinch/lumberjack.v2"
|
|
|
|
"cdr.dev/slog/v3"
|
|
"cdr.dev/slog/v3/sloggers/sloghuman"
|
|
"cdr.dev/slog/v3/sloggers/slogjson"
|
|
"cdr.dev/slog/v3/sloggers/slogstackdriver"
|
|
"github.com/coder/coder/v2/agent"
|
|
"github.com/coder/coder/v2/agent/agentcontainers"
|
|
"github.com/coder/coder/v2/agent/agentcontextconfig"
|
|
"github.com/coder/coder/v2/agent/agentexec"
|
|
"github.com/coder/coder/v2/agent/agentssh"
|
|
"github.com/coder/coder/v2/agent/boundarylogproxy"
|
|
"github.com/coder/coder/v2/agent/reaper"
|
|
"github.com/coder/coder/v2/buildinfo"
|
|
"github.com/coder/coder/v2/cli/clilog"
|
|
"github.com/coder/coder/v2/codersdk"
|
|
"github.com/coder/coder/v2/codersdk/agentsdk"
|
|
"github.com/coder/serpent"
|
|
)
|
|
|
|
// defaultAgentFlightRecorderSize is the default number of log entries below the
|
|
// current log level that the agent keeps in memory and emits when it logs an
|
|
// error, such as losing its connection to coderd.
|
|
const defaultAgentFlightRecorderSize = 1000
|
|
|
|
func (r *RootCmd) workspaceAgent() *serpent.Command {
|
|
var (
|
|
logDir string
|
|
flightRecorderSize int64
|
|
scriptDataDir string
|
|
pprofAddress string
|
|
noReap bool
|
|
sshMaxTimeout time.Duration
|
|
tailnetListenPort int64
|
|
prometheusAddress string
|
|
debugAddress string
|
|
slogHumanPath string
|
|
slogJSONPath string
|
|
slogStackdriverPath string
|
|
blockFileTransfer bool
|
|
blockReversePortForwarding bool
|
|
blockLocalPortForwarding bool
|
|
agentHeaderCommand string
|
|
agentHeader []string
|
|
devcontainers bool
|
|
devcontainerProjectDiscovery bool
|
|
devcontainerDiscoveryAutostart bool
|
|
socketServerEnabled bool
|
|
socketPath string
|
|
agentFirewallLogProxySocketPath string
|
|
)
|
|
agentAuth := &AgentAuth{}
|
|
cmd := &serpent.Command{
|
|
Use: "agent",
|
|
Short: `Starts the Coder workspace agent.`,
|
|
// This command isn't useful to manually execute.
|
|
Hidden: true,
|
|
Handler: func(inv *serpent.Invocation) error {
|
|
ctx, cancel := context.WithCancelCause(inv.Context())
|
|
defer func() {
|
|
cancel(xerrors.New("agent exited"))
|
|
}()
|
|
|
|
var (
|
|
ignorePorts = map[int]string{}
|
|
isLinux = runtime.GOOS == "linux"
|
|
|
|
sinks = []slog.Sink{}
|
|
logClosers = []func() error{}
|
|
)
|
|
defer func() {
|
|
for _, closer := range logClosers {
|
|
_ = closer()
|
|
}
|
|
}()
|
|
|
|
addSinkIfProvided := func(sinkFn func(io.Writer) slog.Sink, loc string) error {
|
|
switch loc {
|
|
case "":
|
|
// Do nothing.
|
|
|
|
case "/dev/stderr":
|
|
sinks = append(sinks, sinkFn(inv.Stderr))
|
|
|
|
case "/dev/stdout":
|
|
sinks = append(sinks, sinkFn(inv.Stdout))
|
|
|
|
default:
|
|
fi, err := os.OpenFile(loc, os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0o644)
|
|
if err != nil {
|
|
return xerrors.Errorf("open log file %q: %w", loc, err)
|
|
}
|
|
sinks = append(sinks, sinkFn(fi))
|
|
logClosers = append(logClosers, fi.Close)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
if err := addSinkIfProvided(sloghuman.Sink, slogHumanPath); err != nil {
|
|
return xerrors.Errorf("add human sink: %w", err)
|
|
}
|
|
if err := addSinkIfProvided(slogjson.Sink, slogJSONPath); err != nil {
|
|
return xerrors.Errorf("add json sink: %w", err)
|
|
}
|
|
if err := addSinkIfProvided(slogstackdriver.Sink, slogStackdriverPath); err != nil {
|
|
return xerrors.Errorf("add stackdriver sink: %w", err)
|
|
}
|
|
|
|
// Spawn a reaper so that we don't accumulate a ton
|
|
// of zombie processes.
|
|
if reaper.IsInitProcess() && !noReap && isLinux {
|
|
logWriter := &clilog.LumberjackWriteCloseFixer{Writer: &lumberjack.Logger{
|
|
Filename: filepath.Join(logDir, "coder-agent-init.log"),
|
|
MaxSize: 5, // MB
|
|
// Without this, rotated logs will never be deleted.
|
|
MaxBackups: 1,
|
|
}}
|
|
defer logWriter.Close()
|
|
|
|
sinks = append(sinks, sloghuman.Sink(logWriter))
|
|
logger := inv.Logger.AppendSinks(sinks...).Leveled(slog.LevelDebug)
|
|
logger = logger.Named("reaper")
|
|
|
|
logger.Info(ctx, "spawning reaper process")
|
|
// Do not start a reaper on the child process. It's important
|
|
// to do this else we fork bomb ourselves.
|
|
//nolint:gocritic
|
|
args := append(os.Args, "--no-reap")
|
|
exitCode, err := reaper.ForkReap(
|
|
reaper.WithExecArgs(args...),
|
|
reaper.WithCatchSignals(StopSignals...),
|
|
reaper.WithLogger(logger),
|
|
)
|
|
if err != nil {
|
|
logger.Error(ctx, "agent process reaper unable to fork", slog.Error(err))
|
|
return xerrors.Errorf("fork reap: %w", err)
|
|
}
|
|
|
|
logger.Info(ctx, "child process exited, propagating exit code",
|
|
slog.F("exit_code", exitCode),
|
|
)
|
|
return ExitError(exitCode, nil)
|
|
}
|
|
|
|
logWriter := &clilog.LumberjackWriteCloseFixer{Writer: &lumberjack.Logger{
|
|
Filename: filepath.Join(logDir, "coder-agent.log"),
|
|
MaxSize: 5, // MB
|
|
// Keep up to the debug logs response cap across the active log and rotations.
|
|
MaxBackups: 19,
|
|
}}
|
|
defer logWriter.Close()
|
|
|
|
sinks = append(sinks, sloghuman.Sink(logWriter))
|
|
// Run at Info (Debug with -v) and always attach a flight recorder.
|
|
// Below-level (debug) entries are kept in a rolling in-memory history and
|
|
// emitted when an error is logged (see agent.runLoop), so the detail
|
|
// leading up to a failure is available without logging debug during
|
|
// normal operation. At Debug the recorder has nothing to record.
|
|
level := slog.LevelInfo
|
|
if r.verbose {
|
|
level = slog.LevelDebug
|
|
}
|
|
logger := inv.Logger.AppendSinks(sinks...).Leveled(level).FlightRecorder(int(flightRecorderSize))
|
|
|
|
// Handle interrupt signals to allow for graceful shutdown,
|
|
// note that calling stopNotify disables the signal handler
|
|
// and the next interrupt will terminate the program (you
|
|
// probably want cancel instead).
|
|
//
|
|
// Note that we also handle these signals in the
|
|
// process that runs as PID 1, mainly to forward it to the agent child
|
|
// so that it can shutdown gracefully.
|
|
ctx, stopNotify := logSignalNotifyContext(ctx, logger, StopSignals...)
|
|
defer stopNotify()
|
|
|
|
// DumpHandler does signal handling, so we call it after the
|
|
// reaper.
|
|
go DumpHandler(ctx, "agent")
|
|
|
|
version := buildinfo.Version()
|
|
logger.Info(ctx, "agent is starting now",
|
|
slog.F("url", agentAuth.agentURL),
|
|
slog.F("auth", agentAuth.agentAuth),
|
|
slog.F("version", version),
|
|
)
|
|
client, err := agentAuth.CreateClient()
|
|
if err != nil {
|
|
return xerrors.Errorf("create agent client: %w", err)
|
|
}
|
|
client.SDK.SetLogger(logger)
|
|
// Set a reasonable timeout so requests can't hang forever!
|
|
// The timeout needs to be reasonably long, because requests
|
|
// with large payloads can take a bit. e.g. startup scripts
|
|
// may take a while to insert.
|
|
client.SDK.HTTPClient.Timeout = 30 * time.Second
|
|
// Attach header transport so we process --agent-header and
|
|
// --agent-header-command flags
|
|
headerTransport, err := headerTransport(ctx, &agentAuth.agentURL, agentHeader, agentHeaderCommand)
|
|
if err != nil {
|
|
return xerrors.Errorf("configure header transport: %w", err)
|
|
}
|
|
headerTransport.Transport = client.SDK.HTTPClient.Transport
|
|
client.SDK.HTTPClient.Transport = headerTransport
|
|
|
|
// Enable pprof handler
|
|
// This prevents the pprof import from being accidentally deleted.
|
|
_ = pprof.Handler
|
|
if pprofAddress != "" {
|
|
pprofSrvClose := ServeHandler(ctx, logger, nil, pprofAddress, "pprof")
|
|
defer pprofSrvClose()
|
|
|
|
if port, err := extractPort(pprofAddress); err == nil {
|
|
ignorePorts[port] = "pprof"
|
|
}
|
|
} else {
|
|
logger.Debug(ctx, "pprof address is empty, disabling pprof server")
|
|
}
|
|
|
|
executablePath, err := os.Executable()
|
|
if err != nil {
|
|
return xerrors.Errorf("getting os executable: %w", err)
|
|
}
|
|
err = os.Setenv("PATH", fmt.Sprintf("%s%c%s", os.Getenv("PATH"), filepath.ListSeparator, filepath.Dir(executablePath)))
|
|
if err != nil {
|
|
return xerrors.Errorf("add executable to $PATH: %w", err)
|
|
}
|
|
|
|
subsystemsRaw := inv.Environ.Get(agent.EnvAgentSubsystem)
|
|
subsystems := []codersdk.AgentSubsystem{}
|
|
for _, s := range strings.Split(subsystemsRaw, ",") {
|
|
subsystem := codersdk.AgentSubsystem(strings.TrimSpace(s))
|
|
if subsystem == "" {
|
|
continue
|
|
}
|
|
if !subsystem.Valid() {
|
|
return xerrors.Errorf("invalid subsystem %q", subsystem)
|
|
}
|
|
subsystems = append(subsystems, subsystem)
|
|
}
|
|
|
|
environmentVariables := map[string]string{
|
|
"GIT_ASKPASS": executablePath,
|
|
}
|
|
|
|
// Presence enables it, even with an empty value. Must match agentexec.NewExecer.
|
|
value, enabled := os.LookupEnv(agentexec.EnvProcPrioMgmt)
|
|
if enabled && runtime.GOOS == "linux" {
|
|
logger.Info(ctx, "process priority management enabled",
|
|
slog.F("env_var", agentexec.EnvProcPrioMgmt),
|
|
slog.F("enabled", value),
|
|
slog.F("os", runtime.GOOS),
|
|
)
|
|
} else {
|
|
logger.Info(ctx, "process priority management not enabled (linux-only)",
|
|
slog.F("env_var", agentexec.EnvProcPrioMgmt),
|
|
slog.F("enabled", value),
|
|
slog.F("os", runtime.GOOS),
|
|
)
|
|
}
|
|
|
|
execer, err := agentexec.NewExecer()
|
|
if err != nil {
|
|
return xerrors.Errorf("create agent execer: %w", err)
|
|
}
|
|
|
|
if devcontainers {
|
|
logger.Info(ctx, "agent devcontainer detection enabled")
|
|
} else {
|
|
logger.Info(ctx, "agent devcontainer detection not enabled")
|
|
}
|
|
|
|
reinitCtx, reinitCancel := context.WithCancel(ctx)
|
|
defer reinitCancel()
|
|
reinitEvents := agentsdk.WaitForReinitLoop(reinitCtx, logger, client)
|
|
|
|
// Read and strip env vars before the reinit
|
|
// loop so config survives agent restarts.
|
|
contextConfig := agentcontextconfig.ReadEnvConfig()
|
|
agentcontextconfig.ClearEnvVars()
|
|
|
|
var (
|
|
lastOwnerID uuid.UUID
|
|
lastErr error
|
|
mustExit bool
|
|
)
|
|
for {
|
|
prometheusRegistry := prometheus.NewRegistry()
|
|
|
|
promHandler := agent.PrometheusMetricsHandler(prometheusRegistry, logger)
|
|
var serverClose []func()
|
|
if prometheusAddress != "" {
|
|
prometheusSrvClose := ServeHandler(ctx, logger, promHandler, prometheusAddress, "prometheus")
|
|
serverClose = append(serverClose, prometheusSrvClose)
|
|
|
|
if port, err := extractPort(prometheusAddress); err == nil {
|
|
ignorePorts[port] = "prometheus"
|
|
}
|
|
} else {
|
|
logger.Debug(ctx, "prometheus address is empty, disabling prometheus server")
|
|
}
|
|
|
|
if debugAddress != "" {
|
|
// ServerHandle depends on `agnt.HTTPDebug()`, but `agnt`
|
|
// depends on `ignorePorts`. Keep this if statement in sync
|
|
// with below.
|
|
if port, err := extractPort(debugAddress); err == nil {
|
|
ignorePorts[port] = "debug"
|
|
}
|
|
}
|
|
|
|
agnt := agent.New(agent.Options{
|
|
Client: client,
|
|
Logger: logger,
|
|
LogDir: logDir,
|
|
ScriptDataDir: scriptDataDir,
|
|
// #nosec G115 - Safe conversion as tailnet listen port is within uint16 range (0-65535)
|
|
TailnetListenPort: uint16(tailnetListenPort),
|
|
EnvironmentVariables: environmentVariables,
|
|
IgnorePorts: ignorePorts,
|
|
SSHMaxTimeout: sshMaxTimeout,
|
|
Subsystems: subsystems,
|
|
|
|
PrometheusRegistry: prometheusRegistry,
|
|
BlockFileTransfer: blockFileTransfer,
|
|
BlockReversePortForwarding: blockReversePortForwarding,
|
|
BlockLocalPortForwarding: blockLocalPortForwarding,
|
|
Execer: execer,
|
|
Devcontainers: devcontainers,
|
|
DevcontainerAPIOptions: []agentcontainers.Option{
|
|
agentcontainers.WithSubAgentURL(agentAuth.agentURL.String()),
|
|
agentcontainers.WithProjectDiscovery(devcontainerProjectDiscovery),
|
|
agentcontainers.WithDiscoveryAutostart(devcontainerDiscoveryAutostart),
|
|
},
|
|
SocketPath: socketPath,
|
|
SocketServerEnabled: socketServerEnabled,
|
|
AgentFirewallLogProxySocketPath: agentFirewallLogProxySocketPath,
|
|
ContextConfig: contextConfig,
|
|
})
|
|
|
|
if debugAddress != "" {
|
|
// ServerHandle depends on `agnt.HTTPDebug()`, but `agnt`
|
|
// depends on `ignorePorts`. Keep this if statement in sync
|
|
// with above.
|
|
debugSrvClose := ServeHandler(ctx, logger, agnt.HTTPDebug(), debugAddress, "debug")
|
|
serverClose = append(serverClose, debugSrvClose)
|
|
} else {
|
|
logger.Debug(ctx, "debug address is empty, disabling debug server")
|
|
}
|
|
|
|
select {
|
|
case <-ctx.Done():
|
|
logger.Info(ctx, "agent shutting down", slog.Error(context.Cause(ctx)))
|
|
mustExit = true
|
|
case event, ok := <-reinitEvents:
|
|
switch {
|
|
case !ok:
|
|
// Channel closed — the reinit loop exited
|
|
// (terminal 409 or context expired). Keep
|
|
// running the current agent until the parent
|
|
// context is canceled.
|
|
logger.Info(ctx, "reinit channel closed, running without reinit capability")
|
|
reinitEvents = nil
|
|
<-ctx.Done()
|
|
mustExit = true
|
|
case event.OwnerID != uuid.Nil && event.OwnerID == lastOwnerID:
|
|
// Duplicate reinit for same owner — already
|
|
// reinitialized. Cancel the reinit loop
|
|
// goroutine and keep the current agent.
|
|
logger.Info(ctx, "skipping redundant reinit, owner unchanged",
|
|
slog.F("owner_id", event.OwnerID))
|
|
reinitCancel()
|
|
reinitEvents = nil
|
|
<-ctx.Done()
|
|
mustExit = true
|
|
default:
|
|
lastOwnerID = event.OwnerID
|
|
logger.Info(ctx, "agent received instruction to reinitialize",
|
|
slog.F("workspace_id", event.WorkspaceID), slog.F("reason", event.Reason))
|
|
}
|
|
}
|
|
|
|
lastErr = agnt.Close()
|
|
|
|
slices.Reverse(serverClose)
|
|
for _, closeFunc := range serverClose {
|
|
closeFunc()
|
|
}
|
|
|
|
if mustExit {
|
|
break
|
|
}
|
|
|
|
logger.Info(ctx, "agent reinitializing")
|
|
}
|
|
return lastErr
|
|
},
|
|
}
|
|
|
|
cmd.Options = serpent.OptionSet{
|
|
{
|
|
Flag: "log-dir",
|
|
Default: os.TempDir(),
|
|
Description: "Specify the location for the agent log files.",
|
|
Env: "CODER_AGENT_LOG_DIR",
|
|
Value: serpent.StringOf(&logDir),
|
|
},
|
|
{
|
|
Flag: "script-data-dir",
|
|
Default: os.TempDir(),
|
|
Description: "Specify the location for storing script data.",
|
|
Env: "CODER_AGENT_SCRIPT_DATA_DIR",
|
|
Value: serpent.StringOf(&scriptDataDir),
|
|
},
|
|
{
|
|
Flag: "pprof-address",
|
|
Default: "127.0.0.1:6060",
|
|
Env: "CODER_AGENT_PPROF_ADDRESS",
|
|
Value: serpent.StringOf(&pprofAddress),
|
|
Description: "The address to serve pprof.",
|
|
},
|
|
{
|
|
Flag: "agent-header-command",
|
|
Env: "CODER_AGENT_HEADER_COMMAND",
|
|
Value: serpent.StringOf(&agentHeaderCommand),
|
|
Description: "An external command that outputs additional HTTP headers added to all requests. The command must output each header as `key=value` on its own line.",
|
|
},
|
|
{
|
|
Flag: "agent-header",
|
|
Env: "CODER_AGENT_HEADER",
|
|
Value: serpent.StringArrayOf(&agentHeader),
|
|
Description: "Additional HTTP headers added to all requests. Provide as " + `key=value` + ". Can be specified multiple times.",
|
|
},
|
|
{
|
|
Flag: "no-reap",
|
|
|
|
Env: "",
|
|
Description: "Do not start a process reaper.",
|
|
Value: serpent.BoolOf(&noReap),
|
|
},
|
|
{
|
|
Flag: "ssh-max-timeout",
|
|
// tcpip.KeepaliveIdleOption = 72h + 1min (forwardTCPSockOpts() in tailnet/conn.go)
|
|
Default: "72h",
|
|
Env: "CODER_AGENT_SSH_MAX_TIMEOUT",
|
|
Description: "Specify the max timeout for a SSH connection, it is advisable to set it to a minimum of 60s, but no more than 72h.",
|
|
Value: serpent.DurationOf(&sshMaxTimeout),
|
|
},
|
|
{
|
|
Flag: "tailnet-listen-port",
|
|
Default: "0",
|
|
Env: "CODER_AGENT_TAILNET_LISTEN_PORT",
|
|
Description: "Specify a static port for Tailscale to use for listening.",
|
|
Value: serpent.Int64Of(&tailnetListenPort),
|
|
},
|
|
{
|
|
Flag: "prometheus-address",
|
|
Default: "127.0.0.1:2112",
|
|
Env: "CODER_AGENT_PROMETHEUS_ADDRESS",
|
|
Value: serpent.StringOf(&prometheusAddress),
|
|
Description: "The bind address to serve Prometheus metrics.",
|
|
},
|
|
{
|
|
Flag: "debug-address",
|
|
Default: "127.0.0.1:2113",
|
|
Env: "CODER_AGENT_DEBUG_ADDRESS",
|
|
Value: serpent.StringOf(&debugAddress),
|
|
Description: "The bind address to serve a debug HTTP server.",
|
|
},
|
|
{
|
|
Name: "Human Log Location",
|
|
Description: "Output human-readable logs to a given file.",
|
|
Flag: "log-human",
|
|
Env: "CODER_AGENT_LOGGING_HUMAN",
|
|
Default: "/dev/stderr",
|
|
Value: serpent.StringOf(&slogHumanPath),
|
|
},
|
|
{
|
|
Name: "JSON Log Location",
|
|
Description: "Output JSON logs to a given file.",
|
|
Flag: "log-json",
|
|
Env: "CODER_AGENT_LOGGING_JSON",
|
|
Default: "",
|
|
Value: serpent.StringOf(&slogJSONPath),
|
|
},
|
|
{
|
|
Name: "Stackdriver Log Location",
|
|
Description: "Output Stackdriver compatible logs to a given file.",
|
|
Flag: "log-stackdriver",
|
|
Env: "CODER_AGENT_LOGGING_STACKDRIVER",
|
|
Default: "",
|
|
Value: serpent.StringOf(&slogStackdriverPath),
|
|
},
|
|
{
|
|
Flag: "agent-flight-recorder-size",
|
|
Env: "CODER_AGENT_FLIGHT_RECORDER_SIZE",
|
|
Default: strconv.Itoa(defaultAgentFlightRecorderSize),
|
|
Description: "Number of log entries below the current log level to keep " +
|
|
"in memory and emit on errors. Set to 0 to disable the flight recorder.",
|
|
Value: serpent.Int64Of(&flightRecorderSize),
|
|
},
|
|
{
|
|
Flag: "block-file-transfer",
|
|
Default: "false",
|
|
Env: "CODER_AGENT_BLOCK_FILE_TRANSFER",
|
|
Description: fmt.Sprintf("Block file transfer using known applications: %s.", strings.Join(agentssh.BlockedFileTransferCommands, ",")),
|
|
Value: serpent.BoolOf(&blockFileTransfer),
|
|
},
|
|
{
|
|
Flag: "block-reverse-port-forwarding",
|
|
Default: "false",
|
|
Env: "CODER_AGENT_BLOCK_REVERSE_PORT_FORWARDING",
|
|
Description: "Block reverse port forwarding through the SSH server (ssh -R).",
|
|
Value: serpent.BoolOf(&blockReversePortForwarding),
|
|
},
|
|
{
|
|
Flag: "block-local-port-forwarding",
|
|
Default: "false",
|
|
Env: "CODER_AGENT_BLOCK_LOCAL_PORT_FORWARDING",
|
|
Description: "Block local port forwarding through the SSH server (ssh -L).",
|
|
Value: serpent.BoolOf(&blockLocalPortForwarding),
|
|
},
|
|
{
|
|
Flag: "devcontainers-enable",
|
|
Default: "true",
|
|
Env: "CODER_AGENT_DEVCONTAINERS_ENABLE",
|
|
Description: "Allow the agent to automatically detect running devcontainers.",
|
|
Value: serpent.BoolOf(&devcontainers),
|
|
},
|
|
{
|
|
Flag: "devcontainers-project-discovery-enable",
|
|
Default: "true",
|
|
Env: "CODER_AGENT_DEVCONTAINERS_PROJECT_DISCOVERY_ENABLE",
|
|
Description: "Allow the agent to search the filesystem for devcontainer projects.",
|
|
Value: serpent.BoolOf(&devcontainerProjectDiscovery),
|
|
},
|
|
{
|
|
Flag: "devcontainers-discovery-autostart-enable",
|
|
Default: "false",
|
|
Env: "CODER_AGENT_DEVCONTAINERS_DISCOVERY_AUTOSTART_ENABLE",
|
|
Description: "Allow the agent to autostart devcontainer projects it discovers based on their configuration.",
|
|
Value: serpent.BoolOf(&devcontainerDiscoveryAutostart),
|
|
},
|
|
{
|
|
Flag: "socket-server-enabled",
|
|
Default: "true",
|
|
Env: "CODER_AGENT_SOCKET_SERVER_ENABLED",
|
|
Description: "Enable the agent socket server.",
|
|
Value: serpent.BoolOf(&socketServerEnabled),
|
|
},
|
|
{
|
|
Flag: "socket-path",
|
|
Env: "CODER_AGENT_SOCKET_PATH",
|
|
Description: "Specify the path for the agent socket.",
|
|
Value: serpent.StringOf(&socketPath),
|
|
},
|
|
{
|
|
Flag: "boundary-log-proxy-socket-path",
|
|
Default: boundarylogproxy.DefaultSocketPath(),
|
|
Env: "CODER_AGENT_BOUNDARY_LOG_PROXY_SOCKET_PATH",
|
|
Description: "The path for the boundary log proxy server Unix socket. Boundary should write audit logs to this socket.",
|
|
Value: serpent.StringOf(&agentFirewallLogProxySocketPath),
|
|
Hidden: true,
|
|
UseInstead: []serpent.Option{
|
|
{
|
|
Flag: "agent-firewall-log-proxy-socket-path",
|
|
Env: "CODER_AGENT_FIREWALL_LOG_PROXY_SOCKET_PATH",
|
|
},
|
|
},
|
|
},
|
|
{
|
|
Flag: "agent-firewall-log-proxy-socket-path",
|
|
Default: boundarylogproxy.DefaultSocketPath(),
|
|
Env: "CODER_AGENT_FIREWALL_LOG_PROXY_SOCKET_PATH",
|
|
Description: "The path for the agent firewall log proxy server Unix socket. Agent firewall should write audit logs to this socket.",
|
|
Value: serpent.StringOf(&agentFirewallLogProxySocketPath),
|
|
},
|
|
}
|
|
agentAuth.AttachOptions(cmd, false)
|
|
return cmd
|
|
}
|
|
|
|
func ServeHandler(ctx context.Context, logger slog.Logger, handler http.Handler, addr, name string) (closeFunc func()) {
|
|
// ReadHeaderTimeout is purposefully not enabled. It caused some issues with
|
|
// websockets over the dev tunnel.
|
|
// See: https://github.com/coder/coder/pull/3730
|
|
//nolint:gosec
|
|
srv := &http.Server{
|
|
Addr: addr,
|
|
Handler: handler,
|
|
}
|
|
go func() {
|
|
ln, err := net.Listen("tcp", addr)
|
|
if err != nil {
|
|
logger.Error(ctx, "http server listen", slog.F("name", name), slog.F("addr", addr), slog.Error(err))
|
|
return
|
|
}
|
|
defer ln.Close()
|
|
logger.Info(ctx, "http server listening", slog.F("addr", ln.Addr()), slog.F("name", name))
|
|
if err := srv.Serve(ln); err != nil && !xerrors.Is(err, http.ErrServerClosed) {
|
|
logger.Error(ctx, "http server serve", slog.F("addr", ln.Addr()), slog.F("name", name), slog.Error(err))
|
|
}
|
|
}()
|
|
|
|
return func() {
|
|
_ = srv.Close()
|
|
}
|
|
}
|
|
|
|
// extractPort handles different url strings.
|
|
// - localhost:6060
|
|
// - http://localhost:6060
|
|
func extractPort(u string) (int, error) {
|
|
port, firstError := urlPort(u)
|
|
if firstError == nil {
|
|
return port, nil
|
|
}
|
|
|
|
// Try with a scheme
|
|
port, err := urlPort("http://" + u)
|
|
if err == nil {
|
|
return port, nil
|
|
}
|
|
return -1, xerrors.Errorf("invalid url %q: %w", u, firstError)
|
|
}
|
|
|
|
// urlPort extracts the port from a valid URL.
|
|
func urlPort(u string) (int, error) {
|
|
parsed, err := url.Parse(u)
|
|
if err != nil {
|
|
return -1, xerrors.Errorf("invalid url %q: %w", u, err)
|
|
}
|
|
if parsed.Port() != "" {
|
|
port, err := strconv.ParseUint(parsed.Port(), 10, 16)
|
|
if err == nil && port > 0 && port < 1<<16 {
|
|
return int(port), nil
|
|
}
|
|
}
|
|
return -1, xerrors.Errorf("invalid port: %s", u)
|
|
}
|
|
|
|
// logSignalNotifyContext is like signal.NotifyContext but logs the received
|
|
// signal before canceling the context.
|
|
func logSignalNotifyContext(parent context.Context, logger slog.Logger, signals ...os.Signal) (context.Context, context.CancelFunc) {
|
|
ctx, cancel := context.WithCancelCause(parent)
|
|
c := make(chan os.Signal, 1)
|
|
signal.Notify(c, signals...)
|
|
|
|
go func() {
|
|
select {
|
|
case sig := <-c:
|
|
logger.Info(ctx, "agent received signal", slog.F("signal", sig.String()))
|
|
cancel(xerrors.Errorf("signal: %s", sig.String()))
|
|
case <-ctx.Done():
|
|
logger.Info(ctx, "ctx canceled, stopping signal handler")
|
|
}
|
|
}()
|
|
|
|
return ctx, func() {
|
|
cancel(context.Canceled)
|
|
signal.Stop(c)
|
|
}
|
|
}
|