mirror of
https://github.com/coder/coder.git
synced 2026-11-21 18:58:17 +08:00
main
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
9bafd0a525 | ci(.github): skip a doc-check run when the PR head already has a finished review (#32198) | ||
|
|
dc55633b47 |
ci(.github): start a new doc-check chat per run and stop it when the run ends (#32185)
## Summary doc-check reused one chat for every push to a PR, and a review kept running in Coder after its job stopped waiting for it. This PR starts a new chat for each run, waits up to 5 hours for it, and stops the review and its workspace when the run ends for any reason. On dogfood at 2026-11-13 17:28 UTC, 58 doc-check review chats and 45 of their sub-agents had status `running`, while 5 workflow runs were in progress and 1 was queued. 34 of the 58 chats were created before that day, and the oldest on 2026-09-29. The count covers only chats shared with me, so it is a lower bound. This combines the workflow half of #32061 with the stop script from #31720, and changes no skill file. ## Changes - **New chat per run.** `force-new-chat` replaces the one chat per PR, so a re-run no longer carries every earlier turn on the PR. The prompt gets a `## Previous run` section with the `doc-check-trace` of the latest finished review. - **One workspace per PR.** The job finds `doc-check-pr-<N>`, wakes and starts it, and passes its ID to the chat. On the first run the chat creates the workspace under that name. - **A started review finishes; pushes are debounced.** The review job's concurrency group is keyed on the PR URL with `cancel-in-progress: false`. GitHub keeps one pending run per group and replaces it on each push, so the next review covers the latest head. A `debounce` job sleeps 5 minutes on `synchronize`, in its own group with `cancel-in-progress: true`, so a burst of pushes starts one review. Other triggers do not wait. Merging the PR still cancels a running review (the `delete-workspace` job keeps `cancel-in-progress: true`), then deletes the workspace. - **Every run stops its review.** `.github/scripts/doc-check-stop-review.sh` runs before the review and in a final `always()` step. It drops queued messages, interrupts the chat and its sub-agents, and stops the workspace. - **5-hour wait.** `wait-timeout-seconds` goes from 2400 to 18000, and the job timeout from 50 to 330 minutes. The job now stays in its concurrency group for the whole review, so the next run for the PR waits for it. - **Previous trace lookup.** `.github/scripts/doc-check-previous-trace.sh` reads the 5 newest review chats for the PR and prints the first trace with an `outcome:` line. It reads more than one chat because a cancelled run's chat has no trace. A failed lookup costs only that context and never fails the job. - The `GITHUB_OUTPUT` delimiter for the prompt is random, because the trace is model output. ### Behavior changes to check | Before | After | Why | |--------|-------|-----| | A review past the wait ended the job with a green notice and kept running (#32097) | The last step stops the review, and the job fails | Nothing watches a review after the job ends | | A review kept running after its PR merged | The merge cancels the run | The result has no reader | | The `doc-check` label forced one review, on the `labeled` event | The label forces a review on every push until it is removed | The push run would otherwise skip on paths with no docs surface | | Runs for one PR queued | A started review finishes; pushes during it collapse into one pending run for the latest head, after a 5-minute debounce | Cancelling threw away prefill and starved busy PRs of a verdict | ## Testing Static checks, all passing: `actionlint`, `zizmor`, `shellcheck --external-sources`, and `shfmt -d`. Script tests, run locally: - `doc-check-stop-review.test.sh`: 48 cases against a fake API. - `doc-check-previous-trace.test.sh`: 22 cases, including CRLF in the trace text and in the PR URL. - Mutation check on the trace script: 25 one-line breaks, and the suite failed on 24. The survivor was a redundant trim, which I removed. - The trace extraction ran against 12 real finished doc-check chats on dogfood and returned a trace for all 12 (750 to 3,500 characters). End to end on dogfood, on this PR, 2026-11-13 UTC: | Run | Trigger | Result | |-----|---------|--------| | [1](https://github.com/coder/coder/actions/runs/43474029515) | dispatch, 17:33 | No workspace and no earlier chat. New chat created the workspace `doc-check-pr-32185` and had 2 running sub-agents at 17:48:23. | | [2](https://github.com/coder/coder/actions/runs/43476429715) | dispatch, 17:48:25 | Cancelled run 1 at 17:48:27. Run 1's last step interrupted the chat and both sub-agents and stopped the workspace by 17:48:51. Run 2 then started the same workspace, opened a second chat in it, finished clean at 17:52:54, and stopped the workspace. | | [3](https://github.com/coder/coder/actions/runs/43477511866) | dispatch, 17:55:08 | Cancelled by run 4 while the workspace was starting. Its last step stopped the workspace mid-start. No chat was created. | | [4](https://github.com/coder/coder/actions/runs/43477572348) | `ready_for_review`, 17:55:31 | Waited for run 3 to end, then opened a third chat in the same workspace. That chat's first message carries run 2's trace under `## Previous run`. | Not covered: - The 5-hour timeout path has not run live. The fake-API tests cover the script, and the summary text is a few lines of shell. - The merge path (`delete-workspace` cancelling a review) has not run live. - No CI job runs the two `.test.sh` suites. The workflow runs on `ubuntu-latest` only, and the scripts were not run on macOS or Windows. ## Follow-ups, not in this PR - Skip a run when the head SHA or the PR diff matches the last reviewed one. - Set a status on the head SHA from the trace. - Cancel a review when its PR closes without merging. Supersedes #31720 and the workflow half of #32061. Linear: DOCS-1745 <details> <summary>Plan and decision log</summary> This is PR 1 of a 5-part stack from the design doc "Design: reliable, low-cost doc-check on Solstice". 1. Workflow base: new chat per run, previous trace, workspace ID, 5-hour wait, cancel-in-progress, and the stop script. **This PR.** 2. Skip in Actions: same head SHA, same PR diff, and a delta since the last reviewed SHA with no docs surface. 3. Visible outcome: a status on the head SHA from the trace, for clean, findings, error, and skipped. 4. Packet and lint in Actions: PR data, diffs, open threads, and deterministic check results, written once. 5. Skill: scope to wrong, missing, and broken, tiers, a sub-agent cap, and one long wait. Gated by an eval. Decisions in this PR: - **Why this first.** A re-run's first prompt was 1.01 times the previous run's last prompt, so each push re-sent the whole history of the PR. A new chat per run removes that, and stopping stale reviews removes load that no job is watching. - **5 hours, not 25 or 40 minutes.** Reviews on the local model took a median of 61 minutes when clean and 247 minutes with findings (2026-11-04 to 11-11). A shorter wait would stop most reviews with findings before they post. With `cancel-in-progress`, a new push still ends the review of the old head at once. - **A timeout fails the job.** This reverses #32097, where a timeout was a notice. A green job with no review reads as a clean result. - **The stop runs twice.** The first step catches a review left by a run whose last step did not finish, for example a runner that died. - **The trace lookup is a script with tests, not inline YAML.** It parses model output, and it has to handle chats with no trace, a trace with no outcome, and CRLF. - **The trace is passed as a record, not as instructions.** The prompt says so, and asks for a review of the whole PR. A delta review needs the skip logic and the ledger from later PRs in the stack. - **No skill change.** The skill changes are gated by an eval that does not exist yet. </details> Follow-up commit `64a09655a1` (finish-then-rerun and debounce) generated by Coder Agents on behalf of @bpmct. _Generated by Coder Agents on behalf of @nickvigilante._ --------- Co-authored-by: Ben Potter <me@bpmct.net> |
||
|
|
7298a29885 |
ci(.github): time out stalled Go module downloads (#30889)
Closes CODAGT-878 When proxy.golang.org is having a bad day, a module download can stall for minutes before the proxy resets the stream with `INTERNAL_ERROR`. `go` has no read timeout of its own, so `retry.sh` only gets to retry once the proxy gives up. In [this `test-e2e` run](https://github.com/coder/coder/actions/runs/38567440844/job/115863803256) the second attempt sat there for 10 minutes, and the job hit its 20 minute timeout during Playwright, even though the third attempt passed straight away. To fix, `retry.sh` now takes a `--timeout`, and kills any attempt that runs longer than that. Killing `go mod download` is safe, and the next attempt picks up where the last one left off, as finished zips are already in the module cache. The timer is in bash as we can't rely on GNU `timeout` on the macOS runners, and a step `timeout-minutes` would kill the whole retry loop (it isn't supported in composite actions anyway). I've gone with 120s per attempt. Across the last 32 runs on main, a healthy download took at most 41s on any platform, with macOS the slowest at 26 to 41s. During the incident on 2026-10-14, attempts that failed on their own took between 66s and 10 minutes, and three passed slowly at 113s, 133s and 151s. Those last two would now get killed, but that only costs a retry, which just needs whatever's left. Three attempts at 120s plus backoff comes to about 6 minutes, which still fits in `test-e2e`, whose slowest green run takes about 11 of its 20 minutes. Separately, `make fmt` runs a bare `go mod tidy`, which needs modules that `go mod download` doesn't fetch (test dependencies of dependencies). So it was downloading 166 modules without any retries, and [failing here](https://github.com/coder/coder/actions/runs/38624596116/job/116064172735). The `fmt` job now runs `go mod tidy` through `retry.sh` instead of `go mod download`. |
||
|
|
056c3e4f55 |
ci(.github): collect Windows PostgreSQL diagnostics (#29362)
Add bounded Windows PostgreSQL listener, connection, process, memory and disk diagnostics alongside the existing server logs. Retain timestamped samples and sanitized failures as an artifact on unsuccessful runs. This gathers evidence for PLAT-307 without claiming its cause or a fix. A concrete runner preserves make's exit status and the caller's module-path environment, and cleans up only its owned sampler. Windows concurrency remains 8 packages and 16 tests per package; the sampler's 500 ms connection deadline, PostgreSQL settings, retries and production telemetry are unchanged. Focused checks cover a strict native Windows sample, connection and memory boundaries, sampling limits, cleanup, make exits and environment preservation. Full local pre-commit, action validation and focused checks pass, including LF and CRLF harness copies. Current-head Windows native smoke, reduced harness and Go tests pass. All applicable CI checks pass, including the required check. Related: [PLAT-307](https://linear.app/codercom/issue/PLAT-307/flake-testtelemetry-multiple-variants-postgres-connection-refused). <details> <summary>Implementation approach</summary> Run an isolated, bounded sampler around make. Select diagnostic fields without credentials, command lines or raw error messages. Validate a completed native sample before Go tests, including cached runs, and keep that sample in the failure-artifact directory. </details> Generated with Coder Agents. |
||
|
|
9a417df940 |
ci: add retry logic for Go module operations (#21609)
## Description Add exponential backoff retries to all `go install` and `go mod download` commands across CI workflows and actions. ## Why Fixes [coder/internal#1276](https://github.com/coder/internal/issues/1276) - CI fails when `sum.golang.org` returns 500 errors during Go module verification. This is an infrastructure-level flake that can't be controlled. ## Changes - Created `.github/scripts/retry.sh` - reusable retry helper with exponential backoff (2s, 4s, 8s delays, max 3 attempts), using `scripts/lib.sh` helpers - Wrapped all `go install` and `go mod download` commands with retry in: - `.github/actions/setup-go/action.yaml` - `.github/actions/setup-sqlc/action.yaml` - `.github/actions/setup-go-tools/action.yaml` - `.github/workflows/ci.yaml` - `.github/workflows/release.yaml` - `.github/workflows/security.yaml` - Added GNU tools setup (bash 4+, GNU getopt, make 4+) for macOS in `test-go-pg` job, since `retry.sh` uses `lib.sh` which requires these tools |