Applies the Vale findings spike review's `ensure`, `should`, and `since` decisions, following `word-choice.md#choose-words-that-have-one-meaning`. ## `ensure`, `ensures`, `ensuring`: every row marked valid - 148 lines in 70 files. - Most became `make sure`, `makes sure`, or `making sure`. - When the subject isn't a person, I used a verb that reads better, such as `guarantees`, `keeps`, `prevents`, `maintains`, or `provides`. - `admin/monitoring/health-check.md:227`: "Make sure that" is now "Make sure", so the step stays within `Coder.StepLength`. - Left as-is: `admin/setup/configuration-reference.md:1339`, which is generated from `codersdk`. ## `should` and `since`: only rows with a verdict or a note - `.style/README.md`: "what should be routed elsewhere" is now "what to route elsewhere". - `.style/content-guidelines.md`: - "Avoid mixing those modes within a single page." - "Every page must be accurate" - "Check docs claims against the actual implementation" - "must include the documentation" - "Support owns the content they produce." - "because they don't change" - `.style/styles/Coder/README.md`: "must link to the matching section". - `admin/external-auth.md`: - "The redirect URI for Gitea is ..." - "Use custom authentication and token URLs ...", which is now active voice. - `envbuilder-releases-known-issues.md`: "so it's experimental and prone to breaking changes". Rows marked unsure with no note were left alone: `content-guidelines.md` lines 49, 330, and 405, and `envbuilder/index.md:24`. The changed files have no new findings from other rules. --- AI-generated by Superblink on behalf of @nickvigilante. --------- Co-authored-by: coder[bot] <137810501+coder[bot]@users.noreply.github.com>
189 lines
9.8 KiB
Markdown
189 lines
9.8 KiB
Markdown
---
|
|
title: Scale Testing
|
|
---
|
|
|
|
Scaling Coder involves planning and testing to make sure it can handle more load without compromising service.
|
|
This process encompasses infrastructure setup, traffic projections, and aggressive testing to identify and mitigate potential bottlenecks.
|
|
|
|
A dedicated Kubernetes cluster for Coder is recommended to configure, host, and manage Coder workloads.
|
|
Kubernetes provides container orchestration capabilities, allowing Coder to efficiently deploy, scale, and manage workspaces across a distributed infrastructure.
|
|
This guarantees high availability, fault tolerance, and scalability for Coder deployments.
|
|
Coder is deployed on this cluster using the [Helm chart](../server/kubernetes/index.md#4-install-coder-with-helm).
|
|
|
|
For more information about scaling, see our [Coder scaling best practices](scale-coder.md).
|
|
|
|
## Methodology
|
|
|
|
Our scale tests include the following stages:
|
|
|
|
1. Prepare environment: create expected users and provision workspaces.
|
|
|
|
1. SSH connections: establish user connections with agents, verifying their ability to echo back received content.
|
|
|
|
1. Web Terminal: verify the pseudo-terminal (PTY) connection used for communication with Web Terminal.
|
|
|
|
1. Workspace application traffic: confirm that specific workspace apps handle user connections and echo back received content.
|
|
|
|
1. Dashboard evaluation: verify the responsiveness and stability of Coder dashboards under varying load conditions.
|
|
|
|
This is achieved by simulating user interactions using instances of headless Chromium browsers.
|
|
|
|
1. Cleanup: delete workspaces and users created in step 1.
|
|
|
|
## Infrastructure and setup requirements
|
|
|
|
The scale tests runner can distribute the workload to overlap single scenarios based on the workflow configuration:
|
|
|
|
| Scenario | T0 | T1 | T2 | T3 | T4 | T5 | T6 |
|
|
|----------------------|----|----|----|----|----|----|----|
|
|
| SSH connections | X | X | X | X | | | |
|
|
| Web Terminal (PTY) | | X | X | X | X | | |
|
|
| Workspace apps | | | X | X | X | X | |
|
|
| Dashboard (headless) | | | | X | X | X | X |
|
|
|
|
This pattern closely reflects how our customers naturally use the system.
|
|
SSH connections are heavily utilized because they're the primary communication channel for IDEs with VS Code and JetBrains plugins.
|
|
|
|
The basic setup of scale tests environment involves:
|
|
|
|
1. Scale tests runner (32 vCPU, 128 GB RAM)
|
|
1. Coder: 2 replicas (4 vCPU, 16 GB RAM)
|
|
1. Database: 1 instance (2 vCPU, 32 GB RAM)
|
|
1. Provisioner: 50 instances (0.5 vCPU, 512 MB RAM)
|
|
|
|
The test is deemed successful if:
|
|
|
|
- Users did not experience interruptions in their workflows,
|
|
- `coderd` did not crash or require restarts, and
|
|
- No other internal errors were observed.
|
|
|
|
## Traffic Projections
|
|
|
|
In our scale tests, we simulate activity from 2000 users, 2000 workspaces, and 2000 agents, with two items of workspace agent metadata being sent every 10 seconds.
|
|
Here are the resulting metrics:
|
|
|
|
Coder:
|
|
|
|
- Median CPU usage for `coderd`: 3 vCPU, peaking at 3.7 vCPU while all tests are running concurrently.
|
|
- Median API request rate: 350 requests per second (RPS) during dashboard tests, 250 RPS during Web Terminal and workspace apps tests.
|
|
- 2000 agent API connections with latency: p90 at 60 ms, p95 at 220 ms.
|
|
- on average 2400 Web Socket connections during dashboard tests.
|
|
|
|
Provisionerd:
|
|
|
|
- Median CPU usage is 0.35 vCPU during workspace provisioning.
|
|
|
|
Database:
|
|
|
|
- Median CPU utilization is 80%, with a significant portion dedicated to writing workspace agent metadata.
|
|
- Memory utilization averages at 40%.
|
|
- `write_ops_count` between 6.7 and 8.4 operations per second.
|
|
|
|
## Available reference architectures
|
|
|
|
- [Up to 1,000 users](../plan/sizing/1k-users.md)
|
|
|
|
- [Up to 2,000 users](../plan/sizing/2k-users.md)
|
|
|
|
- [Up to 3,000 users](../plan/sizing/3k-users.md)
|
|
-
|
|
- [Up to 10,000 users](../plan/sizing/10k-users.md)
|
|
|
|
## Hardware recommendation
|
|
|
|
### Control plane: coderd
|
|
|
|
To maintain stability and reliability of the Coder control plane, it's essential to focus on node sizing, resource limits, and the number of replicas.
|
|
We recommend referencing public cloud providers such as AWS, GCP, and Azure for guidance on optimal configurations.
|
|
A reasonable approach involves using scaling formulas based on factors like CPU, memory, and the number of users.
|
|
|
|
While the minimum requirements specify 1 CPU core and 2 GB of memory per `coderd` replica, we recommend that you allocate additional resources depending on the workload size to maintain deployment stability.
|
|
|
|
#### CPU and memory usage
|
|
|
|
Enabling [agent stats collection](../../reference/cli/server/index.md#--prometheus-collect-agent-stats) (optional) might increase memory consumption.
|
|
|
|
Enabling direct connections between users and workspace agents (apps or SSH traffic) can help prevent an increase in CPU usage.
|
|
It is recommended to keep [this option enabled](../../reference/cli/index.md#--disable-direct-connections) unless there are compelling reasons to disable it.
|
|
|
|
Inactive users do not consume Coder resources.
|
|
|
|
#### Scaling formula
|
|
|
|
When determining scaling requirements, consider the following factors:
|
|
|
|
- `1 vCPU x 2 GB memory` for every 250 users: A reasonable formula to determine resource allocation based on the number of users and their expected usage patterns.
|
|
- API latency/response time: Monitor API latency and response times to make sure performance stays optimal under varying loads.
|
|
- Average number of HTTP requests: Track the average number of HTTP requests to gauge system usage and identify potential bottlenecks.
|
|
The number of proxied connections: For a very high number of proxied connections, more memory is required.
|
|
|
|
#### HTTP API latency
|
|
|
|
For a reliable Coder deployment dealing with medium to high loads, it's important that API calls for workspace/template queries and workspace build operations respond within 300 ms.
|
|
However, API template insights calls, which involve browsing workspace agent stats and user activity data, might require more time.
|
|
Moreover, Coder API exposes WebSocket long-lived connections for Web Terminal (bidirectional), and Workspace events/logs (unidirectional).
|
|
|
|
If the Coder deployment expects traffic from developers spread across the globe, be aware that customer-facing latency might be higher because of the distance between users and the load balancer.
|
|
Fortunately, the latency can be improved with a deployment of Coder [workspace proxies](../../admin/networking/workspace-proxies.md).
|
|
|
|
#### Node Autoscaling
|
|
|
|
We recommend disabling the autoscaling for `coderd` nodes.
|
|
Autoscaling can cause interruptions for user connections, see [Autoscaling](./scale-testing/utility.md#autoscaling) for more details.
|
|
|
|
### Control plane: Workspace Proxies
|
|
|
|
When scaling [workspace proxies](../../admin/networking/workspace-proxies.md), follow the same guidelines as for `coderd` above:
|
|
|
|
- `1 vCPU x 2 GB memory` for every 250 users.
|
|
- Disable autoscaling.
|
|
|
|
### Control plane: provisionerd
|
|
|
|
Each external provisioner can run a single concurrent workspace build.
|
|
For example, running 10 provisioner containers will allow 10 users to start workspaces at the same time.
|
|
|
|
By default, the control plane runs 3 built-in provisioner daemons, but the _Premium_ Coder release allows for running external provisioners to separate the load caused by workspace provisioning on the `coderd` nodes.
|
|
|
|
#### Scaling formula
|
|
|
|
When determining scaling requirements, consider the following factors:
|
|
|
|
- `1 vCPU x 1 GB memory x 2 concurrent workspace build`: A formula to determine resource allocation based on the number of concurrent workspace builds, and standard complexity of a Terraform template.
|
|
_Rule of thumb_: the more provisioners are free/available, the more concurrent workspace builds can be performed.
|
|
|
|
#### Node Autoscaling
|
|
|
|
Autoscaling provisioners is not an easy problem to solve unless it can be predicted when a number of concurrent workspace builds increases.
|
|
|
|
We recommend disabling autoscaling and adjusting the number of provisioners to developer needs based on the workspace build queuing time.
|
|
|
|
### Data plane: Workspaces
|
|
|
|
To determine workspace resource limits and keep the best developer experience for workspace users, administrators must be aware of a few assumptions.
|
|
|
|
- Workspace pods run on the same Kubernetes cluster, but possibly in a different namespace or on a separate set of nodes.
|
|
- Workspace limits (per workspace user):
|
|
- Evaluate the workspace utilization pattern.
|
|
For instance, web application development does not require high CPU capacity at all times, but will spike during builds or testing.
|
|
- Evaluate minimal limits for single workspace.
|
|
Include in the calculation requirements for Coder agent running in an idle workspace - 0.1 vCPU and 256 MB.
|
|
For instance, developers can choose between 0.5-8 vCPUs, and 1-16 GB memory.
|
|
|
|
#### Scaling formula
|
|
|
|
When determining scaling requirements, consider the following factors:
|
|
|
|
- `1 vCPU x 2 GB memory x 1 workspace`: A formula to determine resource allocation based on the minimal requirements for an idle workspace with a running Coder agent and occasional CPU and memory bursts for building projects.
|
|
|
|
#### Node Autoscaling
|
|
|
|
Workspace nodes can be set to operate in autoscaling mode to mitigate the risk of prolonged high resource utilization.
|
|
|
|
One approach is to scale up workspace nodes when total CPU usage or memory consumption reaches 80%.
|
|
Another option is to scale based on metrics such as the number of workspaces or active users.
|
|
It's important to note that as new users onboard, the autoscaling configuration should account for ongoing workspaces.
|
|
|
|
Scaling down workspace nodes to zero is not recommended, as it will result in longer wait times for workspace provisioning by users.
|
|
However, this might be necessary for workspaces with special resource requirements (for example, graphics processing units (GPUs)) that incur significant cost overheads.
|