Scaling to 100,000 RPS: Deploying Distributed Load Testing on Kubernetes with Helm and Docker

TL;DR: Single-machine load tests inevitably hit CPU and socket descriptor limits when pushing beyond a few thousand requests per second. In LocustPilot, we packaged the entire control plane and distributed worker architecture into Kubernetes Helm charts (helm/locust), allowing engineering teams to scale to 100,000+ RPS across cloud nodes in minutes with automated resource isolation.

When a Single Machine Hits the Wall

A single modern laptop or cloud VM running Locust can comfortably generate between 2,000 and 5,000 requests per second (RPS).

However, when preparing for enterprise-scale traffic events (e.g. streaming sports events, flash sales, or banking cutovers), you need tens of thousands of concurrent users. Attempting this on one machine leads to misleading test results:

1. CPU Saturation: Python processes hit 100% CPU on single cores, artificially increasing response times on the client side.
2. Socket Exhaustion (TIME_WAIT): The operating system runs out of ephemeral ports, causing false Connection Refused errors that have nothing to do with your backend.
3. Bandwidth Throttling: Cloud VM network interfaces bottleneck before your target servers are even stressed.

To scale reliably, you must distribute load generation across a cluster of worker nodes.

The Distributed Master-Worker Architecture

In distributed mode, Locust separates responsibilities across two roles:

• Master Node: Runs the LocustPilot web control center, coordinates worker connections, aggregates incoming metrics, and streams telemetry to ReportPortal. It generates zero HTTP traffic itself.
• Worker Nodes: Connect to the master, spawn virtual users, and bombard the target backend with requests.

       ┌───────────────────────────────┐
       │   LocustPilot Master (UI)     │
       │   (Streamlit + ReportPortal)  │
       └──────────────┬────────────────┘
                      │ Port 5557 (ZMQ)
         ┌────────────┼────────────┐
         │            │            │
   ┌─────▼────┐ ┌─────▼────┐ ┌─────▼────┐
   │ Worker 1 │ │ Worker 2 │ │ Worker N │
   └─────┬────┘ └─────┬────┘ └─────┬────┘
         │            │            │
         └────────────┼────────────┘
                      ▼
         [ Target Microservices ]

Packaging for Kubernetes with Helm (helm/locust)

LocustPilot provides a production-ready Helm chart located in helm/locust/.

1. The Optimized Container (k8s.Dockerfile)

FROM python:3.10-slim

WORKDIR /app

RUN apt-get update && apt-get install -y --no-install-recommends \
    curl build-essential && \
    rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8501 5557
CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]

Photo by Fotis Fotopoulos on Unsplash

2. Deploying the Cluster with One Command

Using the pre-built Helm chart, deploying the entire distributed load testing cluster with ReportPortal telemetry requires a single command:

helm upgrade --install locust ./helm/locust \
  --namespace loadtest \
  --create-namespace \
  --set image.repository=my-registry.io/locust-pilot \
  --set image.tag=latest \
  --set replicaCount=10 \
  --set extraEnv[0].name=RP_TOKEN --set extraEnv[0].value=$RP_TOKEN \
  --set extraEnv[1].name=RP_ENDPOINT --set extraEnv[1].value=$RP_ENDPOINT \
  --set extraEnv[2].name=RP_PROJECT --set extraEnv[2].value=$RP_PROJECT

3. Tuning OS Limits for Massive Scale

When running high-volume workers inside Kubernetes, ensure your Pod security contexts allow high file descriptors:

# helm/locust/values.yaml
worker:
  resources:
    limits:
      cpu: 2000m
      memory: 2Gi
    requests:
      cpu: 1000m
      memory: 1Gi
  sysctls:
    - name: net.ipv4.tcp_tw_reuse
      value: "1"
    - name: net.core.somaxconn
      value: "65535"

What Is Next?

In the final installment (Part 5), we will connect all the pieces into your continuous delivery pipeline: Shift-Left Performance Testing with Automated Quality Gates in GitHub Actions and Bitbucket Pipelines.

👉 Explore the Helm Charts on GitHub

FAQ

How many workers are needed for 50,000 RPS?

Depending on endpoint payload size and SSL overhead, each CPU core running FastHttpUser can typically generate 1,500–3,000 RPS. A cluster of 20–25 worker pods can comfortably sustain 50,000+ RPS.

Can workers be dynamically auto-scaled?

Yes. Kubernetes Horizontal Pod Autoscaler (HPA) can scale worker pods based on CPU utilization or custom test duration parameters.