Load Testing: Verifying Performance Under Heavy Traffic

A practical guide to load testing — deliberately generating heavy, realistic traffic against a system to verify how it behaves under load before real users do it for you — covering load testing concepts and terminology, JMeter, k6, and Azure Load Testing, and how this connects to the observability, system design, and scaling guides covered elsewhere in this series.

Table of Contents

1. Introduction
2. Why Load Testing Is a Distinct Discipline from Functional Testing
3. Core Concepts and Vocabulary
4. Types of Load Tests
5. JMeter
6. k6
7. Azure Load Testing
8. Designing a Realistic Load Test
9. Reading Results and Finding the Actual Bottleneck
10. Load Testing in CI/CD
11. Load Testing Stateful and Third-Party-Dependent Systems
12. Choosing Among the Three Tools
13. Common Pitfalls
14. Quick Reference Table
15. Conclusion

Introduction

Load testing deliberately generates heavy, realistic traffic against a system — before real users do it for you, and ideally before a critical launch, sale, or traffic event puts a system under load for the first time with no rehearsal. This guide covers the discipline's core concepts and vocabulary, then three widely used tools spanning a real range of philosophies: JMeter (the long-established, GUI-and-XML-driven veteran), k6 (the modern, developer-centric, code-as-tests tool), and Azure Load Testing (a managed service wrapping and scaling k6 itself). It connects directly to this series' System Design guide's back-of-the-envelope estimation, the OpenTelemetry/Distributed Tracing/Prometheus-Grafana observability trio needed to actually interpret results, and the resilience patterns covered in the Microservices guide.

// A k6 load test script, at its simplest
import http from 'k6/http';
import { sleep } from 'k6';

export const options = { vus: 50, duration: '2m' }; // 50 virtual users, for 2 minutes

export default function () {
  http.get('https://api.example.com/products');
  sleep(1);
}

Fifty simulated users, hitting an endpoint continuously for two minutes — a small, complete example of what this entire discipline builds outward from.

1. Why Load Testing Is a Distinct Discipline from Functional Testing

Functional tests verify correctness; load tests verify behavior under load

As covered throughout this series' xUnit and Integration Tests guides, functional tests (unit and integration tests) verify that a system produces the correct result for a given input — a single request, in isolation, with no concurrent load. Load testing asks a genuinely different question: does the system continue to behave correctly, and within acceptable performance bounds, when many requests arrive concurrently, sustained over time?

Bugs that only manifest under load, and never under functional testing

A functional test: one request, one response — passes cleanly, every time
Under 500 concurrent users: connection pool exhaustion, lock contention, memory pressure,
  cache stampedes, database connection limits — none of which a single-request test could ever surface

This connects directly to concerns raised throughout this series — the connection pooling limits covered in the PostgreSQL guide, the cache stampede problem covered in the Redis guide, the thread pool/connection pool exhaustion covered in the Microservices guide's bulkhead pattern discussion — every one of these is a genuine failure mode that a correctly-passing functional test suite will never reveal, precisely because it only manifests under genuine concurrent load.

The real-world cost of skipping this discipline

A system that's never been load tested is, in a meaningful sense, making its first load test attempt during a real, high-stakes traffic event — a product launch, a marketing campaign, a seasonal sales spike — with real customers as the test subjects and real revenue at stake if it fails. Load testing exists specifically to move that discovery earlier, into a controlled environment where a failure is a data point to act on, not an incident to recover from.

2. Core Concepts and Vocabulary

Virtual users (VUs) and requests per second

50 virtual users, each making a request roughly once per second → approximately 50 requests/second

A virtual user (VU) simulates one concurrent, independent user interacting with the system — the relationship between VU count and actual requests-per-second depends on how quickly each VU's simulated actions complete and how much think-time (deliberate pauses between actions) is built into the test script, which is why "50 VUs" and "50 requests/second" are related but not identical concepts, worth being precise about when comparing test results or communicating a target load to stakeholders.

Throughput, latency, and error rate — the three core measurements

Throughput:  how many requests the system successfully processes per unit of time
Latency:     how long each individual request takes (commonly reported as p50/p95/p99, per this
              series' Distributed Tracing and Prometheus/Grafana guides)
Error rate:   what percentage of requests fail (timeouts, 5xx responses, connection refused)

Every load test ultimately reports some combination of these three — and, critically, they interact: as load increases, throughput typically rises up to a point, then plateaus or degrades, while latency and error rate typically begin rising once the system approaches its actual capacity limit, which is precisely the inflection point load testing exists to find (Section 8 covers reading this inflection point in practice).

Saturation point: where the system stops keeping up

Load:        ▁▂▃▄▅▆▇█ (steadily increasing)
Throughput:   ▁▂▃▄▅▆▇▇  (rises, then plateaus)
Latency:       ▁▁▁▁▁▂▅█  (stays flat, then rises sharply — often exponentially — near the saturation point)

The saturation point is where the system's throughput stops increasing even as offered load continues to increase — beyond this point, additional load doesn't produce additional useful work, it just produces longer queues, higher latency, and eventually errors. Finding this point for a given system, under a given configuration, is one of load testing's most valuable, concrete outputs.

Baseline: the number everything else is compared against

Before interpreting any load test result meaningfully, it's worth establishing a baseline — the system's performance characteristics under light, non-stressed load — since every subsequent, heavier test's results are meaningful primarily in comparison to that baseline, not as an absolute number in isolation.

3. Types of Load Tests

Load test: sustained, expected traffic

Simulating expected peak production traffic, sustained for a representative duration (e.g., 30 minutes)

The most common type — verifying the system handles its genuinely expected peak load (informed by the back-of-the-envelope estimation covered in this series' System Design guide) comfortably, without excessive latency or errors.

Stress test: pushing beyond expected load to find the breaking point

Gradually increasing load well beyond expected peak, until the system genuinely fails or degrades unacceptably

A stress test deliberately goes beyond what's expected, specifically to find the saturation point (Section 2) and understand how the system fails once past it — does it degrade gracefully (rising latency, but still functioning) or fail catastrophically (crashing, cascading failures across dependent services, per this series' Microservices guide's resilience patterns)? This distinction matters enormously for incident preparedness.

Spike test: a sudden, sharp burst rather than a gradual ramp

Load: ▁▁▁▁█████▁▁▁▁ (a sudden, sharp spike, then a return to baseline)

Simulates a sudden traffic surge (a flash sale starting, a link going viral, a DDoS-adjacent traffic pattern) rather than a gradual increase — this specifically tests whether autoscaling (per this series' Kubernetes/Helm and Azure/AWS Compute guides) can react quickly enough, and whether a queue-based architecture (per this series' RabbitMQ/Kafka guides) actually absorbs the burst the way Section 6 of the System Design guide describes, rather than the system being overwhelmed before scaling or queuing mechanisms have a chance to respond.

Soak test (endurance test): sustained load over a much longer duration

Load: a moderate, sustained level, held for 8+ hours or even days, rather than minutes

A soak test runs at a moderate, sustained load for a genuinely long duration — hours or days rather than minutes — specifically to catch problems that only manifest over time: memory leaks, connection pool exhaustion that accumulates gradually, disk space filling up from logs or temp files, or a slow degradation invisible in a short test but very real over a longer production timeframe.

Choosing which type(s) a given system actually needs

Not every system needs every type run routinely — a load test verifying expected peak traffic is the most broadly applicable starting point; stress and spike tests are particularly valuable before a known, high-stakes traffic event; soak tests are worth running periodically for any long-running service, especially one with a history of memory or resource-leak concerns.

4. JMeter

The established, GUI-and-protocol-driven veteran

Apache JMeter has been the long-standing, widely used open-source load testing tool, built around a graphical test-plan designer (though it also supports command-line, headless execution for CI) and a broad, mature protocol support surface — HTTP, JDBC, JMS, SOAP, FTP, and more — reflecting its origins predating the API-centric, HTTP/JSON-dominated web that most systems in this series target.

Building a test plan

<!-- JMeter test plans are XML (.jmx files), typically authored via the GUI rather than hand-written -->
<ThreadGroup>
  <num_threads>50</num_threads>
  <ramp_time>30</ramp_time>
  <duration>120</duration>
</ThreadGroup>

A JMeter Thread Group defines the virtual user count (num_threads), ramp-up period (how long to take reaching that count, avoiding an instantaneous, artificial spike at test start), and total duration — nested underneath it, Samplers (HTTP Request, JDBC Request, etc.) define the actual requests each virtual user makes, and Listeners collect and display results.

Running headless, for CI integration

jmeter -n -t test-plan.jmx -l results.jtl -e -o report-output/

For CI integration (per this series' GitHub Actions and Azure DevOps guides), JMeter runs in non-GUI mode (-n), producing a results file (-l) and, optionally, an HTML report (-e -o) — this is how a load test defined via JMeter's GUI designer gets executed automatically as part of a pipeline rather than only ever run manually by a person clicking through the desktop application.

JMeter's genuine strengths and honest limitations

Strengths: broad protocol support beyond plain HTTP, a mature plugin ecosystem, and a low barrier to entry for testers who prefer a GUI-driven workflow over writing test scripts as code. Limitations: JMeter's own architecture (each virtual user is a full JVM thread) is comparatively resource-heavy per simulated user compared to k6's approach (Section 5), meaning generating very high virtual user counts from a single JMeter instance requires meaningfully more load-generator hardware than an equivalent k6 test would; and its .jmx XML test plans, while GUI-editable, are considerably less naturally version-controlled and code-reviewed than a plain JavaScript test script.

5. k6

Test scripts as actual JavaScript code

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '1m', target: 50 },   // ramp up to 50 VUs over 1 minute
    { duration: '3m', target: 50 },   // hold at 50 VUs for 3 minutes
    { duration: '1m', target: 0 },     // ramp down
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'], // fail the test if p95 latency exceeds 500ms
    http_req_failed: ['rate<0.01'],     // fail the test if error rate exceeds 1%
  },
};

export default function () {
  const res = http.get('https://api.example.com/products');
  check(res, { 'status is 200': (r) => r.status === 200 });
  sleep(1);
}

k6's defining design choice is treating a load test script as genuine, version-controllable JavaScript (executed by a Go-based, resource-efficient runtime underneath, not an actual browser JS engine) — this means a load test can live in the same repository as the application it tests, be code-reviewed through the same pull-request process covered in this series' GitHub Actions guide, and be authored by developers using familiar language constructs rather than a GUI-driven, XML-configuration workflow.

Thresholds: pass/fail criteria built directly into the test

thresholds: {
  http_req_duration: ['p(95)<500', 'p(99)<1000'],
  http_req_failed: ['rate<0.01'],
}

Thresholds let a k6 test script define explicit, automatically-evaluated pass/fail criteria — rather than a human manually eyeballing a results dashboard after the fact, k6 itself reports the test run as failed if p95 latency exceeds 500ms or the error rate exceeds 1%, which is precisely what makes automated load testing in CI (Section 9) genuinely actionable rather than just producing a report someone has to remember to check.

Multiple, realistic scenarios in one script

export const options = {
  scenarios: {
    browsing: { executor: 'constant-vus', vus: 30, duration: '5m', exec: 'browseProducts' },
    checkout: { executor: 'ramping-vus', startVUs: 0, stages: [{ duration: '2m', target: 10 }], exec: 'completeCheckout' },
  },
};

export function browseProducts() { /* simulates a browsing user */ }
export function completeCheckout() { /* simulates a purchasing user */ }

k6's scenarios let one test script simulate genuinely different concurrent user behaviors simultaneously (most users browsing, a smaller number actually checking out) — directly supporting Section 7's emphasis on realistic traffic mixes rather than every simulated user hitting the exact same endpoint identically.

k6's resource efficiency, and why it matters at genuine scale

Because k6's runtime is built in Go and each virtual user is a lightweight goroutine rather than a full OS thread (JMeter's model), a single k6 load-generator machine can typically simulate meaningfully more virtual users than an equivalently-sized JMeter instance — relevant specifically once a test needs to simulate thousands or tens of thousands of concurrent users, at which point load-generator capacity itself becomes a genuine constraint worth minimizing.

6. Azure Load Testing

A managed service, built directly on k6 under the hood

Azure Load Testing is a fully managed Azure service that runs k6 (or JMeter) test scripts at scale, without requiring you to provision, size, or manage the load-generator infrastructure yourself — directly connecting to this series' Azure Compute guide's broader theme of trading infrastructure management for a managed service.

az load test create --test-id my-api-load-test --load-test-resource my-load-testing-resource \
    --load-test-config-file loadtest-config.yaml

# loadtest-config.yaml
testId: my-api-load-test
testPlan: load-test-script.js  # a genuine k6 script, per Section 5
engineInstances: 5              # how many load-generator instances to run in parallel

Why a managed load-generation service solves a genuine problem

Generating truly high load requires the load generator itself to have sufficient network bandwidth and compute capacity — running a large-scale load test from a single developer's laptop, or even a single CI runner, can produce misleading results where the load generator becomes the actual bottleneck, not the system under test. Azure Load Testing (like similar managed offerings such as k6 Cloud, Grafana's own commercial k6 offering) distributes load generation across multiple managed instances (engineInstances above), removing this specific, easy-to-overlook confound.

Automated regression detection tied to app performance metrics

Azure Load Testing can automatically fail a test run based on Azure Monitor metrics from the
SYSTEM UNDER TEST itself (CPU utilization, response time), not just the load generator's own view

A genuinely valuable capability specific to this managed integration: Azure Load Testing can incorporate Azure Monitor metrics from the application/infrastructure under test (per this series' Azure Compute and Prometheus/Grafana guides) directly into the test's pass/fail criteria — not just the load generator's external view of latency and error rate, but the system's own internal resource utilization, giving a fuller picture of whether a test failure stems from the application code itself or from underlying infrastructure constraints.

Integration with Azure DevOps and GitHub Actions

# Azure DevOps pipeline task
- task: AzureLoadTest@1
  inputs:
    azureSubscription: 'my-service-connection'
    loadTestConfigFile: 'loadtest-config.yaml'

As covered in this series' Azure DevOps and GitHub Actions guides, Azure Load Testing integrates as a native pipeline task/action, fitting directly into the same CI/CD pipeline-as-code discipline covered throughout this series — a load test becomes one more automated, version-controlled pipeline stage rather than a separate, manually-triggered activity.

7. Designing a Realistic Load Test

The single most common mistake: testing an unrealistic traffic pattern

// ❌ Every virtual user hitting the exact same endpoint, with no variation and no think-time
export default function () {
  http.get('https://api.example.com/products/1');
}

A load test that has every virtual user repeatedly hit one single endpoint with identical parameters and zero pause between requests produces results that are almost meaningless for predicting real-world behavior — real traffic is a mix of different operations (browsing, searching, checking out), with real users pausing between actions (to read a page, decide what to click next), and real request parameters that vary (different product IDs, not always the same one, which matters enormously for cache hit rates, per this series' Redis guide).

Modeling a realistic traffic mix

export const options = {
  scenarios: {
    browse: { executor: 'constant-vus', vus: 70, exec: 'browse' },   // 70% of traffic: browsing
    search: { executor: 'constant-vus', vus: 20, exec: 'search' },     // 20%: searching
    checkout: { executor: 'constant-vus', vus: 10, exec: 'checkout' }, // 10%: actually purchasing
  },
};

Deriving a realistic mix — informed by actual production traffic data (per this series' Structured Logging and Prometheus/Grafana guides, which capture exactly this kind of real usage pattern) rather than guessing — is what makes a load test's results genuinely predictive of real production behavior, rather than an artificial stress on one specific code path that may not even be representative of where real load actually concentrates.

Including think-time

sleep(Math.random() * 3 + 1); // 1-4 seconds of "reading the page" between actions

As covered in Section 2's VU-vs-requests-per-second distinction, deliberately including realistic pauses between a virtual user's actions is what makes the relationship between VU count and actual generated load match reality — without it, VU count dramatically overstates real request volume, since real users don't fire requests continuously with zero pause.

Varying test data to avoid artificially inflating cache hit rates

const productId = Math.floor(Math.random() * 1000) + 1; // varies across a realistic product catalog range
http.get(`https://api.example.com/products/${productId}`);

If every virtual user requests the exact same resource, a cache (per this series' Redis guide) will report an artificially perfect hit rate that doesn't reflect how a much wider variety of real product IDs would actually behave against that same cache — varying test data across a realistic range is essential for a load test's cache-related findings to be trustworthy.

8. Reading Results and Finding the Actual Bottleneck

The result summary is the start of the investigation, not the end

p50: 120ms   p95: 480ms   p99: 2,340ms   error rate: 2.3%

A results summary like this tells you that something degrades under load — it doesn't tell you why, which is precisely where this series' observability trio becomes essential to the load-testing workflow, not a separate, unrelated concern.

Correlating load test results with distributed traces

During the load test window, pull a representative slow trace (p99 bucket) — per this series'
Distributed Tracing guide — to see EXACTLY which downstream span dominated that specific slow request

This is a direct, practical application of this series' Distributed Tracing guide's root-cause-analysis workflow, specifically triggered by a load test's aggregate findings — rather than guessing at which component is the bottleneck, pulling actual traces from the load test's time window shows the genuine, specific cause (a database connection pool exhausted under concurrent load, an external payment gateway call dominating latency, a lock contention issue), exactly as that guide's flame-graph-driven investigation describes.

Correlating with infrastructure metrics

# Per this series' Prometheus/Grafana guide — checking resource saturation DURING the load test window
rate(process_cpu_seconds_total[1m])
pg_stat_activity_count  # active database connections, checking for pool exhaustion

Checking CPU, memory, database connection pool utilization, and cache hit rate (per this series' Prometheus/Grafana guide) during the exact load test window is what actually reveals which specific resource saturated first — the database's connection pool, the application's CPU, a downstream service's own capacity — turning "latency degraded under load" into a specific, actionable finding.

The load test's real deliverable: a specific, named bottleneck and a concrete next step

A load test that concludes "performance degrades above 500 concurrent users" is a starting point; a load test that concludes "the database connection pool, configured for a maximum of 100 connections, becomes the limiting factor above roughly 450 concurrent users, at which point requests begin queuing for an available connection" is the genuinely actionable outcome this discipline exists to produce — connecting directly back to this series' System Design guide's "identify the bottleneck first" framework.

9. Load Testing in CI/CD

Where load testing fits in the testing pyramid, revisited

As covered in this series' CI/CD Pipelines and Integration Tests guides, load tests are slower and more resource-intensive than even integration tests — they don't belong in the fast, frequent-feedback layers of the testing pyramid, and running a full-scale load test on every single commit is rarely practical or necessary.

Common patterns for when load tests actually run

# A scheduled, periodic load test — not on every commit
on:
  schedule:
    - cron: '0 2 * * 1' # weekly, per this series' GitHub Actions guide's schedule trigger discussion

# OR, gated specifically before a significant release
on:
  workflow_dispatch: # manually triggered before a known high-stakes deployment

Load tests commonly run on a scheduled cadence (catching gradual performance regressions over time, connecting to the soak-test discipline from Section 3) or are deliberately triggered before a significant release or known traffic event — rather than gating every single pull request, which would slow the fast-feedback loop this series' CI/CD Pipelines guide emphasizes for the vast majority of changes.

Using k6's thresholds to make load tests genuinely CI-native

thresholds: {
  http_req_duration: ['p(95)<500'],
  http_req_failed: ['rate<0.01'],
}

As covered in Section 5, k6's threshold mechanism is what makes a load test genuinely automatable in CI, exactly like a unit test's pass/fail assertion — the pipeline step fails automatically if the defined performance criteria aren't met, rather than producing a report a human has to remember to review, connecting directly to this series' CI/CD Pipelines guide's quality-gate discipline.

Detecting performance regressions across deployments

Load test result BEFORE deploying v2.3: p99 = 480ms
Load test result AFTER deploying v2.3:   p99 = 1,840ms  ← a regression, caught before it reached full production traffic

Running a comparable load test against a staging environment immediately before and after a deployment (or, more rigorously, against each candidate build) is a direct, evidence-based way to catch a performance regression before it reaches production — mirroring the trace-based deployment-comparison technique covered in this series' Distributed Tracing guide, but proactively, via deliberately generated load, rather than reactively discovered from real production traffic after the fact.

10. Load Testing Stateful and Third-Party-Dependent Systems

The problem: load testing shouldn't hammer real, external third parties

As covered in this series' Integration Tests guide's WireMock discussion, a load test that genuinely calls a real, external payment gateway or third-party API thousands of times risks real cost, rate-limiting, or violating that provider's terms of service — the same WireMock-based stand-in pattern covered there applies directly here, at load-testing scale, letting a test generate heavy traffic against your own system while safely stubbing out the actual external dependency.

Load testing against a genuinely production-like environment, not a scaled-down staging tier

❌ Load testing against a staging environment with 1/10th the database size and 1/10th the compute
   → results don't meaningfully predict production behavior at real scale

For results to be genuinely predictive, the environment under test needs to be reasonably representative of production's actual scale — infrastructure sizing, database volume, and cache warm state all meaningfully affect load test results, and testing against a dramatically smaller staging environment risks producing results that don't transfer to how the system will actually behave under real production conditions.

Managing test data volume and state for stateful load tests

// Creating genuinely new orders on every load test run needs a cleanup/reset strategy afterward,
// echoing the test data isolation concerns covered in this series' Integration Tests guide

A load test that creates real orders, real user accounts, or other persistent state needs its own data cleanup strategy — either a dedicated, regularly-reset load testing environment, or deliberate cleanup automation run after each test — the same test-data-isolation discipline covered in this series' Integration Tests guide, applied here at a much larger volume and correspondingly larger cleanup cost.

11. Choosing Among the Three Tools

Need broad, non-HTTP protocol support (JDBC, JMS, legacy protocols), or a GUI-first authoring workflow?
        │
        ├── Yes → JMeter
        │
        └── No — primarily HTTP/API testing
                │
                ├── Want test scripts as version-controlled code, tight CI integration,
                │   and don't want to manage load-generator infrastructure yourself?
                │       │
                │       ├── Want a fully managed service (esp. if already on Azure)? → Azure Load Testing
                │       │
                │       └── Want to self-host/self-manage the load generator? → k6 (open-source, self-run)

The practical reality: k6 (self-hosted or via a managed service) is the modern default for HTTP/API-centric systems

Given that the overwhelming majority of systems covered throughout this series are HTTP/JSON APIs (REST, GraphQL) or gRPC services, k6's developer-centric, code-as-tests philosophy — and its resource efficiency at genuine scale — has made it the modern default choice for most new load testing efforts, with Azure Load Testing serving as a natural, managed on-ramp for teams already in the Azure ecosystem who'd rather not provision and scale their own load-generator infrastructure. JMeter remains genuinely valuable specifically for its broader protocol support and established GUI-driven workflow, particularly in organizations with existing JMeter expertise and test-plan investment.

12. Common Pitfalls

Quick Reference Table

Conclusion

Load testing exists to answer a question functional testing structurally cannot: does this system behave correctly and performantly under the concurrent, sustained load real production traffic will eventually place on it? JMeter, k6, and Azure Load Testing represent three genuinely different points on the same spectrum — established GUI-driven breadth, modern code-as-tests efficiency, and fully managed convenience — but all three exist to answer that same question, and all three produce results that are only genuinely useful once correlated with the distributed tracing and infrastructure metrics covered elsewhere in this series' observability guides.

The discipline that makes load testing valuable rather than a checkbox exercise is the same one this entire series has emphasized: design a test that's genuinely representative of real traffic, run it against a genuinely representative environment, and — most importantly — treat the raw numbers as the start of an investigation rather than the end of one, using the observability tooling covered throughout this series to turn "it got slower under load" into a specific, named bottleneck with a concrete path to fixing it.

Found this useful? Feel free to star the repo, open an issue with corrections, or share the saturation point that turned out to be exactly where your system's architecture predicted it would be.