Making API Performance Tests More Realistic: From Endpoint Metrics to Role-Based Journeys

Daniel Balcarek's article API Performance Testing: How to Design Realistic Tests makes a straightforward point: load should reflect how people use the system, and the target load should come from an explicit requirement or telemetry. It made me ask a more useful question about my existing suite: What does its load actually represent?

My suite already measured percentiles, throughput, error rates, warm-up phases, and stepped load. It could detect technical regressions and compare individual endpoints under repeatable conditions. What it could not yet explain was which real usage pattern those requests represented.

I extended it with roles, sessions, think time, read and write journeys, and explicit cleanup rules. Before I obtained performance numbers worth discussing, the new journeys exposed authentication, API-contract, cleanup, and rate-limit problems. That is the central result: a journey must be valid before its latency can mean anything.

What the existing suite already did well

The starting point was useful and technically sound. It already measured:

• p50, p95, and p99 latency instead of relying on averages alone,
• throughput and error rates,
• warm-up phases,
• stepped load,
• and repeatable reports for comparing changes.

P50 is the value below which 50 percent of measured response times fall. P95 and p99 provide the corresponding boundaries for 95 and 99 percent. A warm-up phase lets the system handle a small amount of traffic before the measured phase starts.

These tests answered a clearly scoped question: How do defined endpoints behave under controlled, repeatable load? They helped me detect regressions, compare changes, and apply pressure to individual endpoints.

They did not answer how people actually use the system. A person rarely calls the same endpoint continuously for a minute. They sign in, load a list, open a detail view, read, perhaps comment, and move between contexts. Different roles see different data and can perform different actions.

That difference between endpoint traffic and user behavior became the starting point for the next version of the suite.

What I decided to add

My inventory resulted in five extension areas:

Think time means deliberately waiting between actions instead of letting a virtual user send requests as quickly as possible.

The implemented slice is deliberately narrow. It adds a reviewable journey model, but it has no telemetry-derived workload target and no continuous resource correlation across every relevant system layer.

The role-based journeys

I created three read-oriented journeys:

• A regular user verifies the session, loads an issue list, and opens an issue.
• An agent additionally reads comments and activity.
• An administrator additionally requests administrative overviews and health information.

The final low-load comparison used a 5:3:1 mix of regular-user, agent, and administrator sessions. This was the reduced configuration after a sixth regular-user login received 429. Each session signed in separately, verified the role returned by the server, waited between actions, and started during a ramp-up period rather than at the same instant.

The runner extends an existing PowerShell performance suite; it was not selected as a claim that PowerShell measures better than k6 or another load-testing tool. Each virtual session owns one .NET HttpClient and cookie handler, reuses that client during its journey, and records request duration after the full response body is read. Percentiles use the nearest-rank calculation. This keeps the browser-style cookie flow close to the existing automation, but it also means client and network time are part of the measured latency.

The mix describes sessions, not people. The runner takes one credential pair per role, so the five regular-user sessions reuse one regular-user account, the three agent sessions reuse one agent account, and the administrator has one account. That is a useful low-load session comparison, but not a simulation of nine independent identities.

I also separated read and write tests. The write journey creates an issue, updates it, adds a comment, verifies the result, and removes the generated data again. Because it changes state, it needs stronger safety controls than the read-only journey.

Functional preflight before performance measurement

I used a staged process:

1. Prove that a journey is functionally and technically valid.
2. Check that it remains stable when sessions run in parallel.
3. Only then interpret latency, throughput, and error rate.
4. Use system and resource metrics when investigating causes.

This distinction matters. Preparing a representative performance journey can reveal functional and technical contract problems, but that does not turn every 401 or 422 into a performance finding.

A successful login was not yet a usable session

The login request succeeded, but the following request for the current session returned 401 Unauthorized for every role. The credentials were correct. The test program did not yet reproduce the complete request context of a stateful browser session.

The application uses Laravel 13 and Sanctum for cookie-based authentication. Its documentation requires stateful SPA requests to use the same top-level domain and to send Accept: application/json plus Referer or Origin; the stateful-domain configuration decides which requests may use session cookies. The runner already sent Accept: application/json but was missing suitable Origin and Referer values. Once I added them, role verification succeeded.

A successful login status therefore did not prove that the resulting session was usable. If a journey represents a browser session, authentication and request context are part of its contract.

Safe writes and cleanup on failure paths

The write journey revealed more contract details. A token read before login produced 419 on the first later mutation. The runner now reads the XSRF cookie again before every non-GET request. The login handler regenerates the session ID, but I did not establish whether that rotation or another cookie change caused the earlier token to fail; I treat the re-read as an observed compatibility requirement, not an explanation of Sanctum internals.

The next attempt reached application validation and returned 422 Unprocessable Entity: the creation payload did not yet fully match the current API contract. The update operation also required more fields than I had initially assumed.

Again, these were not performance bottlenecks. But a simplified journey that bypasses real authentication or API contracts measures a path that the application does not actually execute.

The most useful discovery happened on a failure path:

1. The test created an object successfully.
2. A later update failed.
3. The normal control flow never reached cleanup.
4. The generated test artifact remained in the system.

That made cleanup an explicitly tested property of the journey. Two conditions are equally important:

1. The test must remove its own data even after a failed run.
2. It must never delete data that it cannot prove belongs to that run.

The safe control flow needs to retain two independent outcomes: the journey failure and a possible cleanup failure.

primary_error = none
cleanup_error = none

try:
    artifact = create(payload_with_unique_run_marker)
    remember_in_run_manifest(artifact.id)
    update_and_verify(artifact)
catch error:
    primary_error = error
finally:
    try:
        if artifact exists and
           manifest_contains(artifact.id) and
           artifact.project == dedicated_test_project and
           artifact.run_marker == current_run_marker:
            delete_and_purge(artifact)
    catch error:
        cleanup_error = error

if primary_error exists:
    raise primary_error, with cleanup_error attached
if cleanup_error exists:
    raise cleanup_error

In the actual runner, writes are restricted to a dedicated test project. Every generated object receives a unique run marker. A manifest stores only the identifiers and states created by that run. Immediately before deletion, the runner verifies the identifier, project, and marker again. Remote writes also require a separate acknowledgement.

The current runner records the original journey error and cleans up the owned artifact when that cleanup succeeds. It also provides a manifest-bound Cleanup action for recovery. A hard process abort, a lost create response, or a cleanup failure still requires an explicit recovery sweep; the runner does not yet report both a primary and cleanup error as separate terminal results. A regression test explicitly covers this sequence:

create succeeds → update fails → soft delete → permanent removal

A green run that silently leaves artifacts behind is not truly green. A failed run must not respond by deleting data it cannot prove it owns.

The 429 was a finding about identity, not just a blocker

During the first 6:3:1 parallel comparison attempt, the sixth regular-user login returned 429 Too Many Requests. Journeys that were already authenticated continued without errors.

The first mix exceeded a configured login limiter. The final 5:3:1 configuration made this comparison executable, but it is a calibration choice rather than evidence for a user-volume or identity model.

I excluded that run from the comparison and reduced the number of sessions for the affected role. That was a reasonable calibration step, but it was not the main lesson and it is not the final model for a larger test.

The more important conclusion is this: virtual sessions and independent user identities are not interchangeable. Several sessions using one account may primarily test repeated logins for one principal, including the protections attached to that identity. That is not automatically representative of several people signing in.

For future tests, the number and reuse of identities must therefore be part of the workload definition. Login traffic and already authenticated activity may also need separate scenarios. Reducing sessions made this comparison executable; it did not make the underlying modeling question disappear.

What workload model did I actually test?

The runner uses a closed workload model. Each virtual session waits for a response and its think time before starting the next action. If the system responds more slowly, the session completes fewer iterations and the offered request rate can fall.

This is appropriate for comparing complete user journeys. It does not maintain a fixed arrival rate when the system slows down, so it cannot answer a saturation question. The k6 documentation on open and closed workload models makes this throughput dependency explicit.

An open model starts new iterations according to an arrival schedule independent of how long previous iterations take. That is the appropriate next experiment for questions such as: What happens when the system must sustain a specified arrival rate as latency increases? When a test claims to model such a schedule but only sends the next request after the previous response, its percentile view can also suffer from coordinated omission; Gil Tene's wrk2 notes explain that distinction. This closed-loop comparison makes no fixed-arrival claim and does not quantify that effect.

Neither model is universally “more realistic.” They answer different questions. For this initial journey comparison I used the closed model; I did not use it to infer saturation behavior or production capacity.

A low-load engineering comparison—not a capacity result

After functional preflight and parallel validation, I ran three identically configured read-only measurements.

Each virtual session executed its journey for 60 seconds with one second of think time between actions. Sessions started gradually during a 15-second ramp-up. Because every session had its own 60-second window, the complete runner took about 78 seconds from the first start to final completion.

The one-second think time is fixed and identical for every action. That is a deterministic pacing control, not a claim about natural human timing. Together with the fixed journeys and duration, it explains why each retained run produced 513 calls. The runner also has no separate journey warm-up phase; ramp-up staggers session starts but does not replace warm-up.

The reported throughput divides measured API calls by that complete runtime, including ramp-up and final completion. It is therefore a whole-run comparison value, not a steady-state offered load. CSRF initialization and login requests were not counted as performance calls, while the following role verification was counted.

The following table is an aggregate across all three read-only roles and paths. The separate write journey is not included.

Across the three runs, the median run-level p95 was 137.5 milliseconds. The run-level p95 values ranged from 136.8 to 137.7 milliseconds, and median throughput was 6.55 requests per second. None of the runs produced HTTP or worker errors.

The p99 figures are included for continuity with the existing suite, not for a stable tail estimate: with 513 observations, the 99th percentile is determined by only a handful of values. The visible spread from 182.1 to 206.2 milliseconds is descriptive. The retained runs contained only successful HTTP events. In a failed run, the runner records the non-success response before ending that worker, so the status mix and percentiles must be read together rather than treated as a comparable baseline. The internal reports retain a status mix and p95 for every role and path; I have left those low-volume diagnostic values out of this article.

I used 500 milliseconds p95 as a provisional technical comparison threshold. It already existed in the previous suite for authenticated reads, but it was not derived from production telemetry, a user study, or an approved capacity requirement. Staying below it means only that these runs met this technical starting value under the described configuration.

Reproducible for whom?

The three runs were repeatable inside the defined test setup: same runner, three role accounts reused across the 5:3:1 sessions, fixed timing, target environment, dataset, and journeys. That is internal repeatability, not independent reproducibility for a reader.

I have named the runner approach, role mix, timing, request accounting, and journey structure so that the experiment can be assessed. I have not published the source code, fixtures, exact dataset volume, infrastructure topology, or the generator's network position. These omissions make the article an implementation report to adapt, not an independently reproducible benchmark; the cleanup pseudocode is transferable, while the latency values are not.

What snapshots can and cannot show

I captured load-generator CPU load and free memory, plus application-health signals, before and after the comparison. At those two moments, the generator did not show unusual CPU or memory utilization. The report did not continuously capture worker-pool saturation, database connection-pool saturation, or provider and network telemetry, so it cannot rule out short-lived peaks during the runs.

Snapshots provide context, but they cannot establish causality. Good response times do not prove that the overall system is healthy. Likewise, a health signal observed at the same time would not prove that it caused a latency value. Before/after values show temporal coincidence and change, not cause and effect.

Without continuous resource data across the relevant layers, I can detect a regression more readily than I can explain its cause. Database connections, queues, worker capacity, and other system signals remain part of the next stage.

What I learned

Seven lessons stand out for me:

1. Before extending a suite, state clearly which question it already answers well.
2. A journey must be functionally valid before its performance metrics are meaningful.
3. Roles, sessions, permissions, think time, and read/write paths shape the workload model.
4. Virtual sessions, login attempts, and independent identities are different dimensions.
5. Write tests need explicit ownership, cleanup, and a manifest-bound recovery path.
6. Fixed think time makes a useful calibration control, not a faithful model of human timing.
7. Closed and open workload models answer different questions; a short closed-loop comparison is not a capacity result.

The original suite was not wrong. Its metrics, warm-up phases, and load stages were the foundation for this work. The important change was adding a clearer, testable description of the behavior behind the requests.

For me, a more representative performance test is not a particular tool or the largest possible script. It is a reviewable model of who does what, under which conditions, for a defined slice of usage. That model can include roles, sessions, permissions, think time, read and write paths, failure paths, rate limits, ownership of test data, cleanup rules, measurement boundaries, and missing telemetry.

Including those elements does not make a test perfectly realistic. It makes it more representative of the usage slice it claims to model—and makes its blind spots easier to see.

AI assistance disclosure

This article is based on my own test implementation, measurements, and conclusions. I used AI assistance for editorial restructuring and the English adaptation, reviewed the technical claims against the underlying reports and scripts, and remain responsible for the final text.

Sources

• Daniel Balcarek (2026): API Performance Testing: How to Design Realistic Tests, accessed September 21, 2026.
• Laravel (2026): Laravel Sanctum, accessed September 23, 2026.
• Grafana Labs (2026): Open and closed models, accessed September 21, 2026.
• Gil Tene: wrk2 README, accessed September 23, 2026.