Threads, Pools, Promise.all: Why Your Postgres Falls Over First (Spring Boot vs Node)

Two recent DEV posts describe the same incident from opposite sides of the stack. In one, a Spring Boot service learned that threads are not connections. In the other, a Node service found out why await Promise.all() can crash your database. There's also a write-up of a connection pool that collapsed twice in a day, where restarting made it worse.

The pattern is the same everywhere. Concurrency in your app has become almost free: virtual threads in Java 21+, Promise.all in Node. Concurrency in your database has not. Postgres has a hard max_connections, and each connection is a real backend process on the server. The app scales, the pool saturates, and the database (or the pool in front of it) is the first thing to fall over.

Instead of arguing about it, I measured it.

The setup

• PostgreSQL 18.4, local, with max_connections = 25 (deliberately small, so the limits show up fast). As usual, 3 slots are reserved for superusers, which leaves 22 for the app role.
• Java 21 with HikariCP 7.0.2 (the version Spring Boot 4.1.1 manages) and the PostgreSQL JDBC driver.
• Node 20 with node-postgres (pg) and p-limit.
• The workload is a burst of 1,000 concurrent queries, each select pg_sleep(0.05): a 50 ms query, roughly a real indexed lookup plus some network.
• The pool max is 10 on both stacks, which is the default for both HikariCP and node-postgres.

These are runs on one machine, not a benchmark suite. The shape of the results is the point, not the exact milliseconds.

The math nobody does first

A pool of 10 connections running 50 ms queries can serve at most:

10 connections / 0.05 s per query = 200 queries per second

So 1,000 queries need at least 5 seconds, however many threads or promises you throw at them. Everything below is about what happens to the requests that are waiting during those 5 seconds.

Result 1: instances × pool size must fit max_connections

The first failure doesn't even involve a burst. You scale your Node API to three instances, each with the default max: 10:

const pools = [1, 2, 3].map(() => new pg.Pool({ ...cfg, max: 10 }));
await Promise.allSettled(Array.from({ length: 300 }, (_, i) => pools[i % 3].query(QUERY)));

3 instances x max=10: ok=57/300
  'remaining connection slots are reserved for roles with the SUPERUSER attribute': 108
  'sorry, too many clients already': 135

That's 30 wanted connections against 22 available. Most queries didn't just wait: they failed, because Postgres refuses connections beyond its limit and pg passes the error to your query.

The budget has to cover everything that connects:

instances × pool max  +  migrations  +  cron/workers  +  admin/monitoring  ≤  max_connections − reserved

This is exactly what bites during autoscaling and rolling deploys, when old and new instances are briefly up at the same time. It's also why hosted Postgres providers put a pooler in front of the database. Supabase, for example, offers Supavisor in session mode (port 5432) and transaction mode (port 6543). Transaction mode lets many short-lived clients share few connections, but per the Supabase docs it doesn't support prepared statements or session state (SET, advisory locks, LISTEN/NOTIFY). With JDBC, for instance, they list prepareThreshold=0. Know which mode your connection string uses.

Result 2: the same burst, both stacks

Spring Boot / HikariCP (connection timeout = how long a caller waits for a free connection):

scenario                              ok   poolTimeout  rejected  p50     p99     wall
platform threads (200), timeout 1s   576   424          0         1742ms  3013ms  3089ms
virtual threads, timeout 1s          175   825          0         1003ms  1009ms  1063ms
virtual + bulkhead(20, 100ms)         28     0        972          195ms   266ms   319ms
virtual threads, timeout 30s (dflt) 1000     0          0         2695ms  5173ms  5220ms

Node / node-postgres:

Promise.all x1000, max=10, acquire timeout 1s:  ok=200/1000  ('timeout exceeded when trying to connect': 800)
Promise.all x1000, max=10, no timeout:          ok=1000/1000 in 5089 ms   peak waitingCount = 990
p-limit(10) x1000, max=10, acquire timeout 1s:  ok=1000/1000 in 5100 ms   peak waitingCount = 0
p-limit(10) + shed when queue > 100:            ok=110, busy(503)=890 in 583 ms

What this tells us:

1. Virtual threads removed the queue you didn't know you had. With a fixed pool of 200 platform threads (Tomcat's default server.tomcat.threads.max is 200), at most 200 requests were waiting on Hikari at once. The rest waited in the executor's queue, where Hikari's timeout doesn't apply. With virtual threads, all 1,000 requests hit the pool at the same moment (998 threads waiting at peak), and with a 1 s timeout, 82% failed. Turning on spring.threads.virtual.enabled=true is great for I/O-bound apps, but it moves the bottleneck straight to the pool.
2. Long timeouts "fix" errors by hiding latency. With Hikari's default 30 s connection timeout, or with node-postgres's default connectionTimeoutMillis: 0 (wait forever), everything succeeded, with p99 above 5 s. That's fine for a batch job, but for a mobile client with a 10 s HTTP timeout and a user tapping "retry", it's a slow outage.
3. A limiter doesn't make anything faster. p-limit(10) took the same ~5.1 s as the unbounded Promise.all; the pool's waiting count just dropped from 990 to 0. What it gives you is control: the queue now lives in your code, where you can measure it and cap it.
4. Load shedding is the only option that keeps latency bounded. The bulkhead (Java) and the queue cap (Node) rejected most of the burst within ~0.3–0.6 s, with a clean "busy" error the client can retry with backoff. Whether that's right depends on the endpoint. For a user-facing read, failing fast is usually better than a 5-second spinner.

The Node pattern

import pg from 'pg';
import pLimit from 'p-limit';

export const pool = new pg.Pool({
  connectionString: process.env.DATABASE_URL,
  max: Number(process.env.PG_POOL_MAX ?? 10),   // per process!
  connectionTimeoutMillis: 2_000,               // default is 0 = wait forever
  idleTimeoutMillis: 10_000,
});

const limit = pLimit(Number(process.env.PG_POOL_MAX ?? 10));
const MAX_QUEUE = 100;                          // beyond this, shed load

export class BusyError extends Error { status = 503; }

export function withDb(fn) {
  if (limit.pendingCount >= MAX_QUEUE) {
    return Promise.reject(new BusyError('database busy, retry later'));
  }
  return limit(() => fn(pool));
}

// usage
await Promise.all(ids.map(id => withDb(db => db.query('select ... where id = $1', [id]))));

Map BusyError to 503 with a Retry-After header in your error middleware. And when you see Promise.all(items.map(query)) over a list of user-controlled size, that's a code-review comment.

The Spring Boot pattern

# application.properties
spring.threads.virtual.enabled=true
spring.datasource.hikari.maximum-pool-size=10
spring.datasource.hikari.connection-timeout=2000

@Component
class DbBulkhead {
    private final Semaphore permits = new Semaphore(20);   // ~2x pool size

    <T> T call(Callable<T> work) throws Exception {
        if (!permits.tryAcquire(100, TimeUnit.MILLISECONDS)) {
            throw new ResponseStatusException(HttpStatus.SERVICE_UNAVAILABLE, "busy, retry later");
        }
        try { return work.call(); } finally { permits.release(); }
    }
}

The Semaphore logic is exactly what I measured above (in a plain Java harness against HikariCP). The Spring wiring is shown as a sketch. If you already use Resilience4j, its Bulkhead does the same job with metrics built in.

Two Spring-specific traps make pool exhaustion worse:

• Holding a connection while doing something slow. A @Transactional method that calls an external HTTP API (or an LLM) keeps its connection for the whole call. Do the slow call outside the transaction, then open a short transaction to write the result.
• Open Session in View. With Spring Data JPA, spring.jpa.open-in-view is enabled by default (Boot logs a warning about it), which can keep a connection tied to the request while the view renders. For APIs, set it to false.

How big should the pool be?

Usually smaller than you think. HikariCP's own pool sizing guide starts from connections ≈ (core_count × 2) + effective_spindle_count on the database server, and argues that a small pool with a queue in front of it beats a big pool. Practical steps:

1. Start from the database's capacity and the budget formula above, not from your thread count.
2. Divide by the maximum number of app instances you can run at once (including during deploys).
3. Load-test, and increase only if the database has CPU headroom while the pool is saturated.

The metrics to watch before it breaks

• Hikari (via Micrometer/Actuator): hikaricp.connections.pending (threads waiting), hikaricp.connections.active, hikaricp.connections.acquire (time to get a connection) and hikaricp.connections.timeout (a counter you want to stay at zero).
• node-postgres: pool.totalCount, pool.idleCount, pool.waitingCount, plus your limiter's pendingCount.
• Postgres: select state, count(*) from pg_stat_activity group by state;. Lots of idle in transaction means connections are held by code that isn't talking to the database.

Alert on waiting and acquire time, not just on errors. In every run above, the waiting count spiked long before the first error.

Takeaways

• Threads and promises are cheap; connections are a budget. Write the budget down.
• Virtual threads and Promise.all push all of your concurrency onto the pool. Put a limit in front of it.
• Pick a timeout on purpose: short for user-facing paths, longer for batch jobs.
• Shed load with a clear 503 rather than letting latency grow without limit.
• Measure pending/waiting, and look at pg_stat_activity before you resize anything.

Which default bit you first: Hikari's 30 seconds, or node-postgres's "wait forever"? Tell me in the comments.