One backend. One PostgreSQL database. An admin panel, a web app, and two mobile clients all hitting the same API.
It worked fine — until it didn't. p99 latency crept past 4 seconds, the job queue started backing up, and the instinct was to add more servers. That instinct is almost always wrong. Here's the actual sequence that fixed it, with the code — not just the theory.
Measure before you touch anything
Don't scale based on a feeling. Track these five things before changing infrastructure:
- CPU/RAM on your app servers
- API latency — p95 and p99, not the average (a 200ms average can hide a p99 of 4s)
- Query latency, active connections, lock contention
- Error rate per endpoint
- Queue depth and job processing time
A basic dashboard covering these tells you whether the problem is application, database, or queue — before you spend money on the wrong layer.
Make the application do less work first
This is the cheapest scaling win, and the one most often skipped.
Cursor-based pagination instead of large offsets
app.get('/api/orders', async (req, res) => {
const { cursor, limit = 25 } = req.query;
const query = cursor
? `SELECT * FROM orders WHERE id > $1 ORDER BY id LIMIT $2`
: `SELECT * FROM orders ORDER BY id LIMIT $1`;
const params = cursor ? [cursor, limit] : [limit];
const { rows } = await pool.query(query, params);
res.json({ data: rows, nextCursor: rows.at(-1)?.id ?? null });
});
An indexed WHERE id > cursor avoids the cost of skipping hundreds of thousands of rows just to discard them — unlike OFFSET 50000, which still scans everything before it.
Kill N+1 queries
Fetching 100 orders, then querying the customer for each one separately, turns 1 query into 101:
-- Before: 1 query + 100 more in a loop
SELECT * FROM orders LIMIT 100;
-- After: 1 query
SELECT orders.id, orders.total, users.name
FROM orders
JOIN users ON users.id = orders.user_id
LIMIT 100;
The dangerous part: N+1 is invisible with 10 records and painful with 10,000.
PostgreSQL: indexes, EXPLAIN ANALYZE, connection pooling
CREATE INDEX idx_orders_user_id ON orders(user_id);
EXPLAIN ANALYZE
SELECT * FROM orders WHERE user_id = 123;
EXPLAIN ANALYZE shows you sequential scans, bad row estimates, and unnecessary sorts — don't guess why a query is slow, look.
Five app servers each opening 20 raw connections is 100 connections fast, and Postgres has a hard limit. Pool them:
const { Pool } = require('pg');
const pool = new Pool({
host: process.env.DB_HOST,
max: 20, // per instance, not global
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 2000,
});
For anything beyond a handful of app instances, put PgBouncer in front of Postgres so pooling is handled centrally instead of per-process.
Redis: cache-aside, not a database
Cache-aside is the default pattern: check Redis → fall back to Postgres on a miss → write through → return.
const redis = require('ioredis')();
async function getDashboardSummary(userId) {
const cacheKey = `dashboard:${userId}`;
const cached = await redis.get(cacheKey);
if (cached) return JSON.parse(cached);
const summary = await computeDashboardSummary(userId); // hits PostgreSQL
await redis.set(cacheKey, JSON.stringify(summary), 'EX', 60); // 60s TTL
return summary;
}
The hard part isn't reading from cache, it's invalidation — when the underlying data changes, update or evict the key, or you'll serve a stale permission set long after it changed.
Redis also isn't your source of truth; use it for caching, rate limiting, and sessions, not durable business data.
Background jobs, with real retry logic
Your API shouldn't block on report generation, PDF exports, or bulk email. Using BullMQ:
const { Queue, Worker } = require('bullmq');
// Producer — API route
const reportQueue = new Queue('reports', { connection: redisConnection });
app.post('/api/reports', async (req, res) => {
const job = await reportQueue.add('generate-report', { userId: req.user.id });
res.status(202).json({ status: 'processing', jobId: job.id });
});
// Worker — separate process
new Worker('reports', async (job) => {
const { userId } = job.data;
await generateAndStoreReport(userId);
}, {
connection: redisConnection,
attempts: 3,
backoff: { type: 'exponential', delay: 5000 },
});
Retries alone aren't enough — a worker can crash after sending an email but before marking the job done, and the retry will fire again:
new Worker('invoices', async (job) => {
const { invoiceId } = job.data;
const alreadySent = await redis.get(`invoice-sent:${invoiceId}`);
if (alreadySent) return; // retried, but the email already went out
await sendInvoiceEmail(invoiceId);
await redis.set(`invoice-sent:${invoiceId}`, '1', 'EX', 86400);
});
Without that idempotency check, a retried job means a duplicate invoice email in someone's inbox.
Rate limiting, tiered by plan
An endpoint that normally sees 100 req/min can suddenly see 20,000 — spike, bad integration, or abuse. Enforce limits consistently across instances via Redis:
const rateLimit = require('express-rate-limit');
const RedisStore = require('rate-limit-redis');
const limiter = rateLimit({
store: new RedisStore({ sendCommand: (...args) => redis.call(...args) }),
windowMs: 60 * 1000,
max: (req) =>
req.user?.plan === 'enterprise' ? 5000 :
req.user?.plan === 'paid' ? 1000 : 100,
});
app.use('/api/', limiter);
Gotchas that don't show up until production
- N+1 queries are invisible at small scale. They only hurt once a customer has thousands of rows, by which point they're already in production.
- Forgotten cache invalidation serves stale data silently — a user balance or permission set that's wrong for minutes or hours with no error thrown anywhere.
-
In-memory session state breaks the moment you add a second node.
const sessions = {}living in process memory means a request routed to a different instance won't see it. Sessions, cache, and uploaded files need to live in Postgres, Redis, or object storage — not app memory — before you run more than one instance. - Jobs without idempotency duplicate side effects on retry, not just waste compute — a resent email or a double-charged webhook is a worse bug than a slow query.
Scale infrastructure last, not first
Once the application and database are genuinely optimized: vertical scaling (bigger machine) is simple but has a ceiling. Horizontal scaling (more machines behind a load balancer) needs a stateless app first — shared sessions, shared cache, shared storage — or you'll get inconsistent behavior depending on which node a request hits.
Skip Kubernetes and microservices until there's a concrete reason — a workload that needs to scale independently, or a team boundary large enough to justify separate deploy lifecycles.
Neither one makes a slow query fast or an unindexed table efficient.
Takeaway
Measure → Optimize → Test → Scale
Most of the wins above cost nothing but engineering time — pagination, an index, a cache with a sane TTL, moving slow work off the request path.
Infrastructure is the last lever to pull, not the first.
I'm Haseeb — I build production SaaS backends (Node.js, PostgreSQL, Redis, Stripe) at Seebify. Full write-up with the business-side reasoning and a complete stage-by-stage scaling roadmap here.
