A few months into my first backend role, I sat in on a design review where a senior engineer said “we should just shard on user ID” and everyone nodded like it was obvious. I nodded too. I had no idea what sharding actually meant beyond “splitting a database, I guess.” I went home that night and spent three hours untangling it from a stack of blog posts that all assumed I already knew what a partition key was.
That happened to me more times than I’d like to admit. Not because the concepts are hard — most of them aren’t — but because nobody sits you down and explains them in order. You pick up “cache” from one article, “consistent hashing” from a conference talk you half-watched, and “idempotency” from a Slack thread where someone else got paged at 2 a.m. because a retry double-charged a customer.
So this is the list I wish I’d had. Not textbook definitions — I’ll link to those if you want them — but the version I’d explain to a friend over coffee, with the mistakes and “wait, why does that matter” moments left in. I’ve grouped it the way these things actually show up in a real system: the stuff that sits between a user and your server, how you store data, what happens once you have more than one machine, how services talk to each other, and how you keep the whole thing standing up under load.
It’s long. Bookmark it and come back to the sections you need.
Foundation Concepts
This is the layer most people skip because it feels “too basic” for an interview. It’s also the layer where I’ve seen the most experienced engineers get tripped up, because these words get used loosely in everyday conversation and nobody double-checks the precise meaning until something breaks.
1. Client
A client is whatever initiates a request — a browser, a mobile app, a CLI tool, even another backend service calling out to yours. The thing to internalize early: “client” isn’t a type of device, it’s a role. Your API server is a client the moment it calls a third-party payment provider.
Browser → "give me /home"
Server → "here's /home"
2. Server
A server receives a request, does some work, and responds. What confused me for a while was assuming “server” meant one physical machine. In practice it usually means a process — you can run five server processes on one box, or one logical server spread across fifty boxes behind a load balancer.
3. Load Balancer
A load balancer distributes incoming traffic across multiple servers so no single machine gets overwhelmed. The part people gloss over is health checking — a decent load balancer is constantly pinging your servers and quietly pulling the unhealthy ones out of rotation. The first time I watched this happen live, during a deploy that briefly broke one instance, I understood why everyone insists on running more than one server even for “small” apps.
Users
|
Load Balancer
|
Server 1 Server 2 Server 3
If you want to see this on your own machine, Nginx as a reverse proxy/load balancer takes about ten minutes to set up locally with Docker:
# nginx.conf
upstream backend {
server app1:3000;
server app2:3000;
}
server {
listen 80;
location / {
proxy_pass http://backend;
}
}
4. Horizontal Scaling
Adding more machines instead of making one machine bigger. This is the default answer in most modern architectures because it also buys you redundancy — lose one server, the other four keep serving.
5. Vertical Scaling
Making one machine bigger — more RAM, more CPU. It’s the lazy-but-honest first move for a lot of early-stage products, myself included. I’ve bumped a single Postgres instance from 4GB to 32GB of RAM more than once instead of dealing with replication, and it bought real time. Eventually you hit a ceiling, and worse, that one machine is now a single point of failure for everything.
6. Stateless Service
A service that doesn’t remember anything about you between requests — every request carries whatever context it needs (a token, an ID, whatever). This is the property that makes horizontal scaling easy: any server can handle any request, so the load balancer doesn’t have to think.
7. Stateful Service
The opposite — the server remembers something, like a session or a game state, and future requests may need to land on that same server. I avoided building anything stateful for years because “stateless is best practice,” until I worked on a real-time multiplayer feature where it was genuinely unavoidable. Stateful isn’t wrong, it’s just harder to scale and recover from failure — you have to be deliberate about it.
8. API Gateway
A single entry point that sits in front of multiple backend services and routes requests to the right one, often also handling auth, rate limiting, and logging so individual services don’t each reinvent that logic.
Mobile App
|
API Gateway
|
User Service / Order Service / Payment Service
9. Reverse Proxy
Sits between users and your servers, forwarding requests without the client knowing (or caring) which backend actually handled it. Nginx and HAProxy are the usual suspects. In practice, a lot of small teams use “reverse proxy” and “load balancer” interchangeably, which is fine day-to-day, but a reverse proxy does more — TLS termination, compression, hiding your internal topology.
10. CDN (Content Delivery Network)
A network of geographically distributed servers that cache static assets (images, video, JS, CSS) close to the user. The first time I checked our analytics after putting images behind a CDN, the page load improvement for users outside our home region was bigger than any code optimization I’d shipped that quarter.
11. DNS (Domain Name System)
Translates a name like example.com into an IP address like 192.0.2.1. Think of it as the internet's phone book — nobody memorizes phone numbers, everyone memorizes names.
12. SSL/TLS
Encrypts traffic between client and server so a network eavesdropper can’t read it in plain text. “SSL” is the term everyone still says out of habit; what’s actually running under the hood in any modern system is TLS. If you’re serving anything over plain HTTP in production in 2026, that’s worth fixing today, not next sprint.
Databases & Storage
Nothing exposed my gaps faster than databases. I could write a SELECT query in my sleep and still not have a clear answer for "why did we pick Postgres over Mongo here."
13. Database
A system for storing, organizing, and retrieving data — the alternative being “a folder of text files,” which does not scale past a weekend project.
14. SQL Database (Relational Database)
Data lives in tables with rows and columns, and tables relate to each other through keys.
Users
+----+--------+
| ID | Name |
+----+--------+
| 1 | Alex |
| 2 | Emma |
+----+--------+
Orders
+-----+---------+---------+
| ID | User ID | Product |
+-----+---------+---------+
| 101 | 1 | Laptop |
| 102 | 2 | Phone |
+-----+---------+---------+
SQL databases earn their keep when your data has real relationships and you need transactions you can actually trust. Postgres and MySQL are the two I reach for by default.
15. NoSQL Database
Data doesn’t have to fit a fixed table shape — documents, key-value pairs, graphs, wide columns. Useful when your data’s shape changes often or you’re dealing with volume that makes rigid schemas painful. MongoDB, Redis, and Cassandra are the common names here. I’ll admit I over-used MongoDB early in my career because it felt “flexible,” and later spent a painful week adding data validation because that flexibility had let inconsistent documents pile up unnoticed.
16. Schema
The blueprint of your database — what tables exist, what columns they have, what type each column is.
Users
-------------------
id integer
name string
email string
age integer
17. Primary Key
A unique identifier for each row. No duplicates allowed.
18. Foreign Key
A column that stores another table’s primary key, creating a relationship between the two.
Orders
OrderID UserID
101 1
19. Index
A structure that lets the database jump straight to matching rows instead of scanning the whole table. It’s the book-index analogy every article uses because it’s exactly right: instead of reading every page, you flip to the page you need. The tradeoff nobody mentions enough — every index you add makes writes a little slower, because the index has to be updated too. I’ve seen tables with a dozen “just in case” indexes that quietly tanked write throughput.
20. ACID
Four guarantees that make transactions trustworthy:
A - Atomicity : all operations succeed, or none do
C - Consistency : data stays valid according to your rules
I - Isolation : concurrent transactions don't step on each other
D - Durability : once committed, it survives a crash
This is why banks don’t run on eventually-consistent NoSQL stores for the ledger itself.
21. Transaction
A group of operations treated as one atomic unit. Deduct ₹1000 from Account A, credit it to Account B — either both happen or neither does. Without a transaction wrapping that, a crash between the two steps means money vanishes.
22. Replication
Keeping multiple copies of the same database, usually with one primary handling writes and replicas serving reads. It buys you both availability (a replica can take over) and read throughput.
23. Sharding (Partitioning)
Splitting one large database into several smaller ones, each holding a slice of the data.
Server 1 -> Users A-H
Server 2 -> Users I-P
Server 3 -> Users Q-Z
This is the concept that started this whole article — the one I had to learn the hard way after nodding along in that design review. It’s simple in diagram form and genuinely tricky in practice, mostly because of what’s next.
24. Shard Key
The value used to decide which shard a piece of data lands on — often something like user_id % 3. Picking a bad shard key is the single most common sharding mistake. Pick something that isn't evenly distributed and you've just built yourself a hot partition.
25. Hot Partition
One shard gets disproportionately more traffic than the others. The classic example: shard by user ID, then a celebrity account with 200 million followers pushes every request for that account onto one unlucky server while the rest sit idle.
26. Read Replica
A copy of the database used only for reads, taking load off the primary.
Writes -> Primary Database -> Replica 1 / Replica 2 / Replica 3
Reads can lag slightly behind writes on the primary — which is fine for a news feed, and very much not fine for “did my payment go through,” so know which reads you’re routing to a replica.
27. Write-Ahead Log (WAL)
Changes get written to a log before they’re applied to the actual data. If the database crashes mid-update, it replays the log on restart and recovers cleanly instead of losing the write.
28. Cache
Frequently accessed data stored somewhere much faster than your primary database — usually in memory.
Application -> Cache -> Database
29. Cache-Aside
The most common caching pattern: check the cache first, and only hit the database on a miss, then populate the cache for next time.
check cache
found? -> yes -> return it
-> no -> read database
-> write to cache
-> return it
Locally, this is genuinely a ten-minute setup with Redis and Docker — no paid caching service required:
docker run -d --name redis-cache -p 6379:6379 redis:7-alpine
import redis
import json
r = redis.Redis(host="localhost", port=6379)
def get_user(user_id):
cached = r.get(f"user:{user_id}")
if cached:
return json.loads(cached)
user = db.query_user(user_id) # your real DB lookup
r.setex(f"user:{user_id}", 600, json.dumps(user)) # TTL = 10 min
return user
30. Cache Eviction
When the cache fills up, something has to go. Two common policies:
LRU - evict the Least Recently Used item
LFU - evict the Least Frequently Used item
31. TTL (Time To Live)
How long a cached item stays valid before it expires and gets refreshed from the source of truth. Short TTL = fresher data, more database load. Long TTL = faster responses, more risk of showing stale data. There’s no universally “correct” number — it’s a judgment call per piece of data, and I’ve gotten it wrong in both directions.
32. Cache Stampede
Thousands of requests for the same popular item all hit right as its cache entry expires, and they all fall through to the database at once.
cache expires -> 10,000 requests -> database overloaded
The fix that actually worked for us was staggering TTLs slightly (so not everything expires at the exact same millisecond) plus locking so only one request repopulates the cache while the rest wait.
33. Object Storage
Built for large, mostly-static files — images, videos, PDFs, backups. Amazon S3 is the household name; Cloudflare R2 and self-hosted MinIO are worth knowing if you want to avoid vendor lock-in or just want a local dev setup:
docker run -d -p 9000:9000 -p 9001:9001 \
-e "MINIO_ROOT_USER=admin" -e "MINIO_ROOT_PASSWORD=password123" \
minio/minio server /data --console-address ":9001"
34. Blob Storage
“Blob” stands for Binary Large Object — in practice this is just another name for the same idea as object storage. Different clouds, different marketing terms, same concept.
Distributed Systems
This is the section where I stopped being able to reason about the system by just imagining “one computer, but bigger.” Once you have multiple machines, failure stops being an edge case and becomes a constant you design around.
35. Distributed System
Multiple computers cooperating to behave like one application — Netflix, WhatsApp, Google Search. You get scalability and resilience, and in exchange you inherit network failures, clock differences, and the coordination problem.
36. CAP Theorem
During a network partition, you can’t have both perfect consistency and full availability — you have to pick.
C - Consistency : everyone sees the same data
A - Availability : the system keeps responding
P - Partition Tolerance : it keeps working despite network splits
Partition tolerance isn’t really optional — networks fail, period — so CAP in practice is a C-vs-A decision during those failures. I used to think CAP was a permanent architectural label (“we’re an AP system”), but it’s really about what you choose in the moment things go wrong, and different parts of the same system can choose differently.
37. Consistency
Every user sees the latest data, everywhere, immediately. Change your username, everyone sees the new one right away. Guaranteeing this across distributed servers costs coordination time.
38. Availability
The system keeps responding even when parts of it fail. Lose one server, users barely notice because others pick up the slack.
39. Eventual Consistency
Updates propagate gradually — for a few seconds, different users might see different versions of the same data, but everyone converges eventually. Change a profile picture and one friend sees it instantly while another sees the old one for a moment. It’s a deliberate tradeoff for speed and scale, not a bug.
40. Strong Consistency
The opposite tradeoff — every read reflects the latest write, no matter which server answers, at the cost of coordination overhead. This is what you want for account balances, not for “like” counts.
41. Linearizability
The strongest consistency guarantee — the system behaves as if every operation happened one at a time in a single global order, even though many machines are actually involved. Useful when correctness matters more than raw speed.
42. Network Partition
Part of your system loses the ability to talk to the rest, usually due to a network issue rather than a machine dying outright. This is the exact scenario CAP theorem is built to reason about.
43. Consensus
Getting multiple servers to agree on one decision — who’s the new leader after a crash, what the next committed value is. Raft and Paxos are the algorithms that solve this properly; I wouldn’t recommend hand-rolling your own unless you enjoy debugging split-brain at 3 a.m.
44. Quorum
The minimum number of nodes that must agree before an operation counts as successful — e.g., 3 out of 5 servers. Majority voting like this is what keeps a distributed system from committing conflicting writes.
45. Consistent Hashing
A technique for spreading data across servers so that adding or removing a server only reshuffles a small slice of the data, instead of nearly everything. Widely used in distributed caches and databases — it’s one of those ideas that sounds abstract until you see the alternative (naive modulo hashing) rebalance basically your entire dataset because you added one more node.
46. Leader Election
The process of picking one server to coordinate the group. If the leader dies, the rest elect a replacement automatically, no human paged at 3 a.m. required — assuming it’s implemented correctly.
47. Split Brain
Two servers both believe they’re the leader and both start accepting writes independently, producing conflicting data. This is exactly the failure mode that consensus algorithms and quorum rules exist to prevent, and exactly the failure mode you get when someone tries to skip them.
48. Vector Clock
A way to figure out the order of events across machines without relying on wall-clock time (which is never perfectly synced). Each node keeps its own counter, and comparing counters tells you what happened before what — useful when multiple users edit the same data at nearly the same instant.
49. Clock Skew
Different servers’ clocks drift slightly out of sync, even with regular synchronization. This is why distributed systems generally avoid trusting raw timestamps to determine ordering of events.
50. Idempotency
Doing the same operation twice produces the same result as doing it once. Click “Pay Now” twice because your connection lagged — an idempotent payment system charges you once, not twice. This is one of the ideas I underrated as a junior engineer and now consider close to non-negotiable for anything involving money or irreversible side effects.
51. Idempotency Key
A unique ID attached to a request so the server can recognize a retry and return the original result instead of processing it again.
def create_payment(request):
existing = db.find_by_idempotency_key(request.idempotency_key)
if existing:
return existing # already processed, return the same result
payment = process_payment(request)
db.save(payment, key=request.idempotency_key)
return payment
Communication Patterns
Once you have more than one service, “call it directly and wait” stops being the only option, and often stops being the right one.
52. Message Queue
A holding area for messages until a consumer is ready to process them. The sender doesn’t wait around — it drops the message and moves on. This decoupling is what lets one part of your system have a bad day without taking the rest down with it.
53. Producer
Whatever creates and sends a message — e.g., an Order Service publishing “order placed” onto a queue.
54. Consumer
Whatever reads and processes messages off the queue — e.g., a Payment Service picking up that “order placed” event.
55. At-Most-Once Delivery
The message is sent once, and if something fails along the way, it’s just gone. Fast, simple, and only acceptable where losing an occasional message genuinely doesn’t matter.
56. At-Least-Once Delivery
The system retries until delivery is confirmed, which means a message might arrive more than once. This is the default in most real systems, and it’s exactly why idempotency (see #50) isn’t optional — your consumer has to handle duplicates gracefully.
57. Exactly-Once Delivery
No duplicates, no losses. Genuinely hard to guarantee end-to-end in a distributed system; most “exactly-once” systems in practice are at-least-once delivery plus idempotent processing that makes duplicates harmless.
58. Dead-Letter Queue (DLQ)
A separate queue where messages land after repeatedly failing to process, instead of retrying forever and blocking everything behind them. Checking the DLQ regularly is one of those unglamorous habits that saves you from silently losing data for weeks.
59. Pub/Sub (Publish-Subscribe)
One producer publishes a message, and every subscriber gets their own copy — the producer never needs to know who’s listening. Common for notifications, analytics pipelines, and fanning one event out to several independent services.
60. Event Streaming
Events get appended to a continuous, ordered log rather than removed once consumed, so multiple consumers can read at their own pace and even replay history. Kafka is the platform most people mean when they say this. Locally:
docker run -d --name kafka -p 9092:9092 \
-e KAFKA_NODE_ID=1 \
-e KAFKA_PROCESS_ROLES=broker,controller \
-e KAFKA_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093 \
-e KAFKA_CONTROLLER_QUORUM_VOTERS=1@localhost:9093 \
apache/kafka:latest
61. Backpressure
A signal telling a fast producer to slow down because its consumer can’t keep up, preventing an unbounded pile-up of unprocessed work. Skipping this is how a traffic spike turns into an outage instead of just a slowdown.
62. WebSocket
A persistent, two-way connection between client and server — either side can push a message at any time without re-establishing the connection. The obvious fit for chat, multiplayer games, live dashboards.
63. Server-Sent Events (SSE)
A one-way channel from server to client over a single long-lived HTTP connection. Simpler than a WebSocket when you only need updates flowing in one direction — live scores, notifications, a news ticker.
64. Long Polling
The older workaround before WebSockets were widely supported: the client asks, the server holds the request open until it actually has something new, responds, and the client immediately asks again. Less efficient, but it works anywhere plain HTTP works.
Performance & Reliability
The unglamorous section, and the one that separates a demo from something that survives real traffic.
65. Latency
How long a single request takes, round trip. The delay you feel between clicking “Login” and seeing your dashboard.
66. Throughput
How much work the system gets through per unit of time, usually measured in requests per second. A system can have great latency and terrible throughput, or vice versa — they’re not the same axis, and optimizing one can quietly hurt the other.
67. Availability
Expressed as a percentage of uptime.
99.9% uptime -> ~8.7 hours of downtime per year
99.99% uptime -> ~52 minutes of downtime per year
99.999% uptime -> ~5 minutes of downtime per year
Each extra “nine” gets exponentially more expensive to guarantee — going from 99.9% to 99.99% is a much bigger engineering lift than the numbers make it look.
68. Single Point of Failure (SPOF)
Any component whose failure takes down the whole system — a single unreplicated database, a single server with no backup. Finding and eliminating these is a good chunk of what “production-ready” actually means.
69. Redundancy
Keeping backup copies of critical components so a failure of one doesn’t become a failure of the whole system, ideally without users even noticing.
70. Circuit Breaker
Stops your service from hammering another service that’s already failing. After enough failures, the breaker “opens” and blocks calls for a cooldown period, then cautiously lets a few through to check for recovery. Without one, a single struggling downstream dependency can cascade into an outage for everything that depends on it.
71. Timeout
The maximum time you’ll wait for a response before giving up. Every external call in production should have one — “wait forever” is not a strategy, it’s a slow-motion outage waiting to happen when the dependency on the other end stalls.
72. Retry
Trying a failed request again, on the assumption the failure was transient. Useful, but retrying aggressively against a struggling service is how you turn a small problem into a bigger one — which is exactly why the next term exists.
73. Exponential Backoff
Increasing the wait time between retries instead of hammering immediately.
import time
def call_with_backoff(fn, max_retries=5):
for attempt in range(max_retries):
try:
return fn()
except TransientError:
wait = 2 ** attempt # 1s, 2s, 4s, 8s, 16s
time.sleep(wait)
raise Exception("max retries exceeded")
74. Rate Limiting
Capping how many requests a user or client can send in a given window — say, 100 requests per minute — and rejecting the rest. Protects your system from abuse and from well-meaning clients that just poll too aggressively.
75. Load Shedding
Deliberately rejecting some requests when the system is overloaded, so the majority still get served well instead of everyone getting a slow, degraded experience. It sounds harsh until you’ve been on the other side of an outage where nobody got shed and nobody got served either.
What I’d tell my past self
None of these terms are hard on their own. What’s hard is that they only really click once you’ve felt the problem they solve — the slow query that needed an index, the retry storm that needed backoff, the celebrity account that turned into a hot partition. If you’re early in this, don’t wait for production to teach you the hard way. Read the term, picture the failure it prevents, and you’ll recognize it instantly the first time it shows up in a design review.
If this was useful, I’d genuinely like to know which of these tripped you up the most — drop it in the comments. And if you’re prepping for interviews, my honest advice hasn’t changed: you don’t need to design YouTube from scratch, you need to be able to explain why a cache exists in the first place.
Tags: System Design, Software Engineering, Backend Development, Databases, Distributed Systems
