Designing a URL Shortener (like bit.ly)

Framework: Problem → Constraint → Decision → Alternative → Trade-off → Failure Mode

This is the "hello world" of system design interviews — but it's deceptively deep. The interesting parts aren't the obvious pieces (a redirect service), they're the second-order decisions: how you generate the short code, how you make reads fast at absurd scale, and how you avoid the two classic failure modes (hot keys and analytics write amplification).

1. Problem

Build a service that takes a long URL and returns a short one (e.g. https://bit.ly/3xK9pQ), such that visiting the short URL redirects the user to the original long URL.

Functional requirements:

• Shorten a long URL → return a unique short code.
• Redirect: given a short code, look up and 302/301-redirect to the original URL.
• (Optional) Custom aliases, expiration dates, click analytics.

Non-functional requirements / scale assumptions:

• Read-heavy: redirects vastly outnumber creations. A realistic ratio is 100:1 to 1000:1 reads to writes (people click links far more often than they shorten new ones).
• Back-of-envelope: 100M new URLs/day → ~1,150 writes/sec average. At a 100:1 read:write ratio, that's ~115K reads/sec average — spiky well above that for viral links.
• Redirect latency must be very low (~tens of ms) — a URL shortener sitting in the critical path of every click on a shared link is effectively invisible infrastructure; users notice it only when it's slow.
• High availability for redirects (a link going viral is the worst possible time for the redirect service to be down) — reads should stay up even if the write path degrades.
• Short codes should be unguessable-ish (not sequential enough to enumerate every URL in the system) but don't need cryptographic security for the base case.

2. Constraint

The constraints that shape every decision below:

1. The short code must be short (6-8 chars) but the keyspace must be enormous — 100M URLs/day × several years of retention means billions of entries. Base62 (a-z, A-Z, 0-9) over 7 characters gives 62^7 ≈ 3.5 trillion combinations — comfortably enough, with room for a non-trivial "already taken" collision rate.
2. Redirects must be fast and horizontally scalable — this is a lookup-by-key problem at massive read volume, which points immediately toward a key-value store + aggressive caching, not a relational join-heavy schema.
3. No two writers can generate the same code at the same time — code generation must be coordination-free (or cheaply coordinated) because writes happen from many stateless app servers concurrently.
4. Click analytics (if required) must not slow down or endanger the redirect path — the redirect is the SLA-critical operation; analytics is best-effort by comparison.

3. Decision

3.1 Short code generation: Counter + Base62 encoding, sharded

The core decision: assign every new URL a unique, monotonically-increasing integer ID (from a distributed counter / range-allocated ID block), then encode that integer in Base62 to produce the short code.

• A central ID-generation service (or a Snowflake-style ID scheme, or pre-allocated ID ranges handed out to each app server so they can mint IDs locally without a round trip per request) produces the integer.
• Base62-encode the integer → e.g. 125 → "cb". This is a pure, stateless, deterministic function — no coordination needed at encode time, only at ID-allocation time.
• Store the mapping short_code → long_url in a key-value store (DynamoDB, Cassandra, or even a sharded relational table keyed by short_code).

Why this over random-string-and-check? Generating a random 7-char string and checking for collision before insert requires a read-then-write per creation — under high write volume this is wasted round trips and a race window between the check and the insert unless you use a conditional write. A pre-allocated, monotonic counter sidesteps collisions entirely: every ID is unique by construction, no check required.

3.2 Storage: Key-value store, sharded by short_code hash

Sharding by a hash of short_code (not by creation time / sequential ID) spreads both reads and writes evenly across shards. This matters because...

3.3 Redirect path: Cache-first (Redis) in front of the KV store

Given the 100:1+ read:write ratio, an LRU cache in front of the database captures the vast majority of redirect traffic — especially because real-world link popularity follows a Zipfian/power-law distribution (a small number of links get a huge fraction of all clicks). Cache hit → redirect in single-digit ms without touching the database at all.

3.4 Analytics: Asynchronous, off the critical path

Click events (timestamp, referrer, geo, device) are fired onto a message queue (Kafka/Kinesis) from the redirect handler after the redirect response has already been sent — never inline/blocking. A separate consumer aggregates these into an analytics store (e.g., a columnar/OLAP store or pre-aggregated counters in Redis) on its own schedule.

4. Alternative (and why each was rejected as the primary approach)

5. Trade-off

Every decision above is a trade-off, not a free win:

• Counter + Base62 vs. random string: gains collision-free generation and predictable write latency, but loses the "any node can mint an ID with zero coordination" property — you now depend on an ID-allocation service (or pre-fetched ID ranges) being available. This is mitigated by handing out ID ranges (e.g., blocks of 10,000 IDs) to each app server in advance, so 99.99% of ID mints are purely local and the allocator is only hit occasionally.
• Cache-first redirects: gains massive read throughput and low latency, but loses strong consistency — if a URL's target is updated (rare, but supported by some shorteners) or a link is deleted/deactivated, cached readers may serve the stale mapping until TTL expiry or explicit cache invalidation. Acceptable trade-off because URL→target mappings are near-immutable in practice.
• Async analytics: gains a redirect path that never blocks on analytics infra, but loses real-time click counts — the count shown on a dashboard lags real clicks by however long the queue+consumer take to catch up (typically seconds, not milliseconds). Acceptable because nobody needs a live click counter with sub-second accuracy.
• Sharding by short_code hash: gains even load distribution, but loses the ability to cheaply range-scan "all URLs created in the last hour" (a query a naive sequential-ID layout would support trivially) — if that query is needed, it has to be served from a separate index/analytics store, not the primary KV store.

6. Failure Mode

This is where the URL shortener problem gets interesting — the "trivial" system has real failure modes that mirror classic distributed-systems patterns:

6.1 Hot Partition / Hot Key — the viral link problem

A single short URL going viral (posted on the front page of a major site, shared in a mass text) concentrates all of its read traffic onto one shard/partition and, if uncached, one DB row. This is the textbook Hot Partition failure mode: one key gets so much traffic that its home shard degrades for every other key sharing that shard too — a single viral link can take down redirects for unrelated URLs that happen to co-locate on the same partition.

• Mitigation: cache-first design (Section 3.3) absorbs the vast majority of hot-key traffic before it ever reaches the sharded store. For truly extreme spikes, a second layer — a CDN-edge cache or even a static-redirect rule pushed to the CDN — takes it further upstream than application-layer Redis.

6.2 ID-allocator becoming a single point of failure

If ID generation is centralized (one counter service) and it goes down, all writes (new shortens) fail even though reads (redirects) are completely unaffected by this outage. This is a good interview point: correctly identify that this failure is write-path-only and does not threaten the SLA-critical redirect path, then propose the pre-allocated ID-range mitigation from Section 5 so that even a brief allocator outage doesn't stall writes (app servers keep minting from their local range until it's exhausted).

6.3 Cache stampede on a cold, suddenly-popular key

A link goes viral at the exact moment its cache entry expires (or it was never cached because it's brand new but shared immediately) — thousands of concurrent redirect requests all miss the cache simultaneously and all hammer the database for the same key at once. This is the same cache stampede pattern documented in article: Caching Strategies: 8 Patterns Every Engineer Should Know, and the same fix applies: request coalescing (in-flight de-duplication) so only one request per key actually reaches the database while the rest wait on that result.

6.4 Analytics write amplification

If click-event volume isn't decoupled from the redirect path (Decision 3.4), a viral link doesn't just amplify read load — it amplifies write load on whatever store holds analytics, at the same multiplier. Without the async-queue buffer, this write spike can back up and, in the worst case, apply backpressure all the way to the redirect handler itself, turning an analytics slowdown into a user-facing outage — the exact Cascading Failure pattern from 10 Must-Know System Design Failure Modes.

6.5 Short-code exhaustion / collision creep (long-tail failure)

Not an acute outage, but a slow-burn failure mode: if the ID-range or code length was under-provisioned relative to actual growth, the system approaches keyspace exhaustion years into production, forcing a length migration (7→8 chars) that must be backward-compatible with every already-issued short code still in circulation. Worth calling out in an interview as the kind of failure mode that doesn't page anyone at 3am but will show up in a capacity-planning review — the fix is over-provisioning the code length up front (each added Base62 character multiplies the keyspace by 62×, so the cost of provisioning generously is nearly free).

Key Takeaways

• The "trivial" URL shortener interview question is really a read-scaling + hot-key problem wearing a simple disguise — the moment you introduce "what if a link goes viral," every classic caching/sharding failure mode becomes relevant.
• Counter + Base62 beats random-string-and-check specifically because it makes collisions structurally impossible rather than statistically rare — a good general principle: prefer designs that eliminate a failure mode by construction over designs that merely reduce its probability.
• Decoupling the SLA-critical path (redirect) from the best-effort path (analytics) via an async queue is the same pattern that shows up in payment webhooks, notification delivery, and demand-forecasting pipelines elsewhere in this vault — it's a general recipe, not a URL-shortener-specific trick.