Scalability Fundamentals

Vertical scaling has a ceiling. Horizontal scaling has a coordination problem. Caching has an invalidation problem. Pick your problems deliberately.

01.1 · CONCEPT

Load Balancing

A single endpoint hides a pool of workers. The load balancer decides which one gets the next request — and notices when one stops answering.

ALGORITHMS

Round-robin is fair but blind. Least-connections beats it under uneven request weights. Consistent hashing wins when cache locality matters.

L4 vs L7

L4 (TCP) is faster and protocol-agnostic. L7 (HTTP) can route by path, header, or cookie — the price is parsing every request.

HEALTH CHECKS

Active probes catch dead servers fast but cost requests. Passive checks are free but slow to react. Most prod setups run both.

FIG · 01.1

01.2 · CONCEPT

Caching Layers

Every layer between user and database is an opportunity to skip work. The closer to the user, the cheaper the hit — and the harder the invalidation.

LATENCY BUDGET

Disk: ~10ms. Network round-trip same-AZ: ~0.5ms. Redis hit: ~1ms. CPU cache: ~ns. Move hot data up the stack relentlessly.

WRITE STRATEGIES

Write-through is safe but slow. Write-back is fast but loses data on crash. Write-around skips the cache on writes — good for write-heavy, read-rare data.

INVALIDATION

TTL is the lazy default. Explicit purge on write is correct but couples writers to the cache. Versioned keys (key:v17) sidestep both.

FIG · 01.2

01.3 · CONCEPT

CDN & Edge

Distance is latency you can't optimise away — unless you move the bytes closer. CDNs put cached responses (and increasingly compute) in hundreds of POPs around the world.

ANYCAST ROUTING

Every POP advertises the same IP. BGP routes the user to the topologically closest one. No DNS tricks required.

ORIGIN SHIELD

A single mid-tier cache in front of origin collapses thundering-herd misses from hundreds of edge POPs into one request.

EDGE COMPUTE

Workers/Lambda@Edge run logic at the POP — A/B tests, auth, request rewriting — without a round-trip to your region.

FIG · 01.3

NEXT · 02 →

Databases at Scale