System Design — Building for Millions

What you'll learn

• How one server becomes many (scaling)
• Load balancers, replicas, and horizontal vs vertical scaling
• The famous trade-offs every senior engineer weighs
• The mental model behind big-tech architecture

From food truck to restaurant chain 🍔

One server is a food truck: perfect for 50 customers a day. But 50,000? The truck melts down. System design is how you grow from truck to global chain without collapsing.

Two ways to scale:

• Vertical scaling = a bigger truck. Add more CPU/RAM to one server. Simple, but there's a ceiling (and it gets expensive fast).
• Horizontal scaling = more trucks. Run many servers and split the work. Nearly unlimited — this is how big systems actually scale.

When you have many servers, a load balancer stands in front like a host at a busy restaurant, sending each customer to a free table (server). If one server dies, the balancer just stops routing to it — instant resilience.

The database becomes the bottleneck. Common fixes:

• Replicas — copies of the database. Send all the reads to copies, writes to the main. Most apps read far more than they write.
• Caching (you learned it!) — keep hot data in Redis so the database is touched less.
• Sharding — split one giant database into pieces (users A–M here, N–Z there).

The senior mindset = trade-offs. There's rarely a "best" — only "best for this situation":

• More servers = more scale but more complexity and cost.
• Caching = speed but risk of stale data.
• The famous CAP theorem: in a distributed system you can't perfectly have Consistency, Availability, and Partition-tolerance all at once — you choose what to prioritize.

You don't need to design Netflix on day one. But knowing these words and trade-offs is exactly what separates a junior from a senior in interviews.

Rate limiting — the bouncer at the door 🚧

Scaling protects your system from legitimate growth. Rate limiting protects it from abuse and accidents: one buggy script, or one attacker, hammering your API thousands of times a second, starving every real user of capacity.

The idea: track how many requests each client (usually identified by IP address or account) has made recently, and once they cross a threshold within a time window, reject further requests — typically with HTTP status 429 Too Many Requests — until the window resets.

A common approach is the sliding window: keep a list of timestamps for each client's recent requests, drop any older than the window size, and check whether what's left is still under the limit:

client makes a request →
  remove timestamps older than (now - windowMs)
  if remaining timestamps >= limit → reject (429)
  else → record this timestamp, allow the request

Sensitive endpoints (like login) usually get a much stricter limit than general browsing endpoints — a login endpoint is the classic target for a password-guessing attack, so it's worth deliberately slowing down repeated attempts even for legitimate-looking traffic. Rate limiting is a small piece of code, but it's one of the cheapest, highest-leverage defenses a backend can have.

Real-world case study: this platform's own rate limiter 🔍

This exact algorithm runs, right now, in this platform's Cloudflare Worker (worker/src/index.ts). Its rateLimited(ip, isAuth) function keeps a Map of recent request timestamps per client, filters out anything older than the window, and rejects the request once the count is at or above the limit — the same sliding-window shape you just built.

Notice the deliberate asymmetry mirroring this lesson's "protect sensitive endpoints harder" point: general API paths allow 60 requests per minute, but anything under /api/auth/ is capped at just 10 per minute:

const RATE_GENERAL = { limit: 60, windowMs: 60_000 };
const RATE_AUTH = { limit: 10, windowMs: 60_000 };

The router checks pathname.startsWith("/api/auth/") and picks the stricter bucket specifically because auth endpoints are the classic target for credential-stuffing and brute-force attacks — exactly the reasoning this lesson just walked through, applied for real.

There's also an explicit, honest trade-off documented right in the code: the rate limiter is "best-effort per-isolate," meaning each Cloudflare Worker isolate keeps its own in-memory map rather than sharing counts globally across every server instance. A real system-design decision was made here — a perfectly accurate global rate limiter would need a shared store (like a Durable Object or Redis), which costs latency and complexity on every single request. For this platform's scale, an approximate, free, per-isolate limiter is judged "good enough" — a textbook example of this lesson's core idea that there's no universally "best" answer, only the best trade-off for the situation at hand.

Track complete! 🎓

Your backend foundation is built:

• Programming logic + databases + REST APIs
• Authentication & security done right
• Node.js, caching, queues, Docker, and scaling
• Rate limiting — you built a sliding-window limiter, the same shape and auth-vs-general split this platform's own Worker runs in production

Path to paid work: build a real API (users + auth + a database), containerize it with Docker, deploy it to a cloud host, and put it on your GitHub. A working, deployed backend with clean auth is a portfolio piece that gets interviews. Keep building!