Caching & Queues — Fast and Resilient

What you'll learn

• Caching — remembering answers so you don't redo slow work
• Queues — handling heavy jobs later, without making users wait
• When to use each
• You'll build a tiny cache yourself

Sticky notes and the to-do tray 📝

Caching = a sticky note on your monitor. If someone asks you the WiFi password ten times a day, you don't walk to the router each time — you write it on a sticky note and read that. A cache stores the answer to a slow operation (a heavy database query, an external API call) so the next identical request is answered instantly from fast memory.

Request → is it in the cache?
   ├── YES → return it instantly (a "cache hit")
   └── NO  → do the slow work, store the result, return it (a "cache miss")

Tools like Redis are super-fast in-memory caches used everywhere. The catch: cached data can go stale — "how long do I trust the sticky note?" is the hard part of caching.

Queues = a to-do tray. Some jobs are slow: resizing a video, sending 10,000 emails, generating a report. You don't make the user stare at a spinner for two minutes. Instead you drop the job in a queue (the tray) and immediately tell the user "we're on it!". A separate worker picks jobs off the tray and processes them in the background.

User uploads video → put "resize this" in the queue → respond "processing!" instantly
Background worker → takes the job → resizes → notifies when done

Tools like RabbitMQ and Redis power queues. The payoff: your app stays snappy, survives traffic spikes (the tray just gets longer), and retries failed jobs automatically.

Eviction and invalidation — when to forget 🗑️

A cache that never forgets isn't a cache, it's a second, worse database — memory is expensive and finite. Two separate problems, both called "the hard part of caching":

Eviction — running out of room. A cache has a size limit. When it's full and a new item needs to go in, something must be thrown out. The most common strategy is LRU (Least Recently Used): evict whichever cached item hasn't been touched in the longest time, on the theory that "if nobody wanted it recently, nobody wants it now." Other strategies exist (LFU = least frequently used, FIFO = oldest inserted), but LRU is the default almost everywhere, including Redis.

Invalidation — the data changed underneath you. Suppose you cache "Sara's profile" for 10 minutes, but Sara updates her bio 30 seconds later. Now the cache is lying. Two common fixes:

• TTL (time-to-live): every cached entry auto-expires after N seconds, capping how wrong it can be.
• Explicit invalidation: the code that updates Sara's bio also actively deletes (or overwrites) her cached profile at the same moment, so nobody serves stale data even briefly.

There's a famous line by Phil Karlton: "There are only two hard things in computer science: cache invalidation and naming things." It's a joke with a real point — deciding exactly when a cached value stops being trustworthy is genuinely one of the trickiest judgment calls in backend engineering, and picking a TTL that's "long enough to help, short enough to stay honest" is a real skill.

Real-world case study: Cloudflare's global cache 🔍

Every request to this platform's static site is served through Cloudflare's CDN — one of the largest caching systems on Earth. The concept is exactly the one in this lesson, just scaled to hundreds of cities: the first time someone in Tokyo requests a page, Cloudflare's Tokyo data center has a cache miss, fetches it from the origin, and stores a copy locally. Every subsequent visitor in Tokyo gets a cache hit — served in milliseconds from nearby, without the request ever crossing an ocean.

The invalidation half of this lesson matters enormously at that scale. Static assets (JS/CSS bundles) are typically named with a content hash in the filename (like main.a1b2c3.js) and cached with a very long TTL — since the filename itself changes whenever the content does, there's nothing to invalidate; a new deploy simply produces new filenames and old cached copies quietly age out, unreferenced. HTML pages, which need to reflect fresher content, are usually cached with a short TTL or a Cache-Control: no-cache header that forces a quick revalidation check.

This two-tier approach — cache the unchanging stuff practically forever, cache the changing stuff briefly or not at all — is the same TTL judgment call from this lesson, just applied at planet scale. One famous real incident illustrates the risk: in July 2019, a bad regex deployed to Cloudflare's edge caused a global outage affecting a huge share of the web for about 30 minutes, a reminder that infrastructure this deeply cached and this widely relied-upon has to treat every deploy with extreme care.

Summary

• Caching = sticky note: store slow answers for instant reuse (watch for stale data)
• Queues = to-do tray: offload heavy jobs to background workers, respond fast
• You built a working cache with hit/miss counting, then added TTL-based expiry
• Eviction (LRU) handles a full cache; invalidation (TTL or explicit) handles stale data — the two "hard parts" of caching
• Cloudflare's global CDN caches this very platform's assets using these exact ideas, at planetary scale

Next: Docker & Deployment — packaging your backend so it runs identically everywhere.