Serverless & Containers — Modern Compute

What you'll learn

• What containers are (a quick recap + why cloud loves them)
• Serverless — running code with zero server management
• When to choose each
• The trade-offs that decide real architectures

Two modern ways to run code ☁️

Traditionally you rented a whole server (EC2) and kept it running 24/7 — paying even when idle, and managing the OS yourself. Two modern approaches improve on that.

Containers (a recap from the backend track). A container packages your app + everything it needs into one portable box that runs identically anywhere (Docker). In the cloud, containers shine because they're:

• Lightweight — many containers share one server efficiently.
• Portable — the same box runs on any cloud.
• Scalable — spin up 100 identical copies in seconds behind a load balancer.

When you have many containers, you need something to manage them — starting, stopping, restarting crashed ones, spreading them across servers. That manager is Kubernetes (often "K8s"): the industry-standard container orchestrator. It's powerful and complex — a whole DevOps specialty on its own.

Serverless (e.g. AWS Lambda). Take it further: don't manage any server, not even a container host. You upload just a function, and the cloud runs it only when triggered — an HTTP request, a file upload, a timer. You pay per execution (often fractions of a cent) and nothing when idle.

Traditional server:  running 24/7, you pay always, you manage the OS
Container:           packaged app, you manage the container/cluster
Serverless:          just a function, provider runs it on demand, zero idle cost

How to choose:

• Serverless wins for event-driven, spiky, or occasional workloads (a form handler, an image resizer, a scheduled job). Cheapest and simplest when traffic is uneven. Downside: "cold starts" (a small delay when a function hasn't run recently) and less control.
• Containers win for long-running services, complex apps, or when you need full control and portability across clouds. Downside: you manage more (especially with Kubernetes).
• Plain servers (EC2) still fit when you need a specific OS setup or steady, predictable load.

There's no universal best — it's the trade-off thinking from the backend system-design lesson, applied to compute.

Auto-scaling and cold starts, up close 📈

Both containers and serverless promise to "scale automatically" — but the mechanics, and the trade-offs, differ in a way that matters when you're picking one.

Container auto-scaling works by watching a metric (usually CPU or memory usage, or requests-per-second) and adding or removing running container copies to match. If traffic doubles, the orchestrator (Kubernetes or a simpler managed service) starts more copies behind the load balancer — but each new copy still takes real time to start: pull the container image, boot the app, pass a health check. That's often several seconds to a minute. Good auto-scaling anticipates load (scaling up before a known traffic spike) rather than only reacting after users are already waiting.

Serverless cold starts are the same idea at a smaller scale. If a Lambda function hasn't run recently, the very first request has to wait while the cloud provisions a fresh execution environment for it — commonly tens to hundreds of milliseconds, sometimes longer for large functions or unusual runtimes. Once "warm," subsequent requests are fast, because the environment stays ready for a while. This is why serverless is a poor fit for latency-sensitive, constantly-hit endpoints (every request should be a "warm" hit) but a great fit for occasional or bursty triggers where an occasional cold start barely matters.

Both mechanisms are solving the same underlying problem — matching running capacity to real demand without a human watching a dashboard — just at different levels of the stack. Understanding why scaling has a delay (something has to actually start up) turns "the cloud scales automatically" from magic into an engineering trade-off you can reason about and design around.

Real-world case study: Kubernetes and Google's Borg 🔍

Kubernetes was open-sourced by Google in 2014, but it wasn't a first attempt — it was a public reimagining of "Borg," an internal container-management system Google had already been running for over a decade to schedule and manage the containers behind services like Search and Gmail at enormous scale.

Google's insight was that the hard problems of running containers reliably — scheduling them onto servers efficiently, restarting crashed ones automatically, scaling based on load, rolling out updates without downtime — were the same problems every company running containers at any real scale would eventually face. Rather than keep that solved internally, Google released Kubernetes as an open-source project and donated it to the newly formed Cloud Native Computing Foundation.

A decade later, Kubernetes had become the industry-standard container orchestrator this lesson describes, run by companies of every size on every major cloud provider — not because those companies invented it themselves, but because Google had already spent ten years hardening the exact orchestration problem they were facing. It's a rare case of a company's internal infrastructure tool becoming the entire industry's shared standard, and it's why "learn Kubernetes" is such common advice for anyone going deep on cloud and DevOps careers.

Summary

• Containers: portable app boxes; Kubernetes orchestrates them at scale
• Serverless: upload a function, run on demand, zero idle cost (cold-start trade-off)
• Choose by workload: serverless for spiky/event-driven, containers for long-running/portable
• It's trade-off thinking applied to compute
• Auto-scaling and cold starts both take real time because something genuinely has to start up
• Kubernetes began as Google's internal Borg system before becoming the industry standard

Next: Cloud Architecture — combining everything into well-designed, cost-aware systems.