MLOps — From Notebook to Production

What you'll learn

• Why a working model is only half the job
• What MLOps covers: deploy, monitor, retrain
• Model drift and why models get worse over time
• The lifecycle that keeps AI reliable

Building the car vs running the taxi fleet 🚕

Training a great model in a notebook is like building a beautiful car in a garage. MLOps is running the whole taxi fleet: getting cars on the road, keeping them fueled, spotting breakdowns, and replacing worn parts. It's DevOps (which you may have seen in the backend track) applied to machine learning.

A model that's 95% accurate in your notebook is worthless until it's serving real users reliably. MLOps covers everything after training:

• Deployment — wrap the model in an API so apps can send it data and get predictions (often in a Docker container — remember those?).
• Monitoring — track accuracy, speed, and errors in production. Is it still good today?
• Versioning — track model versions and the exact data each was trained on, so you can roll back a bad one.
• Automation (CI/CD) — pipelines that retrain, test, and redeploy without manual work.

The sneaky problem: model drift. Unlike normal code, a model can silently get worse over time even if you never touch it — because the world changes. A model trained on 2020 shopping habits degrades as fashions, prices, and behavior shift. The data it sees in production drifts away from its training data.

The fix is a loop, not a one-time launch:

Train → Deploy → Monitor → (accuracy dropping?) → Retrain on fresh data → Deploy → ...

This is why MLOps engineers are in demand: anyone can train a model from a tutorial, but keeping it accurate and reliable in production for years is the hard, valuable part.

Catching drift before users do: monitoring in practice 📊

Saying "monitor for drift" is easy; doing it means comparing numbers over time. The simplest and most common technique: track the distribution of the data the model sees in production, and compare it against the distribution it was trained on.

Concretely, teams often track simple summary statistics — like the average and range of a key input feature — computed on a rolling window of recent production data, and alert when they stray too far from the training-time baseline. For example, if a house-pricing model was trained when average house size was 150 m², and this month's incoming requests average 300 m², something has shifted in the population the model is seeing — maybe a new market segment, maybe a data pipeline bug. Either way, it's a signal to investigate before accuracy visibly degrades.

Beyond data drift, teams also run A/B tests when deploying a new model version: send a small percentage of real traffic to the new "challenger" model, keep most traffic on the proven "champion" model, and compare business metrics (not just accuracy) before fully rolling out. This protects against a model that looks great on a test set but behaves badly on real users.

None of this requires exotic tools to understand the idea: monitoring is fundamentally taking the statistics you learned about earlier in this track (averages, spread) and running them continuously in production instead of once during training.

Real-world case study: Zillow's home-buying algorithm 🔍

Zillow, the online real-estate company, ran a program called Zillow Offers that used a machine learning model — nicknamed internally as a pricing algorithm — to make instant cash offers on homes, aiming to buy, lightly renovate, and resell them for a profit. The model estimated a home's future sale price to decide how much to bid.

In late 2021, Zillow shut the entire program down and took a massive financial write-down, after the algorithm had been systematically overpaying for homes. Housing market conditions shifted quickly and unpredictably during that period, and the model's price predictions hadn't kept pace with the changing reality on the ground — the exact model drift problem from this lesson, except each wrong prediction meant overpaying for an actual house, not just a wrong label in a spreadsheet.

The company had to lay off a large share of the division's staff and absorb hundreds of millions of dollars in losses on homes purchased above what they could resell them for. Zillow's own leadership publicly acknowledged that the algorithm's error rate had been wider than anticipated and that they'd been unable to accurately forecast home prices amid unprecedented market conditions.

This is one of the most vivid, well-documented illustrations of why MLOps — continuous monitoring, and the willingness to catch and react to drift fast — is not a "nice to have." When a model quietly stops matching reality and nobody catches it in time, the cost isn't an abstract accuracy metric; real money, real houses, and real jobs are on the line.

Summary

• A trained model is only half the job; MLOps runs it in production
• Covers deployment, monitoring, versioning, and automated retraining
• Model drift makes models silently decay — fight it with a monitor→retrain loop
• Monitoring in practice means tracking data statistics (like averages) over time and using A/B tests before a full model rollout
• MLOps = DevOps for machine learning
• You wrote a real drift-detection function
• Case study: Zillow's home-buying algorithm shows the real financial cost of undetected model drift

Next: Generative AI — the models that create text, images, and code.