Machine Learning — Learning from Examples

What you'll learn

• What ML really is (vs normal programming)
• Supervised vs unsupervised learning
• Training data, features, labels, and the train/test split
• You'll build a simple classifier by hand

Teaching by showing, not telling 👶

Normal programming: you write the rules. "If email contains 'lottery' and 'winner', mark as spam." But spammers adapt, and you'd write rules forever.

Machine learning flips it: instead of writing rules, you show the computer thousands of examples (emails already labeled spam / not-spam) and it figures out the rules itself. Like a child learning "cat" — you don't define whiskers-and-tail; you point at cats until they just know.

The vocabulary:

• Features — the inputs (words in the email, size of a house).
• Label — the answer you want to predict (spam? price?).
• Training data — many examples with their labels, to learn from.
• Model — the thing that learns the pattern (the weights from last lesson).

Two big families:

• Supervised learning — you have labeled examples (emails marked spam/not). The model learns input → label. Includes classification (which category? spam/not) and regression (what number? house price).
• Unsupervised learning — no labels; the model finds hidden structure on its own, like clustering customers into groups you didn't know existed.

The golden rule: train/test split. You never judge a model on the data it studied — of course it memorized those. You hold back some examples as a test set it never saw, and check if it generalizes to those. A model that aces training but fails the test set is overfitting — it memorized instead of learning (like a student who memorized the answer key but can't solve new problems).

Decision trees: learning a flowchart of questions 🌳

The threshold classifier you're about to build ("spam if score ≥ 0.5") is actually the simplest possible example of one of ML's most intuitive models: the decision tree. Instead of one threshold, a decision tree learns a whole flowchart of yes/no questions, each one splitting the data a bit more cleanly.

Imagine deciding whether to approve a loan:

Is income > $50,000?
├── No  → Is existing debt < $5,000?
│         ├── Yes → Approve
│         └── No  → Reject
└── Yes → Approve

A decision tree doesn't need a human to write this flowchart. Given labeled training examples, it learns which question to ask at each step — specifically, it picks the question (and the threshold, like "$50,000") that best separates the labels ("approve" vs "reject") at that point. It repeats this, splitting each group further, until the groups are mostly pure (mostly one label) or it reaches a stopping point.

Trees are popular for three reasons:

• They're readable — you can literally draw the flowchart and explain a decision to a human, unlike many "black box" models.
• They handle messy, mixed data well — numbers and categories together, no need to normalize first.
• They're the building block of ensembles — a Random Forest trains hundreds of slightly different trees on random slices of the data and lets them vote, which is usually far more accurate than any single tree (this "wisdom of crowds" trick shows up throughout ML).

The one thing to watch for: a tree allowed to grow with no limit will keep asking questions until it perfectly memorizes the training data — a textbook case of the overfitting you just learned about. That's why real trees are limited in depth, or combined into a forest.

Summary

• ML learns rules from labeled examples instead of hand-written rules
• Supervised (labeled: classification/regression) vs unsupervised (clustering)
• Always use a train/test split; beware overfitting (memorizing, not learning)
• You built a threshold classifier

Practice: try scikit-learn's tutorials (Resources). Next: Deep Learning — neural networks, the tech behind modern AI.