Python for Data — Meet pandas

What you'll learn

• What pandas is and why data scientists live in it
• DataFrames: the spreadsheet in code
• Filtering, grouping, and aggregating data
• You'll analyze a real dataset in Python

Excel with superpowers 🐼

Data scientists spend most of their time not on fancy AI, but on cleaning and exploring data. Their tool is pandas — a Python library that's like Excel, but programmable and able to handle millions of rows.

The star is the DataFrame: a table with named columns and rows, just like a spreadsheet.

import pandas as pd

df = pd.DataFrame({
    "name": ["Sara", "Omar", "Lina"],
    "age":  [25, 30, 22],
    "city": ["Riyadh", "Jeddah", "Riyadh"],
})

df.head()          # peek at the first rows
df["age"].mean()   # 25.67 — average age in one line
df[df["age"] > 24] # only rows where age > 24 (filtering)
df.groupby("city").size()   # count people per city

Look how much that does in so little code. The everyday moves:

• Select a column: df["age"]
• Filter rows: df[df["age"] > 24] — keep only matching rows
• Group + aggregate: df.groupby("city")["age"].mean() — average age per city
• Sort: df.sort_values("age")

This "filter, group, aggregate" loop is the daily bread of data analysis. Real datasets come from CSV files (pd.read_csv("data.csv")), and pandas loads, cleans, and reshapes them for you. Master pandas and you can answer real questions from messy real data — the core job.

The gaps in real data: missing values 🕳️

Textbook data is complete. Real data has holes — a customer who skipped the "age" field, a sensor that dropped a reading, a survey question left blank. pandas represents a missing value as NaN ("Not a Number"), and dealing with it well is one of the most common tasks in the job.

pandas gives you three main moves:

df.isna()               # True/False grid: where are the gaps?
df["age"].isna().sum()  # how many missing ages?
df.dropna()             # drop any row that has a missing value
df.fillna(0)            # fill gaps with a specific value
df["age"].fillna(df["age"].mean(), inplace=True)  # fill with the column's mean

Why you can't just ignore NaN: if you compute df["age"].mean() pandas quietly skips the missing rows for you — convenient, but dangerous if you don't realize a third of your data was silently excluded. And if a NaN slips into a machine-learning model later, most models will simply crash.

Choosing a strategy matters:

• Drop rows — fine if missing data is rare and random.
• Fill with mean/median — reasonable for numeric columns when you want to keep every row.
• Fill with a placeholder (like "unknown") — often best for categorical/text columns, so you don't pretend to know something you don't.

There's no universally "correct" choice — the right one depends on why the data is missing, and a good data scientist always asks that question before typing .fillna().

Real-world case study: the WHO and messy country data 🔍

International health and economic organizations like the World Health Organization and the World Bank publish datasets covering nearly every country on Earth — life expectancy, GDP, vaccination rates, and more. These are some of the most widely used real-world datasets in data science courses and tools like gapminder.

They're also a masterclass in messy data: some countries report every year without fail, others have huge gaps during wars, transitions, or simply lack of infrastructure to collect the numbers. Country names change or are spelled inconsistently ("Ivory Coast" vs "Côte d'Ivoire"), and older data is sometimes estimated rather than measured.

Any analyst who naively calls .mean() on this data without first checking how much is missing, and for whom, risks a badly misleading conclusion — for example, silently underrepresenting the poorest countries, which tend to have the most gaps. This is exactly why the "check first, then decide a strategy" habit from this lesson matters: real, high-stakes datasets are never clean by default, and the missing values themselves often carry meaning.

Summary

• pandas = programmable Excel; the DataFrame is its core table
• Daily loop: select → filter → group → aggregate → sort
• ~80% of the job is cleaning and exploring data
• Missing values (NaN) need a deliberate strategy: drop, fill with mean/median, or fill with a placeholder
• You implemented group-and-average and a fill-missing-with-mean function from scratch
• The WHO/World Bank case study shows missing data is rarely random — check it before trusting any average

Practice: pip install pandas, load any CSV with pd.read_csv, and explore it. Next: SQL for Data — querying databases, the other half of a data scientist's toolkit.