Foundations · Lesson 1/7

How Programs Run — From Source Code to a Live Process

45 min 1 quiz Graded labs: 3

What you'll learn

By the end of this lesson you will be able to:

Explain what happens between a line of source code and a running process: compile → assemble → link → load
Recognize an executable by its magic bytes and read the key fields of an ELF header
Describe how a process is laid out in memory (.text, .data, .bss, heap, stack) and why byte order matters
Follow the rules of the road: what you may analyze, and how to report what you find

Tasting a cake to learn the recipe 🍰

Imagine someone hands you a baked cake and asks, "What's in it?" There is no recipe. So you taste it, inspect the crumbs, check the texture, and slowly work out the ingredients and the steps. That is reverse engineering: starting from a finished product (a program, a file format, a device) and working backwards to understand how it was made and what it does.

A compiler is the oven: it turns a human-readable recipe (source code) into something only the machine can eat (machine code). In this track you learn to read the cake.

Rules of the road — before you open any binary ⚖️

Reverse engineering is a normal, valuable skill: defenders use it to understand malware, engineers use it to make systems work together, and researchers use it to find and fix security bugs. It is also an area where the law and licenses really matter. Read this section twice.

Practice only on targets that are clearly yours or clearly offered for practice:

programs you wrote yourself
open-source software, within what its license allows
binaries from CTFs, crackmes and training platforms made for exactly this purpose
anything you have written permission to analyze

Rules differ by country and by contract:

Software licenses (EULAs) often forbid reverse engineering. Breaking a contract can have consequences even where the law allows analysis.
Many countries restrict circumventing technical protection measures (for example the DMCA in the United States). There are usually narrow exemptions, such as good-faith security research or interoperability, and they change over time.
In the EU, the Software Directive allows limited decompilation for interoperability, under conditions.
Accessing a system you do not own, or taking data from it, is a crime almost everywhere, even if you only "looked around".
This lesson is general education, not legal advice. If you are unsure, ask the owner or a lawyer before you start.

If you find a vulnerability: tell the vendor privately and give them time to fix it (coordinated disclosure). Do not publish exploit details for unpatched bugs, never use what you find against real people, and never keep or share other people's data. The roadmap node Law, Ethics & Responsible Disclosure covers this in depth.

Every exercise in this track runs on tiny programs written for the lessons. Nothing here touches a real product or a real network.

From source code to a running process

A program is born in five steps, and each step leaves clues that an analyst can read later.

 hello.c ──compile──▶ hello.s ──assemble──▶ hello.o ──link──▶ hello ──load──▶ process
  source        assembly            object file        executable      running in memory

(In practice gcc also runs a preprocessor first, which expands #include and macros.)

Compile turns C into assembly: readable names for the CPU's instructions.
Assemble turns assembly into an object file: real machine code, but with holes where the addresses of other functions are not known yet.
Link glues object files and libraries together and fills in the holes. A static link copies library code into the executable. A dynamic link only records "I need puts from libc" and leaves the lookup for later.
Load: the operating system's loader maps the file into memory, chooses addresses, resolves the dynamic libraries and jumps to the entry point.
Run: the CPU executes the machine code.

When your code calls a library function such as puts, it usually calls a small stub in the .plt section, which jumps through a slot in the .got table that the dynamic loader filled in with the real address. You will see those names in every disassembly.

You can watch the steps yourself on Linux:

gcc -S hello.c           # stop after compiling: writes hello.s
gcc -c hello.c           # stop after assembling: writes hello.o
gcc hello.c -o hello     # full build
file hello               # what kind of file is this?
readelf -h hello         # print the ELF header
nm hello.o               # list symbols (the names the linker uses)

Safety tip: ldd lists the libraries a program needs, but on some systems it works by actually running the program's loader. Never run ldd on a file you do not trust. readelf -d gives similar information without executing anything.

Run it, then delete greet.o from the list and run again: you will see the famous linker error undefined reference. The linker's real job is exactly this: place pieces, remember where each name landed, then patch every reference. Add a third object of your own to watch the addresses shift.

// A toy linker. Each object file has a size and exports named functions at offsets.
const objects = [
  { name: "main.o",  size: 32, exports: { main: 0 },  imports: ["greet", "puts"] },
  { name: "greet.o", size: 20, exports: { greet: 0 }, imports: ["puts"] },
];
const libc = { puts: 0x7f0000 }; // supplied later by the dynamic loader
let cursor = 0x401000;            // where .text starts

// Pass 1: lay the objects out one after another and record where every name lives.
const symbols = {};
for (const o of objects) {
  for (const [fn, offset] of Object.entries(o.exports)) symbols[fn] = cursor + offset;
  cursor += o.size;
}

// Pass 2: fill in the "holes", the references to names defined elsewhere.
for (const o of objects) {
  for (const sym of o.imports) {
    const addr = symbols[sym] ?? libc[sym];
    console.log(o.name, "calls", sym, "->", addr === undefined ? "UNDEFINED REFERENCE" : "0x" + addr.toString(16));
  }
}

Open in Code Lab

Inside an executable: formats and magic bytes

Every operating system has a container format for programs:

ELF (Linux, BSD, Android native code) starts with 7f 45 4c 46: the byte 7f followed by the letters "ELF"
PE (Windows .exe and .dll) starts with 4d 5a, the letters "MZ", and then points to a PE header further in
Mach-O (macOS, iOS): 64-bit files start with cf fa ed fe

The first bytes are called magic numbers. Tools such as file read them instead of trusting the file extension, and so should you: a file named invoice.pdf can be a program.

An ELF file is organized into sections (used by linkers and analysts) and segments (used by the loader). The sections you will meet most often:

.text: the machine code (read + execute)
.rodata: read-only data such as string constants
.data: initialized global variables (read + write)
.bss: global variables that start as zero. It takes no space in the file, only in memory
.symtab / .strtab: function and variable names. A stripped binary has them removed. It runs the same, but you lose helpful names
.plt / .got: the stubs and tables used to call shared-library functions

The ELF header at the start of the file is a fixed-size table of facts:

byte 4 is the class (1 = 32-bit, 2 = 64-bit)
byte 5 is the byte order (1 = little-endian, 2 = big-endian)
e_type at offset 16 (2 = executable, 3 = shared object, which includes PIE executables)
e_machine at offset 18 (62 = x86-64, 183 = AArch64)
e_entry at offset 24: the entry point, the address where execution starts. It is usually _start, small startup code that sets things up and then calls main

You will parse a real header in the first lab.

Lab exercise 1

PythonGraded in Code Lab

Read an ELF header

You receive the first bytes of an unknown file as hex, separated by whitespace (possibly over several lines). Decide whether it is an ELF file and print its key header fields.

Layout recap (offsets in bytes):

0–3: magic 7f 45 4c 46
4: class (1 = 32-bit, 2 = 64-bit)
5: byte order (1 = little-endian, 2 = big-endian)
16–17: e_type, 18–19: e_machine (both in the file's byte order)
24 and on: e_entry, the entry point: 4 bytes in a 32-bit file, 8 bytes in a 64-bit file

Names: e_type 1 = REL, 2 = EXEC, 3 = DYN, 4 = CORE, anything else other. e_machine 3 = x86, 40 = ARM, 62 = x86-64, 183 = AArch64, anything else other.

Example

Input (32 bytes):

7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
03 00 3e 00 01 00 00 00 40 10 00 00 00 00 00 00

Output (entry in lowercase hex):

class: 64-bit
endian: little
type: DYN
machine: x86-64
entry: 0x1040

If the magic is wrong (or there are fewer than 4 bytes) print not an ELF file. If the magic is right but the class or byte order is invalid, or there are too few bytes to read the entry point, print truncated or corrupt ELF header.

Visible tests (6)

A 64-bit little-endian shared object (x86-64)
A 32-bit little-endian ARM executable
A 32-bit big-endian file with an unknown machine
A Windows program (MZ) is not an ELF file
Empty input
Right magic but cut off before the entry point

Example run

Input: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 03 00 3e 00 01 00 00 00 40 10 00 00 00 00 00 00
Expected output: class: 64-bit endian: little type: DYN machine: x86-64 entry: 0x1040

Starter code

import sys

# The input is hex text such as "7f 45 4c 46 ..." -> a bytes object.
data = bytes(int(tok, 16) for tok in sys.stdin.read().split())

MAGIC = bytes([0x7F, 0x45, 0x4C, 0x46])
TYPES = {1: "REL", 2: "EXEC", 3: "DYN", 4: "CORE"}
MACHINES = {3: "x86", 40: "ARM", 62: "x86-64", 183: "AArch64"}

# TODO 1: if data does not start with MAGIC, print "not an ELF file" and stop.
# TODO 2: read the class (byte 4) and the byte order (byte 5); validate them and the length.
# TODO 3: decode e_type, e_machine and e_entry with int.from_bytes(..., "little" or "big").

More lines in Code Lab: 3

Try in Code Lab

Hints and instant feedback are in Code Lab. Optional practice: it does not block finishing the lesson.

JavaScript and Python also run in your browser without an account; other languages use the secure server runner and need a free account.

A process in memory — and byte order

When the loader has finished, the program lives in a virtual address space: a private map of the memory the process may touch.

high addresses   ┌───────────────┐
                 │     stack     │  local variables, return addresses (grows down)
                 ├───────────────┤
                 │      ...      │  shared libraries and other mappings
                 ├───────────────┤
                 │     heap      │  malloc'ed memory (grows up)
                 ├───────────────┤
                 │  .bss / .data │  global variables
                 ├───────────────┤
                 │    .rodata    │  constants (read-only)
                 │     .text     │  code (read + execute)
low addresses    └───────────────┘

Each region has permissions: read, write, execute. Code is normally r-x, data and stack rw-, and a well-configured system avoids regions that are both writable and executable. On Linux you can see the map of a live process in /proc/<pid>/maps: your third lab reads a simplified version of it.

Modern systems also use ASLR (address space layout randomization): the base addresses of the stack, heap, libraries and position-independent executables (PIE) change on every run. The same instruction has a different absolute address each time, but its offset from the module's base stays the same. The debugging lesson uses this to rebase addresses.

Byte order (endianness)

A 32-bit number takes four bytes. Which one goes first in memory?

Little-endian (x86-64, and ARM as normally run) stores the least significant byte first. 0x12345678 is stored as 78 56 34 12.
Big-endian stores 12 34 56 78. Network protocols use big-endian (called network byte order), and so do some file formats.

When you read a hex dump you must know which one applies: the little-endian bytes 40 10 00 00 are the number 0x1040, not 0x40100000. Mixing this up is the most common beginner mistake in binary analysis.

struct.pack("<I", n) packs an unsigned 32-bit integer as little-endian (<), and > means big-endian. Run it and compare the two byte sequences, then change n and predict the output before you run it.

import struct

n = 0x12345678
print("little-endian:", struct.pack("<I", n).hex(" "))
print("big-endian:   ", struct.pack(">I", n).hex(" "))

# The same four bytes mean different numbers depending on how you read them.
raw = bytes([0x40, 0x10, 0x00, 0x00])
print("read as little:", hex(int.from_bytes(raw, "little")))
print("read as big:   ", hex(int.from_bytes(raw, "big")))

Open in Code Lab

Lab exercise 2

CGraded in Code Lab

Swap the bytes

Network protocols and many file formats use big-endian, while your CPU is little-endian, so analysts constantly convert. For each number you must show how it sits in memory on a little-endian machine, and then its byte-swapped value.

Input: a first line N, then N lines, each an unsigned 32-bit number in decimal.

Output: one line per number: the value in hex (0x + 8 lowercase digits), ->, the four bytes in little-endian memory order as two-digit hex separated by spaces, ->, the byte-swapped value in the same hex style.

Use shifts and masks (not a cast of the host memory), so your answer does not depend on the machine.

Example

Input:

1
305419896

Output:

0x12345678 -> 78 56 34 12 -> 0x78563412

Visible tests (5)

The classic 0x12345678
Boundaries: zero and the largest 32-bit value
A small number ends up in the top byte
Values with the high bit set
No numbers at all

Example run

Input: 1 305419896
Expected output: 0x12345678 -> 78 56 34 12 -> 0x78563412

Starter code

#include <stdio.h>
#include <stdint.h>

/* TODO: return v with its four bytes in reverse order. */
static uint32_t swap32(uint32_t v) {
    return v;
}

int main(void) {
    int n;
    if (scanf("%d", &n) != 1) return 0;
    for (int i = 0; i < n; i++) {

More lines in Code Lab: 7

Try in Code Lab

Hints and instant feedback are in Code Lab. Optional practice: it does not block finishing the lesson.

JavaScript and Python also run in your browser without an account; other languages use the secure server runner and need a free account.

Lab exercise 3

JavaScript (Node.js)Graded in Code Lab

Which region is that address in?

A debugger just stopped at an address, and you want to know whether it is code, heap, stack or a library. You have the process's memory map (simplified from /proc/<pid>/maps) and a list of addresses to look up.

Input

a line M, then M region lines: start-end perms name (hex numbers without 0x; end is exclusive; the name has no spaces)
a line Q, then Q lines, each an address like 0x401a2b

Output: one line per query: <address as given> -> <name> (<perms>), or <address as given> -> unmapped when no region contains it.

Addresses can be larger than 32 bits, so use BigInt.

Example

Input:

2
00400000-00401000 r-xp demo
7ffd5a3c0000-7ffd5a3e1000 rw-p [stack]
2
0x400fff
0x401000

Output:

0x400fff -> demo (r-xp)
0x401000 -> unmapped

Visible tests (4)

Hits in code, data and stack
Boundaries: start is inside, end is outside
Addresses above 32 bits and below the first region
An empty memory map

Example run

Input: 4 00400000-00401000 r-xp demo 00600000-00601000 rw-p demo 01d3a000-01d5b000 rw-p [heap] 7ffd5a3c0000-7ffd5a3e1000 rw-p [stack] 4 0x400100 0x600abc 0x1d40000 0x7ffd5a3d0000
Expected output: 0x400100 -> demo (r-xp) 0x600abc -> demo (rw-p) 0x1d40000 -> [heap] (rw-p) 0x7ffd5a3d0000 -> [stack] (rw-p)

Starter code

const lines = require("fs").readFileSync(0, "utf8").split("\n");
let pos = 0;

const regionCount = Number(lines[pos++]);
const regions = [];
for (let i = 0; i < regionCount; i++) {
  const [range, perms, name] = lines[pos++].trim().split(/\s+/);
  const [start, end] = range.split("-");
  // TODO 1: store start and end as BigInt (BigInt("0x" + start)), plus perms and name.
  regions.push({ perms, name });
}

More lines in Code Lab: 7

Try in Code Lab

Hints and instant feedback are in Code Lab. Optional practice: it does not block finishing the lesson.

JavaScript and Python also run in your browser without an account; other languages use the secure server runner and need a free account.

Reading the clues

Every stage you met leaves evidence you will use later:

Compile leaves the program's logic: loops, branches and constants survive, even though names are gone
Link leaves imports: the library functions a program calls tell you a lot about what it does (files? sockets? crypto?)
Load explains why addresses differ: ASLR and PIE are why you rebase before comparing a debugger with a disassembler
Strings and sections (.rodata, .data) hold messages, paths and configuration

Before you move on, make a habit of asking three questions about any file you analyze: What is it? (magic bytes, file), Am I allowed to analyze it? (rules of the road), and Where will I run it, if at all? (a disposable virtual machine, never your everyday computer).

Quick quiz

Question 1 of 5

In which order do the build steps happen?

Question 2 of 5

A little-endian CPU stores the 32-bit value 0x12345678. What are the bytes in memory, lowest address first?

Question 3 of 5

What is special about the .bss section?

Question 4 of 5

You receive a file named invoice.pdf from an unknown sender. What is the sensible first step?

Question 5 of 5

Which of these is an appropriate practice target for a beginner?

Summary and what to practice next

A program travels source → assembly → object file → executable → process and every step leaves clues
An executable starts with magic bytes (7f 45 4c 46 for ELF); the header tells you class, byte order, type, machine and entry point
A process has .text, .rodata, .data, .bss, heap and stack, each with its own permissions, and ASLR moves them around on every run
Byte order decides what a hex dump means: little-endian 40 10 00 00 is 0x1040
Stay on your own, open-source, practice or permitted targets, respect licenses, and disclose problems responsibly

Practice next: write a tiny C program, compile it with gcc -S, -c and a full build, and run readelf -h, nm and file on each output. Then look at the same program built with -O0 and -O2 and notice what changes. The next lesson reads the assembly you just produced.