Foundations · Lesson 2/7

Assembly Basics — Reading the CPU's Recipe Cards

46 min 1 quiz Graded labs: 3

What you'll learn

By the end of this lesson you will be able to:

  • Read short x86-64 functions written in Intel syntax and say what they compute without running them
  • Use registers, memory operands, flags and conditional jumps to follow loops and if statements
  • Explain how the stack, call and ret work, and which registers carry arguments and results (calling conventions)
  • Recognize the same ideas in ARM64 assembly

A kitchen with a tiny workbench 🧑‍🍳

Imagine a kitchen where the cook has a very small workbench with a handful of bowls. Every step of the recipe is a card that does one tiny thing: "put 5 in bowl A", "add the content of bowl B to bowl A", "if bowl A is empty, skip to card 12". Ingredients that do not fit on the bench live in the pantry, and the cook must walk there to fetch or store them.

That is a CPU. The bowls are registers: a few very fast slots inside the processor. The pantry is memory. The recipe cards are instructions, and the card number the cook is looking at is the instruction pointer. Assembly is simply the human-readable spelling of those cards. High-level code like total += i becomes two or three cards, and a reverse engineer's job is to read the cards and say "ah, that is a loop that adds up numbers".

Everything in this lesson uses tiny programs written for learning. You do not need to run real binaries to follow it.

Registers, instructions and memory (x86-64, Intel syntax)

An x86-64 CPU has sixteen 64-bit general-purpose registers: rax, rbx, rcx, rdx, rsi, rdi, rbp, rsp and r8 to r15. Each can be used in smaller pieces:

  • rax is all 64 bits, eax the low 32 bits, ax the low 16 bits, al the low 8 bits
  • rsp is the stack pointer and rbp is often the frame pointer; the others are mostly interchangeable
  • rip is the instruction pointer; you cannot use it like a normal register

Syntax. Two spellings exist. Intel syntax (used in this track, and by Ghidra, IDA and objdump -M intel) writes destination first: mov rax, rbx copies rbx into rax. AT&T syntax (the default in GNU tools) reverses the order and decorates names: movq %rbx, %rax. When you read a listing, first check which one you are looking at.

The instructions you will see in almost every function:

  • mov dst, src: copy. mov rax, 5, mov rax, rbx, mov rax, [rbx]
  • add, sub, imul, and, or, xor, shl, shr, inc, dec, neg: arithmetic and logic on the destination
  • xor eax, eax: the standard idiom for "set to zero" (shorter than mov eax, 0)
  • lea dst, [expr]: load effective address. It computes expr but does not read memory, so compilers also use it as a cheap calculator: lea eax, [rdi+rsi*4+3] means eax = rdi + rsi*4 + 3
  • cmp, test, jmp, jcc, call, ret, push, pop: covered below

Memory operands use square brackets: [rbx] is "the value stored at the address in rbx", and [rbp-8] is "8 bytes below rbp", typical for a local variable. The assembler often adds a size: dword ptr [rbp-4] is a 4-byte (32-bit) access, qword ptr is 8 bytes, byte ptr is 1.

Two rules that confuse every beginner:

  • Writing a 32-bit register (mov eax, 1) zeroes the upper 32 bits of rax. Writing ax or al leaves the other bits alone.
  • An x86 instruction can have at most one memory operand. To copy memory to memory you go through a register.

Here is a real function and its compilation. The C source:

int add3(int a, int b) { return a + b + 3; }

Typical compiler output without optimization (the arguments are first saved to the stack, then reloaded):

add3:
    push rbp
    mov  rbp, rsp
    mov  dword ptr [rbp-4], edi    ; save a
    mov  dword ptr [rbp-8], esi    ; save b
    mov  edx, dword ptr [rbp-4]
    mov  eax, dword ptr [rbp-8]
    add  eax, edx                  ; a + b
    add  eax, 3
    pop  rbp
    ret

And with optimization (-O2) the whole function collapses to two instructions:

add3:
    lea  eax, [rdi+rsi+3]
    ret

Same behaviour, very different look. Reverse engineers read both: debug builds are noisy but literal; optimized builds are compact and clever.

Run it and compare the three printed lines about writes. Then change mov al, 0xFF to a 16-bit write (mask 0xFFFF) and predict the output before you run. The last lines show why a reverse engineer must always ask whether a value is signed or unsigned: the bits do not say.

# How the pieces of rax behave. Python integers stand in for the 64-bit register.
MASK64 = (1 << 64) - 1

rax = 0x1122334455667788
print("rax =", hex(rax))
print("eax =", hex(rax & 0xFFFFFFFF))   # low 32 bits
print("ax  =", hex(rax & 0xFFFF))       # low 16 bits
print("al  =", hex(rax & 0xFF))         # low 8 bits
print("ah  =", hex((rax >> 8) & 0xFF))  # bits 8-15

# mov eax, 1   -> writing a 32-bit register ZEROES the upper half
rax = 1
print("after mov eax, 1 :", hex(rax))

# mov al, 0xFF -> writing an 8-bit register keeps the other bits
rax = 0x1122334455667788
rax = (rax & ~0xFF & MASK64) | 0xFF
print("after mov al, 0xFF:", hex(rax))

# Two's complement: the same 64 bits can be read as a signed number
rax = MASK64                              # all bits set
signed = rax - (1 << 64) if rax >> 63 else rax
print("0xFFFFFFFFFFFFFFFF as signed:", signed)

Lab exercise 1

PythonGraded in Code Lab

Emulate a listing

The fastest way to understand assembly is to be the CPU. Write a tiny emulator for straight-line code and report what the function returns.

The input is a listing, one instruction per line. Every register is 64 bits wide and starts at 0. The registers are rax, rbx, rcx, rdx, rsi and rdi.

  • Text after ; is a comment, blank lines are ignored, and mnemonics and registers are case-insensitive.
  • Operands are separated by a comma. A source operand is a register or an integer immediate (decimal such as -5, or hex such as 0xff).
  • Instructions: mov, add, sub, imul, and, or, xor (two operands: op dst, src), shl and shr (op dst, count with an immediate count from 0 to 63; shr is a logical shift), inc, dec, neg (one operand), and ret.
  • All arithmetic wraps around modulo 2^64.
  • Execution stops at the first ret (or at the end of the listing). Print rax as a signed decimal number: rax = <value>.

Example

Input:

mov rax, 7
imul rax, 5     ; rax = 35
add rax, 3
ret

Output:

rax = 38
Visible tests (6)
  • The worked example: 7 * 5 + 3
  • Comments, blank lines, upper case, hex, and/or
  • Register-to-register operations and a negative result
  • Wrap-around: the largest signed value plus one
  • Shifts, and nothing runs after ret
  • Empty listing: every register is zero
Example run
Input
mov rax, 7
imul rax, 5     ; rax = 35
add rax, 3
ret
Expected output
rax = 38
Starter code
import sys

MASK = (1 << 64) - 1
regs = {name: 0 for name in ("rax", "rbx", "rcx", "rdx", "rsi", "rdi")}


def value(operand):
    """The content of a register, or an immediate as an unsigned 64-bit number."""
    return regs[operand] if operand in regs else int(operand, 0) & MASK


def to_signed(v):

More lines in Code Lab: 11

Try in Code Lab

Hints and instant feedback are in Code Lab. Optional practice: it does not block finishing the lesson.

JavaScript and Python also run in your browser without an account; other languages use the secure server runner and need a free account.

Flags and jumps: how if and while look in assembly

A CPU has no if or while. It has flags and jumps.

Most arithmetic instructions leave a trace in the FLAGS register. The ones you need first:

  • ZF (zero flag): the result was zero
  • SF (sign flag): the top bit of the result is 1 (negative as a signed number)
  • CF (carry flag): an unsigned carry or borrow happened
  • OF (overflow flag): a signed overflow happened

The workhorse pair is compare, then jump:

  • cmp a, b computes a - b, throws the result away and keeps only the flags
  • test a, b computes a AND b and keeps the flags. test eax, eax is the idiom for "is eax zero (or negative)?"
  • jmp label always jumps. jcc label jumps only if the condition holds

Memorize the conditions after cmp a, b:

  • je / jz: equal (a == b), jne / jnz: not equal
  • Signed: jl (a < b), jle, jg (a > b), jge
  • Unsigned: jb (below), jbe, ja (above), jae

The signed and unsigned families exist because the same bits mean different things: the byte 0xFF is 255 unsigned but -1 signed. When you see jl or jg the compiler saw signed types (such as int), and jb or ja means unsigned types or pointers. That is a free hint about the original source.

Now a loop. The C function adds the numbers from 1 to n:

long sum_to(long n) {
    long total = 0;
    while (n != 0) { total += n; n--; }
    return total;
}

In assembly (n arrives in rdi, the result leaves in rax):

sum_to:
    xor  eax, eax        ; total = 0
.loop:
    test rdi, rdi        ; n == 0 ?
    jz   .done           ; yes: leave the loop
    add  rax, rdi        ; total += n
    dec  rdi             ; n--
    jmp  .loop           ; go back to the test
.done:
    ret

The pattern to recognize: a label that is jumped back to (a loop), a compare or test followed by a conditional jump (the loop condition), and a body in between. A forward jump that skips a few instructions is an if. Draw arrows from every jump to its target and the program's shape appears.

Look at the last two pairs: 127 vs 255 is below as unsigned numbers but not less as signed ones (127 > -1), and 0x80 vs 1 is the reverse. Same bits, same cmp, different jump. Add your own pair and predict jl and jb before you run.

// What does "cmp a, b" do to the flags? Emulated on 8-bit values.
function cmp8(a, b) {
  const result = (a - b) & 0xff;
  const sa = (a << 24) >> 24;            // a viewed as a signed byte
  const sb = (b << 24) >> 24;
  const diff = sa - sb;
  return {
    ZF: result === 0,
    SF: (result & 0x80) !== 0,
    CF: a < b,                           // unsigned borrow
    OF: diff < -128 || diff > 127,       // signed overflow
    sa, sb,
  };
}

const pairs = [[5, 5], [3, 200], [127, 255], [0x80, 1]];
for (const [a, b] of pairs) {
  const f = cmp8(a, b);
  const jl = f.SF !== f.OF;              // signed "less": SF != OF
  const jb = f.CF;                       // unsigned "below": CF
  console.log(
    "cmp " + a + ", " + b + "  signed view: " + f.sa + " vs " + f.sb +
    "  | je:" + f.ZF + "  jl:" + jl + "  jb:" + jb
  );
}

Lab exercise 2

PythonGraded in Code Lab

Add labels, flags and jumps

Extend the emulator so it can run loops. The arithmetic part is already written in the starter code (it is your solution of the previous lab in compact form). You add labels, the flags and the jumps.

New rules:

  • A line ending with : is a label (for example top:). It is not an instruction and costs no step.
  • cmp a, b sets the flags from a - b without changing a register. add, sub, and, or, xor, inc, dec and neg set the flags from their result. In this toy machine no other instruction touches the flags.
  • Flags: ZF = the result is zero, SF = the result is negative (as a signed 64-bit number).
  • Jumps: jmp; je/jz (ZF); jne/jnz (not ZF); jl (SF); jge (not SF); jle (ZF or SF); jg (not ZF and not SF). The target is a label name. (Real CPUs also look at OF; in these programs values stay small, so you may ignore it.)
  • Execution starts at the first instruction and stops at ret or when it runs past the last instruction.
  • Count executed instructions (ret counts, labels do not). If a program would execute more than 10000 instructions, print just step limit exceeded.

Otherwise print two lines: rax = <signed value> and steps = <count>.

Example

Input:

mov rcx, 3
again:
dec rcx
jnz again
ret

Output:

rax = 0
steps = 8

(mov once, then dec and jnz three times each, then ret: 1 + 6 + 1 = 8.)

Visible tests (6)
  • The worked example: a countdown loop
  • sum_to(5) with cmp and je
  • Factorial of 6 with a do-while loop
  • Signed comparison: max(-7, 3) with a forward jump
  • An endless loop hits the step limit
  • Boundary: exactly 10000 steps is allowed
Example run
Input
mov rcx, 3
again:
dec rcx
jnz again
ret
Expected output
rax = 0
steps = 8
Starter code
import sys

MASK = (1 << 64) - 1
LIMIT = 10000
regs = {name: 0 for name in ("rax", "rbx", "rcx", "rdx", "rsi", "rdi")}
flags = {"zf": False, "sf": False}


def value(operand):
    return regs[operand] if operand in regs else int(operand, 0) & MASK

More lines in Code Lab: 52

Try in Code Lab

Hints and instant feedback are in Code Lab. Optional practice: it does not block finishing the lesson.

JavaScript and Python also run in your browser without an account; other languages use the secure server runner and need a free account.

The stack, call and ret, and calling conventions

A function needs somewhere to keep temporary values and, above all, to remember where to come back to. That place is the stack: a region of memory that grows toward lower addresses, with rsp pointing at its top.

  • push x: rsp decreases by 8, then x is stored at [rsp]
  • pop x: the value at [rsp] is loaded into x, then rsp increases by 8
  • call target: pushes the return address (the address of the next instruction) and jumps to target
  • ret: pops the top of the stack into rip, so execution continues at whatever address was on top

That last line is the key to many bugs and attacks: ret trusts the top of the stack completely. If a function pushes something and forgets to pop it, or if data overwrites the saved return address, ret jumps to the wrong place. Understanding this is the first step to understanding stack-based vulnerabilities defensively, and the next lab shows the safe, simulated version of what happens.

A stack frame is the slice of stack one function call uses: the return address, saved registers and local variables. Unoptimized code builds it with the classic prologue and tears it down with an epilogue:

    push rbp            ; save the caller's frame pointer
    mov  rbp, rsp       ; rbp now marks the base of this frame
    sub  rsp, 32        ; reserve room for locals (locals live at [rbp-8], [rbp-16], ...)
    ...
    leave               ; = mov rsp, rbp ; pop rbp
    ret

Optimized code often skips the frame pointer and addresses locals relative to rsp instead, so do not rely on seeing push rbp.

Calling conventions are the agreement on where arguments and results travel. On 64-bit Linux, macOS and BSD (the System V AMD64 ABI):

  • integer and pointer arguments go in rdi, rsi, rdx, rcx, r8, r9, in that order; more arguments go on the stack
  • the result comes back in rax
  • rbx, rbp and r12 to r15 are callee-saved (a function must restore them); the other registers may be changed freely by a call

On 64-bit Windows the first four integer arguments use rcx, rdx, r8, r9 instead. When you open an unknown function, the first question is "which registers does it read before writing them?": those are its arguments. The second is "what is in rax at ret?": that is its result.

Example: a function that starts with lea eax, [rdi+rsi] reads rdi and rsi before writing them, so on Linux it takes at least two arguments, and it returns their sum in eax.

Lab exercise 3

CGraded in Code Lab

Simulate the stack

Write a small simulator of push, pop, call and ret, and watch what happens when a function leaves the stack unbalanced. It is a harmless model: nothing is executed, you only track the stack.

The input has at most 100 lines, each one instruction:

  • push N: put the integer N (can be negative) on the stack as data
  • pop: remove the top entry (ignore its value)
  • call NAME: put a return address on the stack. Instead of a real address, use the number of this instruction line (the first instruction is 1)
  • ret: if the top is a return address R, remove it and print return to R. If the top is data, print crash: ret popped data V (V is the value) and stop immediately

If pop or ret finds the stack empty, print crash: stack empty and stop. When the input ends normally, print the remaining stack from top to bottom on one line: stack: followed by the entries, each d<value> for data or r<line> for a return address. An empty stack prints stack: empty.

Example

Input:

push 7
call f
push 9
pop
ret

Output:

return to 2
stack: d7
Visible tests (6)
  • The worked example: a balanced call
  • Nested calls return in reverse order
  • An unbalanced push makes ret land on data
  • Empty input
  • ret on an empty stack
  • Pending calls and a negative value, listed top first
Example run
Input
push 7
call f
push 9
pop
ret
Expected output
return to 2
stack: d7
Starter code
#include <stdio.h>
#include <stdlib.h>

#define MAX 256

typedef struct {
    int is_return;  /* 1 = return address, 0 = plain data */
    long value;     /* the data, or the line number of the call */
} Cell;

int main(void) {
    Cell stack[MAX];

More lines in Code Lab: 19

Try in Code Lab

Hints and instant feedback are in Code Lab. Optional practice: it does not block finishing the lesson.

JavaScript and Python also run in your browser without an account; other languages use the secure server runner and need a free account.

The same ideas on ARM64

Phones, recent Macs and many servers use ARM64 (also called AArch64). The instructions differ, the concepts do not:

  • Registers: x0 to x30 are 64 bits wide; w0 to w30 are their low 32 bits. sp is the stack pointer, x29 is the usual frame pointer, x30 is the link register (lr), and xzr/wzr always read as zero
  • Fixed-size instructions: every instruction is 4 bytes, so addresses of instructions are multiples of 4. x86 instructions vary from 1 to 15 bytes
  • Load/store architecture: arithmetic works only on registers. Memory is touched only by ldr (load) and str (store), such as ldr x0, [x1, #8]
  • Three-operand arithmetic: add x0, x1, x2 means x0 = x1 + x2. The destination is still written first
  • Compare and branch: cmp x0, x1 sets the flags, then b.eq, b.ne, b.lt (signed), b.lo (unsigned) branch. cbz x0, label branches if x0 is zero
  • Calls: bl label puts the return address in x30 and jumps; ret jumps to x30. There is no automatic push, so a function that calls another one must save x30 on the stack first
  • Calling convention: arguments in x0 to x7, result in x0

The add3 function from earlier on ARM64:

add3:
    add  w0, w0, w1        ; a + b   (arguments arrive in w0 and w1)
    add  w0, w0, #3        ; + 3     (the result leaves in w0)
    ret

Notice the habits that carry over: find the argument registers, find the result register, find the loops (backward branches) and the conditions (compare plus conditional branch).

A method for reading any small function

  1. Identify the interface: which registers are read before they are written (arguments) and what is in the result register at ret (return value)
  2. Mark the control flow: draw an arrow for every jump or branch; backward arrows are loops
  3. Translate block by block into plain words or pseudo-code. Name things as you learn their role (rdi becomes n)
  4. Test your idea on a small input by hand, exactly like the emulator did
  5. Check the types: signed or unsigned jumps? dword or qword? Is that a pointer or a number?

Use a tool such as Compiler Explorer (godbolt.org) to compile your own tiny C functions and compare them with what you read. Writing C and reading the output is the fastest way to build a feel for it.

Quick quiz

Question 1 of 5

On 64-bit Linux (System V AMD64), where does a function receive its first two integer arguments?

Question 2 of 5

What does `cmp rax, rbx` do?

Question 3 of 5

What does `lea eax, [rdi+rsi*4+3]` do?

Question 4 of 5

How does a function call remember its return address on ARM64, compared with x86-64?

Question 5 of 5

A function pushes a value on the stack, never pops it, and then executes `ret`. What happens?

Summary and what to practice next

  • Assembly is the CPU's recipe cards: registers (bowls), memory (pantry), instructions (cards). In Intel syntax the destination comes first
  • cmp/test set flags, conditional jumps read them: backward jumps are loops, forward jumps are ifs, and jl/jg (signed) versus jb/ja (unsigned) tell you the original types
  • The stack grows down; call pushes the return address and ret blindly pops it
  • System V AMD64: arguments in rdi, rsi, rdx, rcx, r8, r9, result in rax. ARM64: arguments in x0-x7, result in x0, return address in x30
  • Reading method: interface, control flow, translate, test by hand, check types

Practice next: open godbolt.org, write five tiny C functions (max of two numbers, absolute value, sum of an array, string length, factorial) and compile them with -O0 and -O2. For each, say in words what the assembly does before you read the C again. The next lesson moves from reading code to analyzing whole files statically, and to undoing layers of obfuscation.