Static Analysis & Deobfuscation — Examine the Package Before You Open It
What you'll learn
By the end of this lesson you will be able to:
- Run a static triage on an unknown file: identify it, fingerprint it, list its strings and imports, and judge its entropy, all without executing it
- Read disassembler and decompiler output with the right amount of suspicion
- Tell encoding, obfuscation and encryption apart
- Peel layered encodings such as Base64, XOR and ROT13 to reach the real content, and write the small tools that do it
The suspicious parcel 📦
A parcel arrives and you do not know what is inside. A sensible person does not tear it open on the kitchen table. First you look at the outside: the label, the weight, the shape. You might shake it gently or put it through an X-ray. Only after you have a good guess do you open it, and then only in a safe place.
Static analysis is everything you do to a program without running it: inspecting the label (file type), the contents list (strings and imports), the X-ray (entropy and structure) and the blueprints (disassembly). It is safe, repeatable and always the first step. Its weakness is that a program can hide its real content behind layers of wrapping. Deobfuscation is the art of unwrapping those layers.
A reminder from the first lesson: only analyze files that are yours, open source, made for practice, or that you have permission to examine. Every sample in this lesson is a harmless text written for teaching.
A static triage checklist
Work from the cheapest, safest question to the most expensive one. Each command below only reads the file.
1. What is it? The extension lies; the magic bytes do not.
file sample # ELF 64-bit LSB pie executable, x86-64, dynamically linked, stripped
sha256sum sample # a fingerprint to name this exact file in your notes
A hash identifies a file: change one byte and the hash changes completely. Analysts record it in their notes and compare it with public reputation databases instead of uploading private files. A hash cannot be "decoded" back into the file, it is one-way.
2. What does it say? Text hides in binaries: messages, file paths, URLs, error strings, format strings.
strings -n 6 sample # printable runs of at least 6 characters (the default minimum is 4)
strings -e l sample # 16-bit little-endian text (Windows programs often use UTF-16)
3. What does it use? Programs reach the outside world through library functions, so the imports are a summary of its abilities.
readelf -d sample | grep NEEDED # shared libraries it loads
nm -D sample # dynamic symbols; "U" marks functions it imports
An import list with socket, connect and send suggests networking. fopen and unlink mean files. ptrace often means a debugger or an anti-debugger. One import proves nothing, but the whole list sketches a profile.
4. Is it wrapped? A packed or encrypted section looks like random noise. Shannon entropy measures that on a scale from 0 to 8 bits per byte: zero padding is about 0, English text and normal code sit roughly between 4 and 6, and compressed or encrypted data approaches 7.5 to 8. One high-entropy region inside a mostly normal file is a flag to investigate.
5. Then read it: the section list (readelf -S), then the disassembly (objdump -d -M intel) or a decompiler.
Keep a notebook: file name, hash, what you tried, what you learned, what you still wonder. Static analysis is detective work, and detectives write things down.
Run it: padding scores 0, text lands around 4.4, and the noise comes close to 8. Now try text.upper() or a text twice as long and watch the number barely move: entropy depends on the mix of byte values, not on the length. Real tools such as binwalk -E apply exactly this idea block by block.
import hashlib
import math
from collections import Counter
def entropy(data: bytes) -> float:
"""Shannon entropy in bits per byte: 0 = one repeated value, 8 = looks perfectly random."""
n = len(data)
h = 0.0
for count in Counter(data).values():
p = count / n
h -= p * math.log2(p)
return h
text = b"The quick brown fox jumps over the lazy dog. " * 20
padding = bytes(900)
# Deterministic noise: chained SHA-256 digests look like encrypted or compressed data.
noise = b""
block = b"seed"
while len(noise) < 900:
block = hashlib.sha256(block).digest()
noise += block
for name, blob in [("zero padding", padding), ("English text", text), ("hash-like noise", noise[:900])]:
print(f"{name:16} {len(blob):4} bytes entropy = {entropy(blob):.2f} bits/byte")Lab exercise 1
GoGraded in Code LabWrite a strings extractor
The first tool of every triage is strings. Build a small version of it. Go is a popular choice for analyst tooling because it compiles to one fast, dependency-free program.
The input is the bytes of a file written as hex. The first line is the minimum length N. The rest of the input is whitespace-separated hex bytes (any case, spread over any number of lines).
Find every run of printable ASCII: consecutive bytes in the range 0x20 to 0x7e (space through ~). For every run of at least N bytes print one line:
0x<offset> <the text>
The offset is where the run starts, in lowercase hex padded to at least 4 digits (%04x), followed by two spaces. A run that reaches the end of the input still counts. If nothing qualifies print nothing.
Example
Input:
4
00 48 65 6c 6c 6f 00 01 41 42 00 70 61 73 73 77 6f 72 64 ff
Output:
0x0001 Hello
0x000b password
(AB is only 2 bytes long, so it is skipped.)
- The worked example
- A run of exactly N is kept, N-1 is dropped, and a run may end the input
- Space and tilde are printable, 0x7f and 0x80 are not
- No run is long enough: no output
- Strings inside an ELF-like dump (mixed-case hex, several lines)
- A larger input: 120 repeated records
Example run
- Input
4 00 48 65 6c 6c 6f 00 01 41 42 00 70 61 73 73 77 6f 72 64 ff
- Expected output
0x0001 Hello 0x000b password
package main
import (
"bufio"
"fmt"
"os"
"strconv"
)
func main() {
reader := bufio.NewReader(os.Stdin)
More lines in Code Lab: 22
Hints and instant feedback are in Code Lab. Optional practice: it does not block finishing the lesson.
JavaScript and Python also run in your browser without an account; other languages use the secure server runner and need a free account.
Reading disassembly and decompiler output
When the triage says "this is worth a closer look", you open the file in a disassembler (objdump -d, Ghidra, radare2 / Cutter, IDA, Binary Ninja). Ghidra is free and open source, which makes it a great place to learn. A good disassembler gives you much more than a text dump:
- Cross-references (xrefs): "who uses this string?" and "who calls this function?" are the two questions you will ask most
- A function list, a call graph and a control-flow graph that draws the jumps from the previous lesson as boxes and arrows
- A decompiler that turns assembly back into C-like pseudo-code
A classic workflow is to start from a clue and work outward: find an interesting string (an error message, a path), follow its xref to the code that prints it, and read the few instructions before the message. That code is usually the check or the feature you wanted to understand, found in minutes instead of hours.
A decompiler is a guess, not the truth. Compilation throws away names, comments, types and structure, so the decompiler reconstructs what is probably there. It can be wrong in small ways (a long that was really a pointer, a loop shown as a goto) and it can fail completely on unusual code. Use it for the big picture, rename variables and retype them as you learn, and when something looks impossible, go back and read the assembly. Disassembly is what the CPU will do; decompiled code is a reading aid.
In a stripped binary the function names are gone and you will meet names like FUN_00101189. Give them meaningful names as you understand them (check_password, parse_header). Your renamed project is the real product of the analysis.
Encoding, obfuscation, encryption: three different things
People mix these up, and the confusion causes real mistakes.
- Encoding changes the representation of data so it fits somewhere, with a public, keyless recipe. Base64 and hex are encodings. They hide nothing from anyone who recognizes them.
- Obfuscation makes data or code harder to read but still recoverable by anyone with enough patience. ROT13 and a single-byte XOR are obfuscation. Malware authors and software vendors use it to slow analysis down.
- Encryption makes data unreadable without a secret key, with security that rests on the key and not on keeping the method secret. Done properly (for example AES with a random key), you cannot recover the data by analysis alone.
Seeing Base64 in an analysis is therefore good news: it is a speed bump, not a lock.
The layers you will meet most often
- Hex: each byte becomes two characters
0-9a-f. The text is twice as long as the data. - Base64: every 3 bytes become 4 characters from
A-Z a-z 0-9 + /. If the length is not a multiple of 3, the end is padded with=. So Base64 text has a length divisible by 4 and may end with one or two=. For exampleManbecomesTWFu. - ROT13: every letter moves 13 places in the alphabet, wrapping around. Because the alphabet has 26 letters, applying ROT13 twice gives back the original, so one function both encodes and decodes. Digits and punctuation are untouched.
- XOR with a key byte:
byte ^ key. XOR has a beautiful property:(x ^ k) ^ k = x. Applying the same key again undoes it, so again one function does both jobs. A one-byte key has only 256 possibilities, so it can simply be brute-forced. - Repeating-key XOR cycles through a short key. It is harder, but the same tricks scale up.
How to decide which layer you are looking at
- Only
0-9a-fand an even length: probably hex - Letters, digits,
+and/, length divisible by 4, maybe=at the end: probably Base64 - Readable-looking letters but nonsense words (
Uryyb, jbeyq): probably ROT13 or a Caesar shift - Random-looking bytes with no pattern: XOR, compression or encryption. Compare against a crib, a word you expect inside (
http,password,This program)
Two XOR tricks. In English text the space character (0x20) is the most common byte, so the most common byte in the ciphertext is probably 0x20 ^ key. And in binary data zero bytes are everywhere: a long run of one repeated byte is often the key itself.
Peel one layer at a time. Decode, look at the result, decide what it is, repeat. Undo layers in the reverse order the author applied them, like unwrapping nested boxes from the outside in.
Notice the \n inside the middle value: spaces became newline characters because 0x20 ^ 0x2a = 0x0a. Try decoding with the layers in the wrong order and see the nonsense you get. Then change the key and write down which output changes and which does not.
// Peeling an onion: every layer is a tiny reversible function.
const rot13 = (s) =>
s.replace(/[a-z]/gi, (c) => {
const base = c <= "Z" ? 65 : 97;
return String.fromCharCode(((c.charCodeAt(0) - base + 13) % 26) + base);
});
const xor = (s, key) => Array.from(s, (c) => String.fromCharCode(c.charCodeAt(0) ^ key)).join("");
const secret = "meet at dawn";
// The author's recipe: ROT13, then XOR with 0x2a, then Base64.
const encoded = btoa(xor(rot13(secret), 0x2a));
console.log("encoded :", encoded);
// The analyst undoes the layers in the REVERSE order.
const afterBase64 = atob(encoded);
console.log("after b64 :", JSON.stringify(afterBase64)); // odd characters appear: that is the XOR layer
const afterXor = xor(afterBase64, 0x2a); // XOR is its own inverse
console.log("after xor :", afterXor);
console.log("plaintext :", rot13(afterXor)); // ROT13 is its own inverse tooLab exercise 2
PythonGraded in Code LabPeel the layers
You were handed a message that an author wrapped in several layers. The first line of the input lists the layers in the order the author applied them, separated by commas. The second line is the final wrapped text. Undo the layers (last one first) and print the original message.
The data is a sequence of bytes at every step. The layers are:
b64: the data was replaced by its Base64 texthex: the data was replaced by its lowercase hex textrot13: every ASCII letter was rotated by 13 places (other bytes unchanged)rev: the bytes were reversedxor:K: every byte was XORed withK, which is a decimal number (42) or hex (0x5a) from 0 to 255
The original message is ASCII text. Print it (an empty message prints an empty line).
Example
Input:
xor:42,b64
QkM=
The author XORed the message with 42 and then Base64-encoded it. Undo Base64 first (the bytes 42 43), then undo the XOR with 42.
Output:
hi
- The worked example: xor then Base64
- Three layers: rot13, xor with a hex key, Base64
- Reverse then hex
- ROT13 applied on top of hex text (letters a-f change too)
- Order matters: Base64, then reverse
- An empty message
Example run
- Input
xor:42,b64 QkM=
- Expected output
hi
import base64
import sys
lines = sys.stdin.read().split("\n")
layers = lines[0].strip().split(",")
data = lines[1].strip().encode() if len(lines) > 1 else b""
def rot13(raw):
"""ROT13 on bytes: rotate ASCII letters, keep everything else."""
out = bytearray()
for b in raw:More lines in Code Lab: 15
Hints and instant feedback are in Code Lab. Optional practice: it does not block finishing the lesson.
JavaScript and Python also run in your browser without an account; other languages use the secure server runner and need a free account.
Lab exercise 3
JavaScript (Node.js)Graded in Code LabBreak a one-byte XOR with a crib
A configuration string was obfuscated with a single-byte XOR key. You do not know the key, but you know a word that must appear in the plaintext: the crib. Try all 256 keys and keep the first that reveals the crib.
Input:
- Line 1: the crib (lowercase ASCII, case-sensitive match)
- The remaining lines: the ciphertext as hex bytes. Any case, with or without spaces, possibly across several lines.
For every key from 0 to 255 decrypt every byte (byte ^ key) and check whether the plaintext contains the crib. Use the smallest key that does and print:
key = 0x<two hex digits, lowercase>
text = <the plaintext>
If no key works (or the ciphertext is empty) print no key found.
Example
Input:
net
44 4f 5e
Output:
key = 0x2a
text = net
(0x44 ^ 0x2a is n, 0x4f ^ 0x2a is e, 0x5e ^ 0x2a is t.)
- The worked example
- A configuration line, continuous lowercase hex
- Key 0x00: the text was never changed
- Key 0xff, the top of the range
- Upper-case hex split over two lines
- No key reveals the crib
Example run
- Input
net 44 4f 5e
- Expected output
key = 0x2a text = net
const lines = require("fs").readFileSync(0, "utf8").split("\n");
const crib = (lines[0] || "").trim();
// The ciphertext: join the remaining lines, drop every whitespace character, read the hex in pairs.
const hex = lines.slice(1).join("").replace(/\s+/g, "");
const bytes = [];
for (let i = 0; i + 1 < hex.length; i += 2) bytes.push(parseInt(hex.slice(i, i + 2), 16));
// TODO 1: loop over every key from 0 to 255.
// TODO 2: build the plaintext for that key: String.fromCharCode(byte ^ key) for every byte.
// TODO 3: if it contains the crib, print "key = 0x.." (two lowercase hex digits) and "text = ..." and stop.
More lines in Code Lab: 1
Hints and instant feedback are in Code Lab. Optional practice: it does not block finishing the lesson.
JavaScript and Python also run in your browser without an account; other languages use the secure server runner and need a free account.
Putting it together
A realistic static-analysis session on a small, permitted practice binary goes like this:
fileandsha256sum: record what it isstrings -n 6: you notice a long string made only of letters, digits and=, and a messageAccess denied- You decode the long string: Base64 gives bytes that look random, so you suspect XOR. You guess the crib
httpand brute-force the key, and a URL-like text appears. Nothing was executed. - In the disassembler you follow the xref of
Access deniedback to the function that prints it and read the comparison just above it - You rename functions and variables as you learn, and write down what you found
Limits of static analysis. When the real content is encrypted with a key the program only creates while it runs, or when the code is unpacked into memory at runtime, a static view shows only the wrapper. That is when you move on to dynamic analysis, running the program under control in a disposable lab, and to the next roadmap topics on binary formats and anti-analysis tricks.
Be a good citizen with what you decode. If the decoded content holds credentials, personal data or a vulnerability in someone's product, do not use or spread it. Report it responsibly, as the first lesson explained.
Question 1 of 5
Which of these is static analysis?
Question 2 of 5
You find the string `TWFuIGlzIGRpc3Rpbmd1aXNoZWQ=` in a sample. What is the best first guess?
Question 3 of 5
Why is a one-byte XOR key considered weak obfuscation?
Question 4 of 5
One section of an executable has an entropy of about 7.9 bits per byte, while the rest is around 5. What does that most likely mean?
Question 5 of 5
A decompiler shows a variable as `long`, but the assembly clearly uses it as an address. What should you do?
Summary and what to practice next
- Static analysis examines a file without running it:
file, hash,strings, imports, entropy, then disassembly - Start from a clue (a string) and follow xrefs; treat decompiler output as a reconstruction to be corrected, not as the truth
- Encoding (Base64, hex) hides nothing, obfuscation (ROT13, one-byte XOR) only slows you down, encryption needs a key
- Peel layers one at a time, in reverse order; recognize them by alphabet, length, padding and by using a crib
- Static analysis has limits: when content only exists at run time, you need dynamic analysis
Practice next: build your own small obfuscation chain for a sentence (ROT13, then XOR, then Base64) and write the decoder; extend your strings tool to report UTF-16LE text; and run the triage checklist on a small program you compiled yourself, then check your findings against the source code. Next up: running programs under a debugger, safely.
Practice
Reading is not learning until you run something. Pick one.
Graded lab exercises
- Lab exercise 1GoNot passed yet Try in Code Lab
- Lab exercise 2PythonNot passed yet Try in Code Lab
- Lab exercise 3JavaScript (Node.js)Not passed yet Try in Code Lab