OS Security & Hardening — Least Privilege in Practice

What you'll learn

• How the kernel decides what a process may do: kernel mode and user mode, real and effective UIDs, and the principle of least privilege
• How Linux capabilities split root's power into about forty pieces, and how to read a process's capability sets
• How to sandbox a process with seccomp, no_new_privs, namespaces and systemd's sandboxing options
• What NX, ASLR, stack canaries and RELRO protect against, and what they do not
• A practical hardening routine: patching, minimal services, SSH, firewalls, file permissions and auditing

This lesson is about defending systems you are responsible for. You will write a capability decoder in C, a seccomp policy evaluator in Python and a permission audit in Bash, all working on made-up data.

Rules of the road ⚖️

Security knowledge cuts both ways, so the ground rules come first:

• Practice on machines you own or have written permission to test: your laptop, a VM, a lab container. Probing or "testing" anyone else's system is a crime in most countries, even if you only look around
• Hardening a production system at work is a change like any other: get approval, test it and have a way back. A well-meant firewall rule can take a service down
• Everything in this lesson is defensive and uses synthetic data; no exercise touches a real system or network
• This is general education, not legal advice. If you are unsure, ask the system's owner first
• If you find a vulnerability, report it privately to the vendor or owner and give them time to fix it (coordinated disclosure)

Users, privileges and least privilege 👤

The CPU runs the kernel in a privileged kernel mode and your programs in user mode, where instructions that touch hardware or page tables are forbidden. The only door between the two is the system call: every open, kill or mount asks the kernel, and the kernel checks the caller's credentials first.

Each process carries a real UID (who started it), an effective UID (whose rights it uses now) and a saved UID, and the same three for groups. They differ in setuid programs: /usr/bin/passwd is owned by root and has the setuid bit, so while you run it its effective UID is 0 and it can update /etc/shadow. That power is exactly why every setuid program is a target, and why distributions keep the list short.

The defence is the principle of least privilege: every program and every person gets the minimum rights needed, for the minimum time.

• Services run as dedicated system accounts with a nologin shell, never as root or as you
• Privilege separation: OpenSSH's sshd keeps a small privileged monitor and does the risky network parsing in an unprivileged child process
• sudo rules name specific commands for specific people, not ALL
• A program that starts as root to grab something (a low port, a log file) drops privileges right after, in the right order: supplementary groups, then the GID, then the UID, and then it checks that it cannot get root back. In the wrong order it keeps root's group:

Capabilities: root in pieces 🧩

Since Linux 2.2 the kernel splits root's power into capabilities (capabilities(7)); recent kernels define 41 of them, numbered 0 to 40. A few examples:

• CAP_NET_BIND_SERVICE (10): bind to ports below 1024
• CAP_CHOWN (0): change any file's owner
• CAP_DAC_OVERRIDE (1): bypass file read, write and execute permission checks
• CAP_SYS_PTRACE (19): trace other processes and read their memory
• CAP_SYS_ADMIN (21): a huge grab bag (mounting, many device controls and more); the manual page itself calls it "the new root"

Each process has several capability sets: permitted (what it may use), effective (what is switched on right now), inheritable, bounding (an upper limit for it and its children) and ambient (kept across execve of unprivileged programs). The kernel shows them as 64-bit hex masks in /proc/<pid>/status, where bit n set means capability n:

$ grep Cap /proc/self/status       # as root inside a container with Docker's defaults
CapInh: 0000000000000000
CapPrm: 00000000a80425fb
CapEff: 00000000a80425fb
CapBnd: 00000000a80425fb
CapAmb: 0000000000000000

capsh --decode=00000000a80425fb turns a mask into names. Least privilege with capabilities means giving a program only the one it needs: setcap cap_net_bind_service=+ep ./server lets a web server bind port 443 without running as root, and docker run --cap-drop ALL --cap-add NET_BIND_SERVICE does the same for a container. The Code Lab runner that executes your lab code drops all capabilities and sets no-new-privileges.

Some capabilities are, in practice, as good as root, because each one opens a path to full control of the machine: CAP_SYS_ADMIN, CAP_SYS_MODULE (load code into the kernel), CAP_SYS_RAWIO (raw access to devices and I/O ports), CAP_SYS_PTRACE and CAP_DAC_READ_SEARCH (read any file). Seeing one of them on a service is worth a question.

Sandboxing: seccomp, no_new_privs and friends 🏖️

Even an unprivileged process can call hundreds of different system calls, and each one is kernel code that a compromised program can poke at. seccomp shrinks that attack surface:

• Strict mode (the original, 2005): after entering it a process may only call read, write, _exit and sigreturn
• Filter mode (seccomp-bpf, Linux 3.5): the process installs a small BPF program that looks at each system call's number and arguments and returns an action: SECCOMP_RET_ALLOW, SECCOMP_RET_LOG (allow, but log it), SECCOMP_RET_ERRNO (fail with an error such as EPERM), SECCOMP_RET_KILL_PROCESS and a few more
• Filters stack: every installed filter runs, and the action with the highest precedence wins (KILL beats ERRNO, which beats LOG, which beats ALLOW). A filter can never be removed, and it is inherited across fork and kept across execve
• To install a filter without CAP_SYS_ADMIN, a process must first set no_new_privs (prctl(PR_SET_NO_NEW_PRIVS)). It guarantees that execve can never grant more privileges: setuid bits and file capabilities are ignored from then on

Docker, Podman, Chrome, Firefox, OpenSSH and systemd all use seccomp filters. systemd turns the whole toolbox into a few lines of a unit file (systemd.exec(5)), and systemd-analyze security <unit> rates how exposed a service is:

[Service]
User=web
NoNewPrivileges=yes
CapabilityBoundingSet=CAP_NET_BIND_SERVICE
AmbientCapabilities=CAP_NET_BIND_SERVICE
SystemCallFilter=@system-service
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes

The namespaces from the previous lesson are sandboxing tools too (a private mount table, no network), and mandatory access control systems (SELinux, AppArmor) add rules that even root processes must obey.

Memory protections 🧱

C and C++ code can contain memory-safety bugs: a buffer overflow writes past the end of an array, a use-after-free reuses memory that was already released. If input controls what gets overwritten, such a bug can become code execution. The OS and the toolchain add layers that make that much harder:

• NX / W^X: page-table entries have a no-execute bit (the PTE flags from the memory lesson), so the stack and the heap can hold data the CPU refuses to run; the linker marks a non-executable stack with the PT_GNU_STACK header. W^X is the policy NX makes possible: a page is writable or executable, never both. It is a policy, not a law: Linux still lets a process map memory both writable and executable with mmap or mprotect unless something forbids it, such as SELinux's execmem permission or systemd's MemoryDenyWriteExecute=
• ASLR (address space layout randomization): the stack, the heap, shared libraries, mmap regions and, for PIE (position-independent executables), the program itself start at random addresses on every run, so an attacker cannot rely on fixed addresses. /proc/sys/kernel/randomize_va_space is 2 (full randomization) on mainstream distributions
• Stack canaries (-fstack-protector-strong): the compiler puts a random value next to the return address and checks it before the function returns; glibc aborts with *** stack smashing detected *** if it changed
• RELRO (-Wl,-z,relro,-z,now): the dynamic linker's tables (the GOT) become read-only after start-up
• _FORTIFY_SOURCE: adds size checks to calls such as memcpy and strcpy when the compiler knows the buffer size
• Newer CPUs add shadow stacks and indirect-branch tracking (Intel CET) or pointer authentication (ARM PAC)

None of these fixes the bug; they raise the cost of exploiting it, and attackers chain bugs together (an information leak can defeat ASLR, for example). The real fix is still correct code: bounds checks, fuzzing, sanitizers and memory-safe languages. See what ASLR changes from run to run:

A practical hardening routine 🧰

Most break-ins use known holes and sloppy configuration, not movie-style genius. A routine that covers the basics:

1. Patch: install security updates automatically (unattended-upgrades, dnf-automatic), and remember that a kernel update needs a reboot or live patching to take effect
2. Minimal services: every listening program is attack surface. List them with ss -tulpn and systemctl list-units --type=service --state=running, then disable or uninstall what you do not need
3. Accounts and SSH: key-based login, PasswordAuthentication no and PermitRootLogin no in sshd_config, named admins who use sudo, unused accounts locked
4. Firewall: deny inbound traffic by default and open only what you serve (nftables, ufw or firewalld)
5. File permissions: hunt for unexpected setuid programs (find / -xdev -perm -4000 -type f) and world-writable directories without the sticky bit (find / -xdev -type d -perm -0002 ! -perm -1000)
6. Logging and auditing: read journalctl and the authentication log, watch sensitive files with auditd (-w /etc/passwd -p wa -k identity), and send logs to another machine so an intruder cannot erase them
7. Measure: compare against a baseline such as the CIS Benchmarks, run lynis audit system, and keep backups you have actually restored once

Automate step 5 on the kind of listing find and stat produce:

Summary

• Practice only on systems you own or are authorized to test, and treat production hardening as a reviewed change
• The kernel checks credentials at every system call. setuid programs run with their owner's effective UID; least privilege means dedicated service accounts, privilege separation, narrow sudo rules and dropping privileges in the right order (groups, GID, UID)
• Capabilities split root into 41 pieces shown as hex masks in /proc/<pid>/status; grant only what is needed (setcap, --cap-drop ALL) and question CAP_SYS_ADMIN, CAP_SYS_MODULE, CAP_SYS_RAWIO, CAP_SYS_PTRACE and CAP_DAC_READ_SEARCH
• seccomp filters stack, the highest-precedence action wins, they can never be removed, and no_new_privs makes them safe for unprivileged processes; systemd, namespaces and SELinux/AppArmor complete the sandbox
• NX, ASLR, canaries, RELRO and FORTIFY make memory bugs harder to exploit, but only correct code removes them
• Hardening is a routine: patch, minimise services, lock down SSH, firewall by default, audit permissions, log off-host and measure against a baseline
• You wrote a capability decoder, a seccomp evaluator and a permission audit

Practice: in a VM of your own, run grep Cap /proc/self/status as a user and as root, try systemd-analyze security on a few services, and run the two find commands from the routine. Next: I/O and device drivers, with the resources listed for that topic on the course map.