Files, Permissions & the Linux Shell — One Tree, Three Digits, Many Pipes

What you'll learn

• How Linux arranges every file in one tree that starts at /, and how absolute and relative paths are resolved
• Who you are to the kernel: users, groups, UIDs and GIDs
• What the permission bits rwx mean for files and for directories, in symbolic and octal form, and how the kernel picks one class to check
• How redirection (>, <, 2>&1) and pipes (|) rewire a program's input and output
• The text tools that make the shell productive: sort, uniq, cut and awk

You will build a path resolver in JavaScript, a permission checker in C and a shell pipeline that reports on an /etc/passwd file.

One tree, rooted at / 🌳

Windows gives every disk a letter (C:, D:). Linux has one tree. It starts at the root directory /, and every disk, USB stick or network share is mounted onto some directory of that tree. The usual layout follows the Filesystem Hierarchy Standard (FHS):

/
├── bin/, sbin/, usr/   programs (on most current distros /bin is a link to /usr/bin)
├── etc/                system configuration: passwd, group, hosts, ssh/
├── home/               one directory per user: /home/ana, /home/omar
├── root/               the root user's home
├── tmp/                temporary files, writable by everyone
├── var/                data that changes: log/, cache/, lib/
├── dev/                device files: /dev/null, /dev/sda
└── proc/, sys/         virtual files the kernel generates on the fly

The last line hints at a Unix idea: almost everything is a file. Devices, kernel settings and even other processes (/proc/<pid>/) are opened and read with the same calls as a text file.

Absolute and relative paths

• An absolute path starts with / and is read from the root: /home/ana/notes.txt
• A relative path does not start with / and is read from the process's current working directory (pwd prints it): from /home/ana, notes.txt means /home/ana/notes.txt
• Every directory contains two special names: . (itself) and .. (its parent). In /, .. is / again
• Repeated slashes count as one: /etc//ssh is /etc/ssh

POSIX calls this pathname resolution: the kernel walks the path one component at a time, starting at / or at the current directory. (It also follows symbolic links on the way; you will meet those in the file-systems lesson.)

Two details trip up beginners:

• ~ is not a path the kernel understands. The shell replaces a leading ~ with your home directory before the program runs (tilde expansion). A C program that opens "~/notes.txt" looks for a directory literally named ~.
• The current directory belongs to each process, and a child gets a copy at fork. That is why cd must be a shell built-in: an external cd program would change only its own directory and then exit.

Watch the walk with a stack:

Users, groups and permission bits 🔐

To the kernel you are not a name but a number. Every process runs with a user ID (UID) and a group ID (GID), plus a list of supplementary groups. The names live in two text files:

• /etc/passwd: one account per line, name:password:UID:GID:GECOS:home:shell (the password field is just x; the hashes live in /etc/shadow, which only root can read)
• /etc/group: name:password:GID:members

id shows your identity, for example uid=1000(ana) gid=1000(ana) groups=1000(ana),27(sudo). UID 0 is root, the superuser. On most current distributions regular accounts start at UID 1000.

Reading ls -l

-rw-r----- 1 ana staff 2048 Mar  3 10:15 report.txt
│└┬┘└┬┘└┬┘   │   │
│ │  │  │    │   └── group owner
│ │  │  │    └────── user owner
│ │  │  └─────────── other: ---  (nothing)
│ │  └────────────── group: r--  (read)
│ └───────────────── owner: rw-  (read, write)
└─────────────────── type: - file, d directory, l symbolic link

The three letters mean different things for files and for directories:

• r: read the file's bytes / list the names in the directory
• w: change the file / create, delete or rename entries in the directory (so deleting a file depends on the directory's w, not the file's)
• x: run the file as a program / enter the directory and reach names inside it (cd, or any path that passes through it)

Octal: three digits, one per class

Each class is three bits, so it fits in one octal digit: r = 4, w = 2, x = 1. Add them up: rw- = 6, r-x = 5, r-- = 4. So -rw-r----- is 640 and rwxr-xr-x is 755.

chmod 640 report.txt         # owner rw, group r, other nothing
chmod u+x,o-r deploy.sh      # symbolic: add x for the owner, remove r for others
chown ana:staff report.txt   # change owner and group (giving a file away needs root)

New files do not start at 777. A program usually asks for 666 (files) or 777 (directories) and the process's umask removes bits: with the common umask 022, files become 644 and directories 755.

How the kernel decides

When a process opens a file, the kernel picks exactly one class, in this order (see path_resolution(7)):

1. If the process's UID is the file's owner, use the owner bits
2. Otherwise, if the file's group is one of the process's groups, use the group bits
3. Otherwise use the other bits

Only that class counts. A file with mode 075 denies its own owner, even though everybody else may read it. Root skips these checks for reading and writing, but may execute a file only if at least one x bit is set.

Three special bits sit in front of the usual nine: setuid (4000: the program runs with the file owner's UID, which is how passwd can update /etc/shadow), setgid (2000) and the sticky bit (1000: in a directory such as /tmp, mode 1777, a file can be deleted only by its owner, the directory's owner or root). The security lesson comes back to them.

Redirection and pipes 🔀

Every process starts with three open file descriptors: 0 = stdin, 1 = stdout, 2 = stderr. A program simply reads fd 0 and writes fds 1 and 2; the shell decides what they are connected to before it execs the program:

sort names.txt > sorted.txt        # fd 1 goes to a file (created or truncated)
echo done >> build.log             # append instead of truncate
wc -l < names.txt                  # fd 0 comes from a file
make 2> errors.txt                 # only stderr goes to the file
make > all.txt 2>&1                # stdout to the file, then stderr to wherever stdout points now
find / -name '*.conf' 2>/dev/null  # throw the "Permission denied" noise away

Order matters: 2>&1 copies whatever fd 1 points to at that moment. cmd > all.txt 2>&1 sends both streams to the file, while cmd 2>&1 > all.txt sends stderr to the terminal and only stdout to the file.

A pipe | connects the stdout of one command to the stdin of the next. The shell calls pipe() to get a kernel buffer with a write end and a read end, forks once per command and wires the ends to fd 1 and fd 0. All the commands of a pipeline run at the same time: a writer that fills the buffer (64 KiB by default on Linux, see pipe(7)) blocks until the reader catches up, and a reader blocks until data arrives.

ps -e --no-headers | wc -l   # how many processes (ps and wc included)? two programs, one pipe

The exit status of a pipeline is the status of its last command, so a failure on the left can go unnoticed. In Bash, set -o pipefail makes the pipeline fail if any of its commands fails.

The text toolbox: sort, uniq, cut, awk 🧰

Unix tools each do one job and talk through plain lines of text, so pipes can combine them:

• cut -d: -f1,7 /etc/passwd: keep fields 1 and 7, using : as the separator
• sort: sort lines; -n numeric, -r reverse, -t: field separator, -k3,3n sort by field 3 as a number. Prefix LC_ALL=C for plain byte order, the same on every machine
• uniq: drop adjacent duplicate lines; -c prefixes each line with its count. Because only neighbours are merged, you almost always sort first
• awk: a small language for columns. It splits each line into fields $1, $2, ... (-F: changes the separator) and runs pattern { action } for every matching line. NR is the line number, NF the number of fields, and an END { } block runs after the last line

awk -F: '$3 >= 1000 { print $1 }' /etc/passwd     # names of regular users
awk '{ sum += $2 } END { print sum }' sizes.txt    # add up column 2

The most famous pipeline counts things. Which IP addresses hit a web server most often?

cut -d' ' -f1 access.log | sort | uniq -c | sort -rn | head -3
     42 203.0.113.9
     17 198.51.100.4
      5 192.0.2.77

cut extracts the first field, sort groups equal lines, uniq -c counts each group, sort -rn puts the biggest count first and head keeps the top three. (The addresses come from the documentation ranges reserved by RFC 5737.)

Now build the same kind of pipeline over a passwd file.

Summary

• Linux has one tree rooted at /; other disks are mounted into it. Absolute paths start at /, relative paths at the process's current directory; ., .. and repeated slashes are resolved component by component
• ~ is expanded by the shell, and cd is a built-in because the current directory belongs to each process
• The kernel knows UIDs and GIDs. Permission bits rwx for owner, group and other fit in three octal digits; on a directory, w means create/delete entries and x means enter
• The kernel checks exactly one class (owner, else group, else other); root bypasses read and write checks, and execute needs at least one x bit
• Redirection rewires fds 0, 1 and 2 before exec (2>&1 order matters); a pipe connects two concurrently running processes through a kernel buffer
• cut, sort, uniq -c and awk turn text into answers; you wrote a path resolver, a permission checker and a passwd report

Practice: on a Linux machine run ls -l /etc/passwd /etc/shadow, id, umask and stat -c '%a %U %G %n' ~, then rebuild the IP counting pipeline on any log file you own. Next: concurrency and locks, and then file systems, where you will see how a path turns into an inode and disk blocks.