Every developer uses a shell daily. Few understand what it actually does. The shell looks like an application — you type commands, it runs them — but it's really a thin layer over a set of Unix primitives that are fundamental to how operating systems work. Building a shell from scratch is one of the best ways to understand processes, file descriptors, pipes, and signals — concepts that underpin everything from web servers to Docker containers.
A basic shell is surprisingly simple. The core loop is: read a line of input, parse it into a command and arguments, fork a child process, execute the command in the child, and wait for it to finish. That's maybe 50 lines of C. The complexity comes from the features we take for granted: pipes, redirection, background processes, signal handling, and job control.
The Read-Eval-Print Loop
At its core, a shell is a REPL. Read input, evaluate (execute) it, print the results, loop. The 'print' part is handled by the commands themselves — the shell just provides the environment for them to run in.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/wait.h>
// The simplest possible shell
int main(void) {
char line[1024];
while (1) {
printf("$ ");
if (!fgets(line, sizeof(line), stdin))
break; // EOF (Ctrl+D)
// Remove trailing newline
line[strcspn(line, "\n")] = '\0';
// Fork a child process
pid_t pid = fork();
if (pid == 0) {
// Child: execute the command
execlp(line, line, NULL); // This only handles single-word commands
perror("exec");
exit(1);
}
// Parent: wait for child to finish
waitpid(pid, NULL, 0);
}
return 0;
}This 25-line shell actually works — it can run commands like ls, pwd, and date. It doesn't handle arguments, pipes, redirection, or any other feature you'd expect. But it demonstrates the fundamental pattern: fork, exec, wait.
Fork and Exec: The Unix Process Model
The fork/exec split is Unix's most distinctive design decision, and building a shell makes you understand why it exists.
fork() creates an exact copy of the current process. The child has the same memory, the same open files, the same environment variables. exec() replaces the child's program with a new one. These are separate operations because the gap between them — after fork but before exec — is where the shell sets up the child's environment.
This is the key insight. When you type ls > output.txt, the shell forks, then in the child (before exec), opens output.txt and redirects stdout to it, then execs ls. The ls program doesn't know about the redirection — it writes to stdout as always, and the file descriptor manipulation done between fork and exec routes that output to a file.
// How 'ls > output.txt' works
pid_t pid = fork();
if (pid == 0) {
// Child process — between fork and exec
// Open the output file
int fd = open("output.txt", O_WRONLY | O_CREAT | O_TRUNC, 0644);
// Redirect stdout (fd 1) to the file
dup2(fd, STDOUT_FILENO); // Now fd 1 points to output.txt
close(fd); // Close the original fd (no longer needed)
// exec ls — it writes to stdout, which now goes to output.txt
execlp("ls", "ls", NULL);
perror("exec");
exit(1);
}
waitpid(pid, NULL, 0);This design is elegant because it composes. The child can set up any environment before exec — redirect files, change directories, modify environment variables, set resource limits, change user IDs — and the executed program inherits that environment without needing to know about any of it. Every command gets a pre-configured environment, and the shell is the thing that configures it.
Pipes: Connecting Processes
Pipes are the most powerful composition mechanism in Unix, and implementing them reveals how simple the underlying mechanism is.
The pipe() system call creates a pair of file descriptors: one for reading, one for writing. Data written to the write end appears at the read end. To implement ls | grep foo, the shell creates a pipe, forks twice, connects ls's stdout to the write end and grep's stdin to the read end.
// How 'ls | grep foo' works
int pipefd[2];
pipe(pipefd); // pipefd[0] = read end, pipefd[1] = write end
pid_t pid1 = fork();
if (pid1 == 0) {
// First child: ls
close(pipefd[0]); // Don't need read end
dup2(pipefd[1], STDOUT_FILENO); // stdout → pipe write end
close(pipefd[1]);
execlp("ls", "ls", NULL);
exit(1);
}
pid_t pid2 = fork();
if (pid2 == 0) {
// Second child: grep
close(pipefd[1]); // Don't need write end
dup2(pipefd[0], STDIN_FILENO); // stdin → pipe read end
close(pipefd[0]);
execlp("grep", "grep", "foo", NULL);
exit(1);
}
// Parent: close both pipe ends and wait
close(pipefd[0]);
close(pipefd[1]);
waitpid(pid1, NULL, 0);
waitpid(pid2, NULL, 0);Notice the careful closing of unused file descriptor ends. This is critical: if the parent doesn't close both pipe ends, grep will never see EOF on its stdin (because the write end is still open in the parent) and will hang forever. File descriptor leaks in pipe chains are one of the most common bugs when building a shell.
Built-in Commands
Some commands can't be external programs. cd is the classic example. If the shell forks a child and the child calls chdir(), only the child's working directory changes — the parent's directory is unaffected, and after the child exits, the shell is still in the same directory. For cd to work, the shell must execute it in its own process without forking.
Other built-in commands include export (modifying the shell's environment), exit (terminating the shell process), and source (executing a script in the current shell context). These all modify the shell's own state, which can only happen in the shell's own process.
Understanding which commands are built-in and why teaches you something fundamental about process isolation. A child process can't modify its parent. This is a security feature, a reliability feature, and sometimes an inconvenience — but it's core to how Unix processes work.
Signals and Job Control
Press Ctrl+C in your terminal and the running command stops. This seems simple but involves a surprisingly complex interaction between the terminal, the shell, signals, and process groups.
Ctrl+C sends SIGINT to the foreground process group. The terminal driver handles this — not the shell. The shell's job is to put each command into its own process group so that SIGINT goes to the command, not to the shell. If the shell doesn't set up process groups correctly, Ctrl+C kills the shell instead of the running command.
Job control — background processes (&), fg, bg, Ctrl+Z — adds another layer. The shell needs to track which processes belong to which jobs, manage foreground vs. background groups, and handle signals like SIGTSTP (Ctrl+Z, suspend) and SIGCHLD (child process terminated). Implementing this correctly is the hardest part of building a shell.
What You Learn
Building a shell teaches you concepts that come up constantly in software development, even if you never write systems code again.
- File descriptors are the universal interface. Files, pipes, sockets, terminals — they're all file descriptors. Redirection, piping, and network communication all use the same underlying mechanism. Understanding this makes everything from Docker networking to Unix domain sockets to Linux system APIs clearer.
- Process isolation is fundamental. A child process can't modify its parent. Environment variables, working directory, and open files are inherited copies, not shared references. This is why
cdin a subshell doesn't affect the parent, and why Docker containers inherit but don't share the host's environment. - Composition beats features. Unix has no 'find files matching a pattern and count them' command. It has
find,grep, andwc, connected by pipes. The shell's pipe mechanism makes simple tools composable into complex workflows. This design philosophy — small tools connected by standard interfaces — is the intellectual ancestor of microservices, Unix sockets, and API design. - Error handling is mostly about file descriptors. When a pipe breaks, when a child process crashes, when stdin is exhausted — the underlying mechanism is always file descriptors being closed, signals being delivered, or processes exiting with status codes. Once you understand the file descriptor model, error handling patterns across Unix systems become intuitive.
Where to Start
If you want to build a shell, start with the 25-line version above and add features incrementally. First: argument parsing (split the input on spaces). Then: I/O redirection (> and <). Then: pipes. Then: built-in commands (cd, exit). Then: environment variables. Each feature teaches a new Unix concept, and each one is a self-contained exercise.
Use C for the most direct mapping to the system calls. You can build a shell in Python or Rust, but the C implementation makes the underlying system calls explicit — you see exactly what fork(), exec(), dup2(), and pipe() do because you're calling them directly.
You don't need to build a production shell. Even a toy shell that handles basic commands, pipes, and redirection teaches you more about how Unix works than years of using one. The shell is the simplest program that exercises the most important operating system interfaces — and understanding those interfaces makes you a better developer regardless of what language or platform you work in.