Shell

The shell is the operating system's front door — the program that reads your commands, expands them into what the kernel executes, and returns what happened. Everything else in this handbook runs through it. This chapter teaches the POSIX shell grammar — the portable core every sh speaks — with bash's extensions flagged where they genuinely earn their keep. Write POSIX, reach for bash deliberately: that's the discipline.

Which shell is which

echo $SHELL            # your login shell
ls -l /bin/sh          # WHAT sh actually is on this system — varies!

/bin/sh is a symlink: to bash on Slackware, dash on Debian, bash in POSIX mode elsewhere. The distinction matters — scripts shebang'd #!/bin/sh must be portable (no bashisms), while #!/bin/bash buys the extensions. dash exists specifically to be the fast, strict POSIX sh that doesn't forgive sloppy scripts.

The grammar — the eight expansions

When you hit enter, the shell transforms your line in a fixed order — learn it once:

echo ~ $HOME $((1+2)) $(uname) *.txt {a,b} "quoted" 'literal'
# 1 brace  2 tilde  3 param+arith+cmdsubst  4 word-split  5 glob  6 quote-removal

Each stage is a tool and a trap:

• $var / ${var} — parameter expansion; ${var:-default}, ${var#prefix}, ${var%suffix}, ${#var} string ops are POSIX
• $(cmd) — command substitution, nestable, POSIX; backticks are the legacy form
• $((expr)) — arithmetic, integer only
• *.txt — globbing: * any run, ? one char, [abc]/[a-z] classes, [!a] negation
• {a,b,c} — brace expansion — bash only, not POSIX
• Tilde — ~, ~user
• Quotes — '...' fully literal; "..." protects spaces but allows $; "$@" is the special case

Quoting — where every shell bug lives

f="my file.txt"
ls $f          # TWO words — word splitting — the classic bug
ls "$f"        # one word — quote variables ALWAYS
echo '$HOME'   # literal $HOME — single quotes stop everything
echo "$HOME"   # expands

The rule that never lies: quote every variable expansion unless you specifically want word-splitting. rm -rf $dir unquoted is how systems die; rm -rf "$dir" survives spaces.

Redirection — the complete grammar

cmd > out          # stdout to file (truncate)
cmd >> out         # append
cmd 2> err         # stderr
cmd > out 2>&1     # both to one file — order matters!
cmd &> out         # bash shorthand for both
cmd < in           # stdin from file
cmd << EOF         # heredoc — literal input until EOF
cmd <<< "str"      # herestring — bash only
cmd | next         # stdout → stdin — the pipe
cmd1 && cmd2       # run second only if first succeeds
cmd1 || cmd2       # run second only if first fails
cmd ; next         # sequence
cmd &              # background

2>&1 ordering is the subtle one — > out 2>&1 binds stderr to stdout's destination; 2>&1 > out binds stderr to the old stdout (terminal). Order is everything.

Exit codes — the contract

Every command returns 0–255: 0 = success, nonzero = failure classes. $? reads the last one:

grep -q match file && echo found
[ -f file ] && echo exists        # test — the conditional primitive
test -d dir || mkdir dir

The conditional builtins: [/test (POSIX) — -f/-d/-r/-w/-x, -z/-n, =/!=, -eq/-lt/-gt. bash's [[ ]] adds pattern matching and &&/|| inside — not POSIX but genuinely better.

Job control

cmd &              # background it
jobs               # list
fg %1              # foreground job 1
bg %1              # resume in background
Ctrl-Z             # suspend
Ctrl-C             # SIGINT — interrupt
Ctrl-D             # EOF — end stdin
nohup cmd &        # survive hangup

Special shell powers

• Builtins vs binaries — type cd echo test reveals which live in the shell (cd, echo, [, export) vs which are /bin programs; type -a shows all resolutions. Builtins run in-shell — cd must be a builtin to change your process's cwd.
• exec — replaces the shell process; exec program never returns; exec > log redirects the shell itself
• set — the shell's own switches: set -e die on error, set -u die on unset vars, set -x trace, set -o pipefail (bash) — the script-safety preamble
• hash — bash caches binary locations; hash -r clears it (the LFS set +h trick)
• trap — run code on signals: trap 'cleanup' EXIT
• eval — re-parse a string as shell; powerful and dangerous — quote-sanitized input only

Environment and startup

export VAR=val     # into the environment — children inherit
env                # dump it
env -i cmd         # the sterile env — LFS's tool

Startup files differ by invocation: login shells read /etc/profile + ~/.profile (POSIX) — bash adds ~/.bash_profile; interactive non-login read ~/.bashrc; non-interactive (scripts) read neither — which is why a variable set in .bashrc doesn't exist inside a script. su - user gives a login shell; su user doesn't — the - matters.

POSIX vs bash — the honest table

Write POSIX for portability; use bash deliberately for arrays/[[/regex — and say so in the shebang.

The expansions — the complete taxonomy

The order matters because each stage feeds the next — this is the full machine:

{a,b,c}        # 1. brace expansion — a b c — bash only
~/x ~user/x    # 2. tilde — home dirs
$v ${v}        # 3. parameter expansion — the deep one:
  ${v:-default}    # if unset/empty, use default
  ${v:=default}    # ...and assign it
  ${v:?error msg}  # die if unset — the safety check
  ${v:+other}      # if set, use other
  ${#v}            # length
  ${v#pat}         # strip shortest prefix match
  ${v##pat}        # strip longest prefix
  ${v%pat}         # strip shortest suffix
  ${v%%pat}        # strip longest suffix
  ${v/pat/rep}     # substitute — bash
  ${v:2:5}         # substring — bash
$((1+2*3))     # 4. arithmetic — integers, C-like ops
$(cmd)         # 5. command substitution — nests, POSIX
`cmd`          #    the legacy form — same thing, uglier
<(cmd) >(cmd)  # process substitution — bash: cmd as a file
$v word split  # 6. word splitting — unquoted expansions split on IFS
*.txt ?.c      # 7. pathname expansion — the globs
   *           #    any run (not leading dots)
   ?           #    one char
   [abc][a-z]  #    classes/ranges
   [!a] [^a]   #    negation
   globstar ** #    recursive — bash shopt -s globstar
'x' "x" \$     # 8. quote removal — after everything

The trap order: word-splitting happens after parameter expansion — an unquoted $v splits on IFS then globs — the double hazard ls $v suffers.

The special variables — $'s secret life

$$    # this shell's pid
$!    # last background job's pid
$?    # last exit code — the contract
$#    # arg count
$0    # the script/shell name
$-    # this shell's option flags
$@ $* # all args — "each word" vs "one word"
$_    # last arg of previous command — bash
$PWD $OLDPWD $HOME $PATH $IFS $RANDOM $SECONDS $LINENO

IFS — the splitter

IFS (Internal Field Separator) decides where word-splitting breaks — default space tab newline. IFS=: read -r a b <<< "x:y" splits on colons; IFS=$'\n' splits only on lines — the for-loop safety trick. The while IFS= read idiom's IFS= prefix prevents leading-whitespace loss per line.

Coprocesses, exec-fd magic, and the corners

exec 3> out.log          # open fd 3 for writing
echo hi >&3              # write to it
exec 3>&-                # close it
exec 8< input.txt        # open for reading
read -u8 line            # read from fd 8
coproc P { cmd; }        # bash — a process with piped fds
cmd1 |& cmd2             # pipe stdout+stderr — bash

File-descriptor gymnastics is the shell's deepest layer — redirecting whole script output (exec > log 2>&1 at the top is the logging boilerplate).

POSIX sh vs bash — the extended table

The startup file matrix — which runs when

The classic confusion: .bashrc edits don't affect scripts — scripts aren't interactive and read no files.

Where shells go wrong

Exercises

1. Predict, then check: echo $(echo $(echo deep)) — nesting.
2. ls *.{c,h} vs ls "*.txt" — which is literal?
3. false && echo no || echo yes — trace the logic.
4. set -x; ls nonexistent; set +x — watch the trace.
5. Write a one-liner proving cd must be a builtin.
6. env -i bash -c 'env' — what's in a sterile env?

See also

• Linux · Filesystem · Core utilities · Shell scripting