Skip to content
ALB Wiki
Main page · News · Recent changes · All pages · Categories · Random page · Suggestions · Help

Diff — Shell

Comparing rev 1 → rev 2

{{infobox
| title = The Shell
| chapter = Linux Handbook, chapter 9 — POSIX sh first, bash where it earns its keep
| core = The command language · the REPL · the OS's front door
| shells = sh (POSIX) · bash · dash · zsh · ksh — the grammar is mostly shared
| skills = pipes · redirection · globbing · quoting · expansion order
| next | Core utilities — the vocabulary the grammar drives
}}
 
**The shell is the operating system's front door** — the program that reads your commands, expands them into what the kernel executes, and returns what happened. Everything else in this handbook runs through it. This chapter teaches the **POSIX shell grammar** — the portable core every `sh` speaks — with **bash**'s extensions flagged where they genuinely earn their keep. Write POSIX, reach for bash deliberately: that's the discipline.
 
 
## Which shell is which
 
 
```sh
 
echo $SHELL            # your login shell
 
ls -l /bin/sh          # WHAT sh actually is on this system — varies!
 
```
 
 
`/bin/sh` is a symlink: to `bash` on Slackware, `dash` on Debian, `bash` in POSIX mode elsewhere. The distinction matters — scripts shebang'd `#!/bin/sh` must be *portable* (no bashisms), while `#!/bin/bash` buys the extensions. `dash` exists specifically to be the fast, strict POSIX sh that doesn't forgive sloppy scripts.
 
 
## The grammar — the eight expansions
 
 
When you hit enter, the shell transforms your line in a **fixed order** — learn it once:
 
 
```sh
 
echo ~ $HOME $((1+2)) $(uname) *.txt {a,b} "quoted" 'literal'
 
# 1 brace  2 tilde  3 param+arith+cmdsubst  4 word-split  5 glob  6 quote-removal
 
```
 
 
Each stage is a tool and a trap:
 
 
- **`$var` / `${var}`** — parameter expansion; `${var:-default}`, `${var#prefix}`, `${var%suffix}`, `${#var}` string ops are POSIX
 
- **`$(cmd)`** — command substitution, nestable, POSIX; backticks are the legacy form
 
- **`$((expr))`** — arithmetic, integer only
 
- **`*.txt`** — globbing: `*` any run, `?` one char, `[abc]`/`[a-z]` classes, `[!a]` negation
 
- **`{a,b,c}`** — brace expansion — *bash only*, not POSIX
 
- **Tilde** — `~`, `~user`
 
- **Quotes** — `'...'` fully literal; `"..."` protects spaces but allows `$`; `"$@"` is the special case
 
 
## Quoting — where every shell bug lives
 
 
```sh
 
f="my file.txt"
 
ls $f          # TWO words — word splitting — the classic bug
 
ls "$f"        # one word — quote variables ALWAYS
 
echo '$HOME'   # literal $HOME — single quotes stop everything
 
echo "$HOME"   # expands
 
```
 
 
The rule that never lies: **quote every variable expansion unless you specifically want word-splitting.** `rm -rf $dir` unquoted is how systems die; `rm -rf "$dir"` survives spaces.
 
 
## Redirection — the complete grammar
 
 
```sh
 
cmd > out          # stdout to file (truncate)
 
cmd >> out         # append
 
cmd 2> err         # stderr
 
cmd > out 2>&1     # both to one file — order matters!
 
cmd &> out         # bash shorthand for both
 
cmd < in           # stdin from file
 
cmd << EOF         # heredoc — literal input until EOF
 
cmd <<< "str"      # herestring — bash only
 
cmd | next         # stdout → stdin — the pipe
 
cmd1 && cmd2       # run second only if first succeeds
 
cmd1 || cmd2       # run second only if first fails
 
cmd ; next         # sequence
 
cmd &              # background
 
```
 
 
`2>&1` ordering is the subtle one — `> out 2>&1` binds stderr to stdout's *destination*; `2>&1 > out` binds stderr to the *old* stdout (terminal). Order is everything.
 
 
## Exit codes — the contract
 
 
Every command returns 0–255: **0 = success**, nonzero = failure classes. `$?` reads the last one:
 
 
```sh
 
grep -q match file && echo found
 
[ -f file ] && echo exists        # test — the conditional primitive
 
test -d dir || mkdir dir
 
```
 
 
The conditional builtins: `[`/`test` (POSIX) — `-f`/`-d`/`-r`/`-w`/`-x`, `-z`/`-n`, `=`/`!=`, `-eq`/`-lt`/`gt`. bash's `[[ ]]` adds pattern matching and `&&`/`||` inside — not POSIX but genuinely better.
 
 
## Job control
 
 
```sh
 
cmd &              # background it
 
jobs               # list
 
fg %1              # foreground job 1
 
bg %1              # resume in background
 
Ctrl-Z             # suspend
 
Ctrl-C             # SIGINT — interrupt
 
Ctrl-D             # EOF — end stdin
 
nohup cmd &        # survive hangup
 
```
 
 
## Special shell powers
 
 
- **Builtins vs binaries** — `type cd echo test` reveals which live in the shell (`cd`, `echo`, `[`, `export`) vs which are `/bin` programs; `type -a` shows all resolutions. Builtins run in-shell — `cd` *must* be a builtin to change your process's cwd.
 
- **`exec`** — replaces the shell process; `exec program` never returns; `exec > log` redirects the shell itself
 
- **`set`** — the shell's own switches: `set -e` die on error, `set -u` die on unset vars, `set -x` trace, `set -o pipefail` (bash) — the script-safety preamble
 
- **`hash`** — bash caches binary locations; `hash -r` clears it (the LFS `set +h` trick)
 
- **`trap`** — run code on signals: `trap 'cleanup' EXIT`
 
- **`eval`** — re-parse a string as shell; powerful and dangerous — quote-sanitized input only
 
 
## Environment and startup
 
 
```sh
 
export VAR=val     # into the environment — children inherit
 
env                # dump it
 
env -i cmd         # the sterile env — LFS's tool
 
```
 
 
Startup files differ by invocation: **login shells** read `/etc/profile` + `~/.profile` (POSIX) — bash adds `~/.bash_profile`; **interactive non-login** read `~/.bashrc`; **non-interactive** (scripts) read neither — which is why a variable set in `.bashrc` doesn't exist inside a script. `su - user` gives a login shell; `su user` doesn't — the `-` matters.
 
 
## POSIX vs bash — the honest table
 
 
| Feature | POSIX | bash |
 
| --- | --- | --- |
 
| `$(cmd)`, `>`, `|`, `&&` | ✓ | ✓ |
 
| `[[ ]]`, `<<<`, `&>` | ✗ | ✓ |
 
| `{a,b}` braces | ✗ | ✓ |
 
| Arrays `arr=(...)` | ✗ | ✓ |
 
| `local`, `declare` | ✗ | ✓ |
 
| `${var/pat/rep}`, regex `=~` | ✗ | ✓ |
 
| `pipefail`, `BASH_SOURCE` | ✗ | ✓ |
 
 
Write POSIX for portability; use bash deliberately for arrays/`[[`/regex — and say so in the shebang.
 
+
## The expansions — the complete taxonomy
+
+
The order matters because each stage feeds the next — this is the full machine:
+
+
```sh
+
{a,b,c}        # 1. brace expansion — a b c — bash only
+
~/x ~user/x    # 2. tilde — home dirs
+
$v ${v}        # 3. parameter expansion — the deep one:
+
  ${v:-default}    # if unset/empty, use default
+
  ${v:=default}    # ...and assign it
+
  ${v:?error msg}  # die if unset — the safety check
+
  ${v:+other}      # if set, use other
+
  ${#v}            # length
+
  ${v#pat}         # strip shortest prefix match
+
  ${v##pat}        # strip longest prefix
+
  ${v%pat}         # strip shortest suffix
+
  ${v%%pat}        # strip longest suffix
+
  ${v/pat/rep}     # substitute — bash
+
  ${v:2:5}         # substring — bash
+
$((1+2*3))     # 4. arithmetic — integers, C-like ops
+
$(cmd)         # 5. command substitution — nests, POSIX
+
`cmd`          #    the legacy form — same thing, uglier
+
<(cmd) >(cmd)  # process substitution — bash: cmd as a file
+
$v word split  # 6. word splitting — unquoted expansions split on IFS
+
*.txt ?.c      # 7. pathname expansion — the globs
+
   *           #    any run (not leading dots)
+
   ?           #    one char
+
   [abc][a-z]  #    classes/ranges
+
   [!a] [^a]   #    negation
+
   globstar ** #    recursive — bash shopt -s globstar
+
'x' "x" \$     # 8. quote removal — after everything
+
```
+
+
The trap order: word-splitting happens *after* parameter expansion — an unquoted `$v` splits on IFS *then* globs — the double hazard `ls $v` suffers.
+
+
## The special variables — $'s secret life
+
+
```sh
+
$$    # this shell's pid
+
$!    # last background job's pid
+
$?    # last exit code — the contract
+
$#    # arg count
+
$0    # the script/shell name
+
$-    # this shell's option flags
+
$@ $* # all args — "each word" vs "one word"
+
$_    # last arg of previous command — bash
+
$PWD $OLDPWD $HOME $PATH $IFS $RANDOM $SECONDS $LINENO
+
```
+
+
## IFS — the splitter
+
+
`IFS` (Internal Field Separator) decides where word-splitting breaks — default `space tab newline`. `IFS=: read -r a b <<< "x:y"` splits on colons; `IFS=$'\n'` splits only on lines — the for-loop safety trick. The `while IFS= read` idiom's `IFS=` prefix prevents leading-whitespace loss per line.
+
+
## Coprocesses, exec-fd magic, and the corners
+
+
```sh
+
exec 3> out.log          # open fd 3 for writing
+
echo hi >&3              # write to it
+
exec 3>&-                # close it
+
exec 8< input.txt        # open for reading
+
read -u8 line            # read from fd 8
+
coproc P { cmd; }        # bash — a process with piped fds
+
cmd1 |& cmd2             # pipe stdout+stderr — bash
+
```
+
+
File-descriptor gymnastics is the shell's deepest layer — redirecting whole script output (`exec > log 2>&1` at the top is the logging boilerplate).
+
+
## POSIX sh vs bash — the extended table
+
+
| Feature | POSIX | bash |
+
| --- | --- | --- |
+
| `[[ ]]`, `=~` regex | ✗ | ✓ |
+
| `<<<` herestring | ✗ | ✓ |
+
| `&>` redirect | ✗ | ✓ |
+
| `{a,b}` braces | ✗ | ✓ |
+
| Arrays, assoc arrays | ✗ | ✓ |
+
| `local`, `declare`, `typeset` | ✗ | ✓ |
+
| `(( ))`, `let` | ✗ | ✓ |
+
| `<(...)`, `>(...)` proc-sub | ✗ | ✓ |
+
| `coproc` | ✗ | ✓ |
+
| `select` | ✗ | ✓ |
+
| `globstar`, `extglob` | ✗ | ✓ |
+
| `${v:offset:len}` | ✗ | ✓ |
+
| `pipefail`, `BASH_SOURCE` | ✗ | ✓ |
+
| `trap ... ERR`, `RETURN` | ✗ | ✓ |
+
| `$(cmd)`, `${v:-d}`, `for/case/while` | ✓ | ✓ |
+
+
## The startup file matrix — which runs when
+
+
| Invocation | Reads |
+
| --- | --- |
+
| login shell (`su -`, console login, `bash -l`) | /etc/profile → ~/.bash_profile (or .profile) |
+
| interactive non-login (new xterm) | ~/.bashrc |
+
| non-interactive (`bash script.sh`) | $BASH_ENV if set |
+
| sh-as-sh | /etc/profile + ~/.profile on login |
+
| POSIX login shell | /etc/profile + ~/.profile |
+
+
The classic confusion: `.bashrc` edits don't affect scripts — scripts aren't interactive and read no files.
+
 
## Where shells go wrong
 
 
| Symptom | Cause |
 
| --- | --- |
 
| Variable empty in script, set in .bashrc | Startup files don't run non-interactively |
 
| `command not found` right after install | bash hash cache — `hash -r` or `set +h` |
 
| `rm: no such file` on a file that exists | Unquoted var word-split on spaces |
 
| `2>&1` captures nothing | Order wrong — `> file 2>&1` not `2>&1 > file` |
 
| Script works in bash, fails in sh | Bashisms — `[[`, arrays, `<<<` in a `#!/bin/sh` script |
 
| Glob didn't expand | Quoted it — `"*.txt"` is literal |
 
| Ctrl-C doesn't kill it | SIGINT ignored/trapped — try `kill -9` |
 
 
## Exercises
 
 
1. Predict, then check: `echo $(echo $(echo deep))` — nesting.
 
2. `ls *.{c,h}` vs `ls "*.txt"` — which is literal?
 
3. `false && echo no || echo yes` — trace the logic.
 
4. `set -x; ls nonexistent; set +x` — watch the trace.
 
5. Write a one-liner proving `cd` must be a builtin.
 
6. `env -i bash -c 'env'` — what's in a sterile env?
 
 
## See also
 
 
- [[Linux]] · [[Filesystem]] · [[Core utilities]] · [[Shell scripting]]
 
 
[[Category:Linux]]
 
[[Category:Handbook]]
 
[[Category:Shells]]