Diff — Shell

{{infobox

| title = The Shell

| chapter = Linux Handbook, chapter 9 — POSIX sh first, bash where it earns its keep

| core = The command language · the REPL · the OS's front door

| shells = sh (POSIX) · bash · dash · zsh · ksh — the grammar is mostly shared

| skills = pipes · redirection · globbing · quoting · expansion order

| next | Core utilities — the vocabulary the grammar drives

}}

**The shell is the operating system's front door** — the program that reads your commands, expands them into what the kernel executes, and returns what happened. Everything else in this handbook runs through it. This chapter teaches the **POSIX shell grammar** — the portable core every `sh` speaks — with **bash**'s extensions flagged where they genuinely earn their keep. Write POSIX, reach for bash deliberately: that's the discipline.

## Which shell is which

```sh

echo $SHELL            # your login shell

ls -l /bin/sh          # WHAT sh actually is on this system — varies!

```

`/bin/sh` is a symlink: to `bash` on Slackware, `dash` on Debian, `bash` in POSIX mode elsewhere. The distinction matters — scripts shebang'd `#!/bin/sh` must be *portable* (no bashisms), while `#!/bin/bash` buys the extensions. `dash` exists specifically to be the fast, strict POSIX sh that doesn't forgive sloppy scripts.

## The grammar — the eight expansions

When you hit enter, the shell transforms your line in a **fixed order** — learn it once:

```sh

echo ~ $HOME $((1+2)) $(uname) *.txt {a,b} "quoted" 'literal'

# 1 brace  2 tilde  3 param+arith+cmdsubst  4 word-split  5 glob  6 quote-removal

```

Each stage is a tool and a trap:

- **`$var` / `${var}`** — parameter expansion; `${var:-default}`, `${var#prefix}`, `${var%suffix}`, `${#var}` string ops are POSIX

- **`$(cmd)`** — command substitution, nestable, POSIX; backticks are the legacy form

- **`$((expr))`** — arithmetic, integer only

- **`*.txt`** — globbing: `*` any run, `?` one char, `[abc]`/`[a-z]` classes, `[!a]` negation

- **`{a,b,c}`** — brace expansion — *bash only*, not POSIX

- **Tilde** — `~`, `~user`

- **Quotes** — `'...'` fully literal; `"..."` protects spaces but allows `$`; `"$@"` is the special case

## Quoting — where every shell bug lives

```sh

f="my file.txt"

ls $f          # TWO words — word splitting — the classic bug

ls "$f"        # one word — quote variables ALWAYS

echo '$HOME'   # literal $HOME — single quotes stop everything

echo "$HOME"   # expands

```

The rule that never lies: **quote every variable expansion unless you specifically want word-splitting.** `rm -rf $dir` unquoted is how systems die; `rm -rf "$dir"` survives spaces.

## Redirection — the complete grammar

```sh

cmd > out          # stdout to file (truncate)

cmd >> out         # append

cmd 2> err         # stderr

cmd > out 2>&1     # both to one file — order matters!

cmd &> out         # bash shorthand for both

cmd < in           # stdin from file

cmd << EOF         # heredoc — literal input until EOF

cmd <<< "str"      # herestring — bash only

cmd | next         # stdout → stdin — the pipe

cmd1 && cmd2       # run second only if first succeeds

cmd1 || cmd2       # run second only if first fails

cmd ; next         # sequence

cmd &              # background

```

`2>&1` ordering is the subtle one — `> out 2>&1` binds stderr to stdout's *destination*; `2>&1 > out` binds stderr to the *old* stdout (terminal). Order is everything.

## Exit codes — the contract

Every command returns 0–255: **0 = success**, nonzero = failure classes. `$?` reads the last one:

```sh

grep -q match file && echo found

[ -f file ] && echo exists        # test — the conditional primitive

test -d dir || mkdir dir

```

The conditional builtins: `[`/`test` (POSIX) — `-f`/`-d`/`-r`/`-w`/`-x`, `-z`/`-n`, `=`/`!=`, `-eq`/`-lt`/`gt`. bash's `[[ ]]` adds pattern matching and `&&`/`||` inside — not POSIX but genuinely better.

## Job control

```sh

cmd &              # background it

jobs               # list

fg %1              # foreground job 1

bg %1              # resume in background

Ctrl-Z             # suspend

Ctrl-C             # SIGINT — interrupt

Ctrl-D             # EOF — end stdin

nohup cmd &        # survive hangup

```

## Special shell powers

- **Builtins vs binaries** — `type cd echo test` reveals which live in the shell (`cd`, `echo`, `[`, `export`) vs which are `/bin` programs; `type -a` shows all resolutions. Builtins run in-shell — `cd` *must* be a builtin to change your process's cwd.

- **`exec`** — replaces the shell process; `exec program` never returns; `exec > log` redirects the shell itself

- **`set`** — the shell's own switches: `set -e` die on error, `set -u` die on unset vars, `set -x` trace, `set -o pipefail` (bash) — the script-safety preamble

- **`hash`** — bash caches binary locations; `hash -r` clears it (the LFS `set +h` trick)

- **`trap`** — run code on signals: `trap 'cleanup' EXIT`

- **`eval`** — re-parse a string as shell; powerful and dangerous — quote-sanitized input only

## Environment and startup

```sh

export VAR=val     # into the environment — children inherit

env                # dump it

env -i cmd         # the sterile env — LFS's tool

```

Startup files differ by invocation: **login shells** read `/etc/profile` + `~/.profile` (POSIX) — bash adds `~/.bash_profile`; **interactive non-login** read `~/.bashrc`; **non-interactive** (scripts) read neither — which is why a variable set in `.bashrc` doesn't exist inside a script. `su - user` gives a login shell; `su user` doesn't — the `-` matters.

## POSIX vs bash — the honest table

| Feature | POSIX | bash |

| --- | --- | --- |

| `$(cmd)`, `>`, `|`, `&&` | ✓ | ✓ |

| `[[ ]]`, `<<<`, `&>` | ✗ | ✓ |

| `{a,b}` braces | ✗ | ✓ |

| Arrays `arr=(...)` | ✗ | ✓ |

| `local`, `declare` | ✗ | ✓ |

| `${var/pat/rep}`, regex `=~` | ✗ | ✓ |

| `pipefail`, `BASH_SOURCE` | ✗ | ✓ |

Write POSIX for portability; use bash deliberately for arrays/`[[`/regex — and say so in the shebang.

## The expansions — the complete taxonomy

The order matters because each stage feeds the next — this is the full machine:

```sh

{a,b,c}        # 1. brace expansion — a b c — bash only

~/x ~user/x    # 2. tilde — home dirs

$v ${v}        # 3. parameter expansion — the deep one:

  ${v:-default}    # if unset/empty, use default

  ${v:=default}    # ...and assign it

  ${v:?error msg}  # die if unset — the safety check

  ${v:+other}      # if set, use other

  ${#v}            # length

  ${v#pat}         # strip shortest prefix match

  ${v##pat}        # strip longest prefix

  ${v%pat}         # strip shortest suffix

  ${v%%pat}        # strip longest suffix

  ${v/pat/rep}     # substitute — bash

  ${v:2:5}         # substring — bash

$((1+2*3))     # 4. arithmetic — integers, C-like ops

$(cmd)         # 5. command substitution — nests, POSIX

`cmd`          #    the legacy form — same thing, uglier

<(cmd) >(cmd)  # process substitution — bash: cmd as a file

$v word split  # 6. word splitting — unquoted expansions split on IFS

*.txt ?.c      # 7. pathname expansion — the globs

   *           #    any run (not leading dots)

   ?           #    one char

   [abc][a-z]  #    classes/ranges

   [!a] [^a]   #    negation

   globstar ** #    recursive — bash shopt -s globstar

'x' "x" \$     # 8. quote removal — after everything

```

The trap order: word-splitting happens *after* parameter expansion — an unquoted `$v` splits on IFS *then* globs — the double hazard `ls $v` suffers.

## The special variables — $'s secret life

```sh

$$    # this shell's pid

$!    # last background job's pid

$?    # last exit code — the contract

$#    # arg count

$0    # the script/shell name

$-    # this shell's option flags

$@ $* # all args — "each word" vs "one word"

$_    # last arg of previous command — bash

$PWD $OLDPWD $HOME $PATH $IFS $RANDOM $SECONDS $LINENO

```

## IFS — the splitter

`IFS` (Internal Field Separator) decides where word-splitting breaks — default `space tab newline`. `IFS=: read -r a b <<< "x:y"` splits on colons; `IFS=$'\n'` splits only on lines — the for-loop safety trick. The `while IFS= read` idiom's `IFS=` prefix prevents leading-whitespace loss per line.

## Coprocesses, exec-fd magic, and the corners

```sh

exec 3> out.log          # open fd 3 for writing

echo hi >&3              # write to it

exec 3>&-                # close it

exec 8< input.txt        # open for reading

read -u8 line            # read from fd 8

coproc P { cmd; }        # bash — a process with piped fds

cmd1 |& cmd2             # pipe stdout+stderr — bash

```

File-descriptor gymnastics is the shell's deepest layer — redirecting whole script output (`exec > log 2>&1` at the top is the logging boilerplate).

## POSIX sh vs bash — the extended table

| Feature | POSIX | bash |

| --- | --- | --- |

| `[[ ]]`, `=~` regex | ✗ | ✓ |

| `<<<` herestring | ✗ | ✓ |

| `&>` redirect | ✗ | ✓ |

| `{a,b}` braces | ✗ | ✓ |

| Arrays, assoc arrays | ✗ | ✓ |

| `local`, `declare`, `typeset` | ✗ | ✓ |

| `(( ))`, `let` | ✗ | ✓ |

| `<(...)`, `>(...)` proc-sub | ✗ | ✓ |

| `coproc` | ✗ | ✓ |

| `select` | ✗ | ✓ |

| `globstar`, `extglob` | ✗ | ✓ |

| `${v:offset:len}` | ✗ | ✓ |

| `pipefail`, `BASH_SOURCE` | ✗ | ✓ |

| `trap ... ERR`, `RETURN` | ✗ | ✓ |

| `$(cmd)`, `${v:-d}`, `for/case/while` | ✓ | ✓ |

## The startup file matrix — which runs when

| Invocation | Reads |

| --- | --- |

| login shell (`su -`, console login, `bash -l`) | /etc/profile → ~/.bash_profile (or .profile) |

| interactive non-login (new xterm) | ~/.bashrc |

| non-interactive (`bash script.sh`) | $BASH_ENV if set |

| sh-as-sh | /etc/profile + ~/.profile on login |

| POSIX login shell | /etc/profile + ~/.profile |

The classic confusion: `.bashrc` edits don't affect scripts — scripts aren't interactive and read no files.

## Where shells go wrong

| Symptom | Cause |

| --- | --- |

| Variable empty in script, set in .bashrc | Startup files don't run non-interactively |

| `command not found` right after install | bash hash cache — `hash -r` or `set +h` |

| `rm: no such file` on a file that exists | Unquoted var word-split on spaces |

| `2>&1` captures nothing | Order wrong — `> file 2>&1` not `2>&1 > file` |

| Script works in bash, fails in sh | Bashisms — `[[`, arrays, `<<<` in a `#!/bin/sh` script |

| Glob didn't expand | Quoted it — `"*.txt"` is literal |

| Ctrl-C doesn't kill it | SIGINT ignored/trapped — try `kill -9` |

## Exercises

1. Predict, then check: `echo $(echo $(echo deep))` — nesting.

2. `ls *.{c,h}` vs `ls "*.txt"` — which is literal?

3. `false && echo no || echo yes` — trace the logic.

4. `set -x; ls nonexistent; set +x` — watch the trace.

5. Write a one-liner proving `cd` must be a builtin.

6. `env -i bash -c 'env'` — what's in a sterile env?

## See also

- [[Linux]] · [[Filesystem]] · [[Core utilities]] · [[Shell scripting]]

[[Category:Linux]]

[[Category:Handbook]]

[[Category:Shells]]