| | {{infobox |
| | | title = LFS: Building the System Packages |
| | | chapter = Linux Handbook, chapter 5 — ~50 packages built inside the chroot |
| | | stage | LFS book chapter 8 — the longest chapter of the build |
| | | milestone | A complete native toolchain + full userspace |
| | | method = The same ritual, ~50 times: configure, make, check, install |
| | | needs = Inside the chroot with the temp system (chapter 4) |
| | | next = LFS IV — kernel, bootloader, first boot |
| | }} |
| | |
| | **This is the marathon chapter** — some fifty packages compiled *inside* the chroot by the toolchain you built, installing the real system's libraries, utilities, and compilers. Where chapter 4 cross-compiled the scaffolding, chapter 5 builds natively — the temporary toolchain producing the final toolchain and everything around it. The pattern never changes: extract, configure, make, test, install, delete the tree. What changes is the scale — and the discipline the scale demands. |
| | |
| | ## The shape of the marathon |
| | |
| | The book's chapter 8 is a long procession. Grouped by role, it runs roughly: |
| | |
| | - **Compression + archive** — zstd, xz, bzip2, gzip, tar |
| | - **Foundation libraries** — zlib, ncurses (real install), readline, libffi, expat, libxml2, libxcrypt |
| | - **Build machinery** — m4, bison, flex, autoconf, automake, libtool, pkgconf, check, libtasn1 |
| | - **The toolchain, natively** — binutils, gmp, mpfr, mpc, isl, gcc — the *real* ones this time |
| | - **Core userland** — attr/acl, shadow (login/passwd/su), bash, coreutils, diffutils, findutils, gawk, grep, sed, procps-ng, psmisc, util-linux, e2fsprogs, file, inetutils, less, iproute2, kbd, kmod, patch, vim |
| | - **Scripting** — perl, python, wheel/setuptools, ninja, meson, texinfo, gettext |
| - | - **System plumbing** — elfutils, libelf, libpipeline, man-db, man-pages, groff, sysklogd, sysvinit, eudev, dbus-adjacent pieces |
| - | - **Security** — openssl, libcap-ng, markupsafe/jinja2 adjacents, gperf, libseccomp |
| + | - **System plumbing** — elfutils, libelf, libpipeline, man-db, man-pages, groff, sysklogd, sysvinit, eudev (dbus lives in the systemd edition / BLFS — the SysV book omits it) |
| + | - **Security** — openssl, libcap, markupsafe/jinja2 adjacents, gperf, libseccomp |
| | |
| | Each is the same liturgy with its own flags and fix-ups — fifty variations on a theme you've already played fifteen times. |
| | |
| | ## The ritual, refined |
| | |
| | ```sh |
| | cd /sources # inside chroot — no $LFS prefix anymore |
| | tar -xf xz-*.tar.xz |
| | cd xz-*/ |
| | ./configure --prefix=/usr \ |
| | --disable-static \ |
| | --docdir=/usr/share/doc/xz-<ver> |
| | make |
| | make check # run the testsuite where the book says to |
| | make install |
| | cd .. && rm -rf xz-*/ |
| | ``` |
| | |
| | The refinements inside the chroot: `--prefix=/usr` without DESTDIR — you're really installing now; `--disable-static` avoids shipping static libraries where shared ones serve; `make check` runs testsuites the book flags as skippable-only-at-your-peril. **A failing testsuite inside the chroot is a real failure** — unlike the temp stage, these packages are the finished product. |
| | |
| | ## The three packages that matter most |
| | |
| | **GCC (native)** — the final compiler, all languages, threads, libstdc++ complete. After it, the `cc`→`gcc` link is permanent, and the toolchain you just built *replaces* the one that built it — the build pulls itself up by its own bootstraps. |
| | |
| | **Binutils (native)** — the linker the finished system will use. |
| | |
| | **Shadow** — the login/password suite: `useradd`, `passwd`, `su`, login itself. Its install turns `/etc/passwd` from the chapter-4 placeholder into a real account database, and includes the crypto bindings (via libxcrypt) that make `passwd` hash properly. |
| | |
| | ## The toolchain verification — the second proof of life |
| | |
| | After binutils+gcc install, the book's toolchain check verifies the *real* compiler: |
| | |
| | ```sh |
| | echo 'int main(){}' > dummy.c |
| | cc dummy.c -v -Wl,--verbose &> dummy.log |
| | readelf -l a.out | grep ': /lib' |
| | # correct: [Requesting program interpreter: /lib64/ld-linux-x86-64.so.2] |
| | # (or /lib/ld-linux.so.2 on 32-bit) |
| | grep -E -o '/usr/lib.*/S?crt[1in].*succeeded' dummy.log # crt files found |
| | grep 'SEARCH.*/usr/lib' dummy.log | sed 's|; |\n|g' # search paths right |
| | grep found dummy.log # right libstdc++ |
| | rm -v dummy.c a.out dummy.log |
| | ``` |
| | |
| | This is chapter 5's readelf moment — the native toolchain proving it looks inward, not at the host. |
| | |
| | ## The marathon's pacing |
| | |
| | Fifty packages take most of a day even on modern hardware. Pace and hygiene: |
| | |
| | - **Re-extract every tarball fresh** — leftover configure caches from earlier passes are a classic poisoner |
| | - **Delete the tree after install** — disk and confusion both |
| | - **`make check` where the book asks** — a test failure *now* is a bug found cheap; skip it and the bug resurfaces at boot |
| | - **Read the "Contents" + "Short Descriptions" per package** — you should know *what* you installed; that's the whole point of LFS |
| | - **Log everything** — the journal from chapter 2 pays off when package 47 misbehaves |
| | |
| | ## The testsuite grammar — when check matters |
| | |
| | ```sh |
| | make check # where the book asks — the tests |
| | make -k check # continue through failures — see them all |
| | make check 2>&1 | tee check.log # keep the record |
| | ``` |
| | |
| | The book marks which testsuites are safe to skip and which to run — glibc's is expensive-but-optional on a temp toolchain, mandatory-ish on the native build; gcc's catches miscompiles early; coreutils' and util-linux's are the ones that bite. A documented-expected failure (locale-dependent tests under `LC_ALL=POSIX`, known-broken upstream tests) may pass; an *unexpected* one is a real bug. |
| | |
| | ## The strip/cleanup — the final shrink |
| | |
| + | Late in the chapter the book cleans up — and the *way* it strips matters: |
| + | |
| | ```sh |
| - | # late-chapter — the book's cleanup |
| - | rm -rf /tmp/{*,.*} |
| - | find /usr/lib /usr/libexec -name \*.la -delete |
| - | find /usr -depth -name \( -name \*.o -o -name \*.a \) ! -name \*libc_nonshared.a -delete |
| - | # strip the symbols — the size drop |
| - | save_usrlib="$(cd /usr/lib; ls ld-linux* 2>/dev/null)" |
| - | strip --strip-unneeded --preserve-dates /usr/bin/* /usr/sbin/* /usr/lib/* 2>/dev/null |
| - | unset save_usrlib |
| - | # temp tools out — the scaffolding comes down |
| - | rm -rf /tools |
| - | rm -rf /usr/lib/{libcrypto.la,libssl.la} 2>/dev/null # any stragglers |
| + | rm -rf /tmp/{*,.*} # build residue |
| + | find /usr/lib /usr/libexec -name \*.la -delete # dead libtool files |
| + | find /usr -depth -name \( -name \*.o -o -name \*.a \) \ |
| + | ! -name \*libc_nonshared.a -delete # dead object archives |
| | ``` |
| | |
| - | `.la` files bake absolute paths; `.a` archives are useless once shared libs exist; `--strip-unneeded` drops debug symbols (the binary size falls dramatically); `--preserve-dates` keeps mtimes sane. `/tools` — the temporary toolchain — is removed at the end of chapter 8: it served its purpose; the native toolchain replaces it. |
| + | **You cannot strip in place.** The binaries and libraries currently running — the bash you're typing in, the glibc it's linked against — are mmap'd; `strip` rewriting them in place fails or corrupts. The book's procedure copies each to `/tmp`, strips the copy, installs it back: |
| | |
| - | ## The cleanup and the system that emerges |
| + | ```sh |
| + | # the book's shape — strip via /tmp, never in place: |
| + | save_usrlib="$(cd /usr/lib; ls ld-linux* libc.so.6 libstdc++.so* libm.so.6 ...)" |
| + | for LIB in $save_usrlib; do |
| + | objcopy --only-keep-debug --compress-debug-sections=zstd $LIB $LIB.dbg |
| + | strip --strip-unneeded /tmp/$LIB # strip the COPY |
| + | objcopy --add-gnu-debuglink=$LIB.dbg /tmp/$LIB |
| + | install -vm755 /tmp/$LIB /usr/lib # install back atomically |
| + | done |
| + | # same dance for the running binaries: bash find strip get /tmp-copied first |
| + | ``` |
| | |
| - | Late in the chapter, the book strips debugging symbols and removes the scaffolding: |
| + | Then the scaffolding comes down: |
| | |
| | ```sh |
| - | rm -rf /tmp/{*,.*} # build residue |
| - | find /usr/lib /usr/libexec -name \*.la -delete # dead libtool files |
| - | find /usr -depth -name \( -name \*.o -o -name \*.a \) ! -name \*libc_nonshared.a -delete |
| - | strip --strip-unneeded /usr/bin/* /usr/lib/* 2>/dev/null # optional, per book |
| + | rm -rf /tools # the temporary toolchain — its job is done |
| | ``` |
| | |
| - | Then **temporary tools come out** — the `/tools` tree and the now-redundant packages it enabled get removed, shrinking the system to just what the real packages installed. What's left is a complete Linux userspace: shell, utilities, compiler, libraries, account machinery, log files — everything but a kernel and a boot. |
| + | `.la` files bake absolute build paths; `.a` archives are dead weight once shared libs exist; `--strip-unneeded` drops the debug fat (with the `.dbg` files preserving debuggability via `gnu-debuglink`). `/tools` removal at the end of chapter 8 is the symbolic end of the build's dependency on its own scaffolding — everything after comes from the native toolchain. |
| | |
| | ## The packages in detail — what each earns its place for |
| | |
| | The marathon's cast, grouped by role: |
| | |
| | - **The compressors** — zlib (the base lib everything links), bzip2, xz, zstd — every tarball and every lib decompresses through these |
| | - **The foundation libs** — ncurses (terminfo + curses for every TUI), readline (the line-editing every REPL uses), libffi (foreign-function calls), expat/libxml2 (XML parsing), libxcrypt (the password-hash lib shadow needs) |
| | - **The build machinery** — m4 → bison/flex (parser generators) → autoconf/automake (configure generators) → libtool → pkgconf — the packages that *build packages* |
| | - **The toolchain, final** — binutils (real linker), gcc (all languages), gdb (debugger) — the system now compiles itself |
| | - **The userland** — attr/acl (extended attributes), libcap (capabilities), shadow (accounts), bash (the shell), coreutils/diffutils/findutils/gawk/grep/sed (the vocabulary), procps-ng (ps/top), psmisc (killall/fuser), util-linux (mount/fdisk/dmesg/lsblk — the plumbing), e2fsprogs (ext4 tools), file, less, iproute2 (ip/ss), kbd (console), kmod (module tools), inetutils, groff (man-page formatting), man-db + man-pages (the manuals), vim |
| | - **Scripting + build** — perl (everything ancient needs it), python (meson's engine + half the world's glue), wheel/setuptools, ninja + meson (the modern build pair), texinfo, gettext (i18n), markupsafe/jinja2 (meson deps) |
| | - **Crypto + plumbing** — openssl (TLS + the crypto lib), libtasn1/p11-kit (certificate machinery), elfutils/libelf (ELF tooling), libpipeline (man-db's pipes), libseccomp (syscall filters), eudev (device manager), sysklogd (logging), sysvinit (PID 1 itself) |
| | |
| | The ordering is topological — each package's deps built earlier; the book's sequence *is* the dependency graph linearized. |
| | |
| | ## The testsuite discipline |
| | |
| | `make check` runs where the book flags it — the failing tests are *information*: which host assumption leaked, which dependency landed wrong. Some failures are documented as expected (locale-dependent tests under `LC_ALL=POSIX`); the book marks them. The rule: a testsuite failure you *understand* may be skipped; one you don't is a bug in waiting. |
| | |
| | ## The Iana-Etc / man-pages / man-db trio |
| | |
| | Three quiet packages make the system *human*: `iana-etc` supplies `/etc/services` and `/etc/protocols` (the name↔number maps every network tool leans on); `man-pages` gives you documentation for the kernel interfaces you built against; `man-db` makes `man` work. Skipping them gives a system that functions but can't be questioned. |
| | |
| | ## Where this chapter goes wrong |
| | |
| | | Symptom | Cause | |
| | | --- | --- | |
| | | Testsuite failures across many packages | Temp system incomplete — back to chapter 4 | |
| | | configure can't find a lib it *should* find | Ordering — the book's sequence is dependency-sorted; out-of-order builds fail | |
| | | `/usr/lib` paths wrong in binaries | prefix flag missed — `--prefix=/usr` vs DESTDIR confusion | |
| | | `cc` missing late in the chapter | gcc pass-2's symlink skipped | |
| | | `su`/`passwd` broken | Shadow's libxcrypt/PAM flags wrong | |
| | | Disk full mid-chapter | Old build trees not deleted — `rm -rf` after each install | |
| | | Package installs to `/tools` | configure prefix accidentally still `/tools` | |
| | |
| | ## Exercises |
| | |
| | 1. Pick any package; write its configure line *before* reading the book's — compare. |
| | 2. Find three packages whose `make check` is explicitly skippable — and three where it isn't. Why the difference? |
| | 3. `ldd` three freshly-built binaries — prove they link the chroot's glibc, not the host's. |
| | 4. Track your build times — which package burned the most SBUs? (Usually gcc.) |
| | 5. Explain why `/etc/services` matters — find a tool that consults it. |
| | |
| | ## Timeline |
| | |
| | | Step | Result | |
| | | --- | --- | |
| | | ~50 packages | The real userspace, natively built | |
| | | Native binutils+gcc | The system compiles itself | |
| | | Shadow | Real accounts, real passwords | |
| | | Verification | readelf proves inward-looking toolchain | |
| | | Cleanup | Temp tools out, symbols stripped | |
| | | next | LFS IV — kernel, bootloader, first boot | |
| | |
| | ## References |
| | |
| | - linuxfromscratch.org — chapter 8: installing basic system software; stripping; cleaning |
| | - Each package's own documentation — the book's flags come from upstream |
| | |
| | ## See also |
| | |
| | - [[LFS Preparation]] · [[LFS Cross-Toolchain]] · [[LFS Temporary System]] · [[Linux]] |
| | |
| | [[Category:Linux]] |
| | [[Category:LFS]] |
| | [[Category:Handbook]] |
| | |
