Kernel Deep Dive

The Linux kernel is the largest single C project on Earth — ~30 million lines, ~80,000 files, contributed to by tens of thousands. Yet it's a program like any other: you can download it, configure it, compile it, and run the result. This chapter walks the kernel as code — the source tree's map, the .config that shapes it, the build, the modules, and the live interfaces (/proc, /sys, sysctl) it exposes.

The source tree's map

git clone --depth 1 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
# or: wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.x.tar.xz

ls linux/
# arch/     — per-architecture code — x86, arm, riscv...
# kernel/   — the core: scheduler, signals, timers, printk
# mm/       — memory management: paging, slab, swap
# fs/       — filesystems: VFS layer + ext4, btrfs, proc, sysfs...
# drivers/  — the bulk: every device driver on Earth
# net/      — the network stack
# include/  — headers incl. the UAPI boundary (include/uapi)
# init/     — main.c — where the kernel begins
# sound/, security/, ipc/, block/, crypto/, lib/, virt/, Documentation/

init/main.c's start_kernel() is where it all begins; kernel/sched/ holds the scheduler; fs/ the VFS; drivers/ is ~half the tree by size. Reading the tree once makes the kernel less mythical.

The .config — the kernel's personality

make defconfig        # sane defaults for your arch
make menuconfig       # THE interface — the ncurses option tree
make oldconfig        # upgrade a .config across versions
make olddefconfig     # same, auto-defaults
make localmodconfig   # only the modules your hardware uses
grep CONFIG_EXT4 .config
scripts/config -e CONFIG_EXT4_FS    # script edits

Every CONFIG_* option is a C macro — [*] built-in, [M] module, [ ] absent. The critical divides: built-in = in bzImage, available instantly, needed for boot-critical things (rootfs driver, disk controller); module = a .ko file, loadable after boot, fine for everything else. The boot-critical rule — can't load a module from a filesystem you can't yet mount — is the chapter 6 panic's explanation.

The build

make -j$(nproc)              # bzImage + all [M] modules — hours → minutes
make modules                 # just modules
make modules_install         # → /lib/modules/<ver>/
make install                 # → /boot + bootloader hooks (distro-dependent)
# the manual form (chapter 6):
cp arch/x86_64/boot/bzImage /boot/vmlinuz-<ver>
cp System.map /boot/; cp .config /boot/config-<ver>
make clean / mrproper        # the clean levels — mrproper deletes .config too

Artifacts: bzImage (the compressed kernel), System.map (symbol→address table — debugging's map), .config (the recipe — keep forever; every later kernel starts from it).

Modules — the pluggable kernel

lsmod                        # loaded modules
modinfo ext4                 # a module's metadata
modprobe e1000e              # load with deps
modprobe -r e1000e           # unload
insmod foo.ko                # load by file (no deps)
rmmod foo                    # raw unload
depmod -a                    # rebuild the dep map
ls /lib/modules/$(uname -r)/ # the module tree
modprobe.d/*.conf            # blacklists + options

Modules load on demand — udev/modprobe hotplug them as hardware appears. modprobe blacklists keep problem modules out.

The live interfaces — reading the running kernel

uname -a                     # version + build metadata
cat /proc/version            # who compiled it, when
cat /proc/cmdline            # the boot parameters
sysctl -a                    # every tunable
sysctl -w net.ipv4.ip_forward=1   # live-tune — write a file, change a kernel
cat /proc/sys/kernel/hostname
dmesg                        # the kernel's own log
ls /sys/devices/             # the device object tree

sysctl/proc-sys is the kernel's control panel — networking, memory, security knobs, all writable files.

Versioning — the numbers

6.9.3 = major.minor.patch; .x releases are the stable series; LTS versions (6.1, 6.6, 6.12…) get years of backports; -rc are pre-releases. uname -r reports yours; distro kernels carry suffixes (-amd64, -generic, -lts) and distro patches atop the upstream version.

menuconfig — the search and the structure

Inside make menuconfig:

• / — search — type a CONFIG_ name, jump to it
• ? — help on the highlighted option — the real documentation
• z — toggle between [*], [M], [ ]
• space — toggle; h help; arrows navigate

The tree's shape: Processor type and features (CPU), Device Drivers (half the kernel — storage, net, USB, input, graphics), File systems, Networking support, Security options, Kernel hacking (debug), Power management, Virtualization, Executable file formats. The three things that matter for a first build: your storage controller (Device Drivers → Serial ATA/SCSI/NVMe), your root filesystem (File systems → ext4), Devtmpfs automount (Device Drivers → Generic Driver Options) — plus the NIC.

The build products — what each artifact is

arch/x86_64/boot/bzImage    # the kernel — bzipped + self-decompressing
System.map                  # symbol→address — crash logs decode through it
.config                     # the recipe — the single most valuable file
vmlinux                     # the uncompressed ELF — debugging format
/lib/modules/<ver>/*.ko     # the modules — loadable drivers
Module.symvers              # module symbol versions — external builds need it

make modules_install installs the .ko tree to /lib/modules/<ver>/; depmod builds modules.dep so modprobe resolves chains. make install runs the distro's install hooks (or you copy bzImage manually).

The module lifecycle

modinfo -F version e1000e    # a module's version
modinfo -p e1000e            # its parameters
modprobe e1000e param=val    # load with options
echo 'options e1000e param=val' >> /etc/modprobe.d/e1000e.conf
echo 'blacklist badmod' >> /etc/modprobe.d/blacklist.conf
dkms status                  # out-of-tree modules' rebuilder

Modules live as .ko (or .ko.xz/.zst compressed) under /lib/modules/; modules.dep is the dependency map depmod builds; modprobe.d/ holds options + blacklists.

The UAPI boundary — the kernel/userspace contract

include/uapi/ is the kernel's public interface — the headers userspace compiles against (the LFS chapter-3 headers install). Everything there is the stable ABI — the syscall numbers, ioctl codes, structure layouts — the boundary a kernel cannot break.

Building a custom kernel — the whole move

wget https://cdn.kernel.org/.../linux-6.9.3.tar.xz && tar -xf ...
cd linux-6.9.3
zcat /proc/config.gz > .config   # steal the running kernel's config (if enabled)
# or: cp /boot/config-$(uname -r) .config
make olddefconfig
make menuconfig                  # trim to your hardware — the speedup
make -j$(nproc)
sudo make modules_install
sudo cp arch/x86_64/boot/bzImage /boot/vmlinuz-custom
# add a GRUB entry → reboot → test → keep the old kernel as fallback

Always keep the old kernel — GRUB's menu lets you boot the known-good one; the fallback is the safety net.

Where kernel work goes wrong

Exercises

1. zcat /proc/config.gz | grep -c '=y' — your kernel's built-ins count.
2. modinfo three loaded modules — read their parameters.
3. sysctl -a | wc -l — the knob count.
4. Browse /sys/devices — find your network card's object.
5. make menuconfig a downloaded tree — navigate, don't change; read the structure.

Timeline

References

• kernel.org — the tree, the docs
• Documentation/ inside the source — the kernel documents itself

See also

• LFS Kernel and Boot · Inside a Linux System · Processes · Linux