,A PROCESS IS A RUNNING PROGRAM A PID1 AN ADDRESS SPACE1 OPEN FILE DESCRIPTORS1 A PARENT1 A STATE4 ,EVERY PROCESS ON ,LINUX DESCENDS FROM ,,PID #A " THROUGH FORK"<"> THE WHOLE SYSTEM IS ONE FAMILY TREE4 ,THIS CHAPTER TEACHES YOU TO SEE IT3 PS1 TOP1 SIGNALS1 JOB CONTROL1 AND THE _/PROC FILESYSTEM WHERE THE KERNEL EXPOSES EVERY PROCESS'S INNARDS AS READABLE FILES4 ,THE LIFECYCLE FORK1 EXEC1 WAIT1 EXIT PID "7 FORK"<">2 _/_/ CLONE PARENT AND CHILD BOTH CONTINUE IF " _/_/ CHILD BRANCH EXECVP"<8PROG01 ARGS">2 _/_/ REPLACE THIS PROCESS'S IMAGE ELSE WAIT"<@&STATUS">2 _/_/ PARENT COLLECTS THE CHILD'S EXIT CODE FORK"<"> DUPLICATES2 EXEC"<"> REPLACES THE COPY WITH THE NEW PROGRAM2 WAIT"<"> REAPS4 ,EVERY PROCESS IS BORN A CLONE OF ITS PARENT1 THEN RE-SKINNED AS ITS REAL PROGRAM4 ,THE EXIT CODE "<#J#BEE"> THE CHILD LEAVES IS THE CONTRACT THE PARENT COLLECTS @S8 IN THE SHELL4 ,WHEN A CHILD DIES BEFORE ITS PARENT WAITS1 IT BECOMES A ZOMBIE ;,Z IN PS1 DEAD BUT UNREAPED1 HOLDING ONLY ITS EXIT STATUS4 ,AN ORPHAN WHOSE PARENT DIES IS RE-PARENTED TO ,,PID #A4 ,SEEING THEM PS AND FRIENDS PS AUX _? THE CLASSIC FULL LISTING _? ,,USER ,,PID .0,,CPU .0,,,MEM VSZ RSS TTY STAT START TIME COMMAND,' PS -EF _? ,SYS,V STYLE SHOWS ,,PPID PS -EJ,H _? THE FAMILY TREE PS --FOREST -;E _? VISUAL TREE PGREP -;L SSHD _? PIDS BY NAME PIDOF BASH _? SINGLE NAME PIDS TOP _? THE LIVE VIEW SORT BY ;,M"1 ;,P"1 ;,T" HTOP _? THE BETTER LIVE VIEW PSTREE -;P _? THE TREE WITH PIDS ,,STAT CODES ;,R RUNNING1 ;,S SLEEPING1 ;,D UNINTERRUPTIBLE "1 ;,T STOPPED1 ;,Z ZOMBIE1 ;S SESSION LEADER1 "6 FOREGROUND1 ;L MULTITHREADED4 ;,D-STATE PROCESSES CAN'T BE KILLED THEY'RE MID-SYSCALL IN THE KERNEL4 ,JOB CONTROL THE SHELL'S OWN PROCESSES CMD @& _? BACKGROUND JOBS _? THE JOB TABLE .0#A .0#B 444 FG .0#A _? BRING TO FOREGROUND BG .0#A _? RESUME SUSPENDED IN BACKGROUND ,CTRL-;,Z2 BG _? THE SUSPEND-AND-BACKGROUND COMBO DISOWN .0#A _? REMOVE FROM JOB TABLE SURVIVES SHELL EXIT NOHUP CMD @& _? IMMUNE TO ,,SIGHUP SURVIVES LOGOUT ,CTRL-;,C "7 ,,SIGINT TO THE FOREGROUND GROUP2 ,CTRL-;,Z "7 ,,SIGTSTP2 ,CTRL-_* "7 ,,SIGQUIT "4 JOBS -;L SHOWS PIDS4 ,SESSIONS AND PROCESS GROUPS ,A SESSION GROUPS PROCESS GROUPS UNDER A CONTROLLING TERMINAL SETSID STARTS A NEW ONE "4 ,THE LOGIN SHELL LEADS A SESSION2 PKILL -;U ALICE KILLS ALL HER PROCESSES4 ,DAEMONIZATION "7 FORK1 SETSID1 DETACH TTY1 CHDIR _/1 CLOSE FDS1 REDIRECT STDIO TO _/DEV_/NULL THE CLASSIC DANCE SYSTEMD NOW DOES FOR SERVICES4 ,THE FORK_/EXEC_/EXEC FAMILY THE ;,C-LEVEL ANATOMY PID.-T PID "7 FORK"<">2 _/_/ CLONE PARENT"6CHILD BOTH CONTINUE1 DIFFER IN RETURN IF " _< _/_/ CHILD EXECLP"<8PROG01 8PROG01 8ARG01 ,,NULL">2 _/_/ ,,PATH-SEARCHED EXEC .-EXIT"<#ABG">2 _/_/ ONLY REACHED IF EXEC FAILS _> _/_/ PARENT CONTINUES WITH PID "7 CHILD'S PID INT STATUS2 WAITPID"2 _/_/ COLLECT THE EXIT CODE ,,WIFEXITED"2 ,,WEXITSTATUS"2 _/_/ DECODE IT ,THE EXEC FAMILY3 EXECL_/EXECLP_/EXECLE_/EXECV_/EXECVP_/EXECVPE L"7ARG-LIST1 V"7ARGV-ARRAY1 P"7,,PATH-SEARCH1 E"7CUSTOM ENV4 ,AFTER EXEC THE PROCESS'S WHOLE IMAGE IS REPLACED SAME PID1 NEW PROGRAM4 ,THE ZOMBIE_/ORPHAN LOGIC3 A CHILD EXITS ,,SIGCHLD TO PARENT PARENT WAIT"<">;S REAPED2 PARENT DIES FIRST CHILD REPARENTS TO ,,PID #A4 ,THE PROCESS TREE'S STRUCTURE PSTREE -;P _? THE VISUAL FAMILY FROM SYSTEMD_/,,PID#A DOWN PS -EJ,H _? THE SAME TREE AS A LIST PS -O PID1PPID1PGID1SID1TTY1CMD -;E _? ,,PID #A3 SYSTEMD " _? SSHD BASH VIM _? CRON _? GETTY "<#F"> _? DAEMONS444 ,EVERY PROCESS DESCENDS FROM ,,PID #A THE WHOLE SYSTEM IS ONE FORK TREE4 ,SESSION "7 LOGIN CONTEXT2 PROCESS-GROUP "7 A PIPELINE'S JOBS2 CONTROLLING-TTY "7 WHAT ,CTRL-;,C TARGETS4 ,THE SIGNAL-SEND GRAMMAR KILL ,,PID _? ,,SIGTERM #AE ASK POLITELY KILL -#I ,,PID _? ,,SIGKILL UNCATCHABLE KILL -,,HUP ,,PID _? RELOAD DAEMONS' CONVENTION KILL -,,INT ,,PID _? ,CTRL-;,C EQUIVALENT KILL -,,TSTP ,,PID _? ,CTRL-;,Z EQUIVALENT KILL -,,CONT ,,PID _? RESUME KILL -,,USR#A ,,PID _? APP'S PRIVATE TRIGGER NGINX RELOADS1 DD PRINTS PROGRESS KILL -- -,,PGID _? A PROCESS ,,GROUP KILL -;L _? #A#CA STANDARD "6 ,,SIGRTMIN44,,SIGRTMAX REALTIME KILL -#J ,,PID _? 8DOES IT EXIST80 THE PROBE KILLALL NAME _? BY NAME PKILL -;F PATTERN _? BY CMDLINE PKILL -;U ALICE _? ALL OF ALICE'S ,THE TWO UNCATCHABLES3 ,,SIGKILL AND ,,SIGSTOP THE KERNEL'S GUARANTEES THAT NOTHING CAN REFUSE DEATH OR SUSPENSION4 ,EVERYTHING ELSE A PROCESS MAY TRAP_/HANDLE_/IGNORE4 KILL -#J IS THE EXISTENCE PROBE RETURNS THE ERROR WITHOUT KILLING4 -,,USR#A IS THE CONVENTION APPS USE FOR RELOAD_/STATUS DD PRINTS ITS PROGRESS ON ,,USR#A4 _/PROC THE PROCESS'S INNARDS LS _/PROC_/@S@S _? ,,YOUR SHELL'S PROCESS1 AS FILES CAT _/PROC_/@S@S_/STATUS _? STATE1 MEMORY1 UIDS CAT _/PROC_/@S@S_/CMDLINE _\ TR '_*#J' ' ' _? ITS ARGV LS -;L _/PROC_/@S@S_/FD _? ITS OPEN FILES THE FDS6 CAT _/PROC_/@S@S_/ENVIRON _\ TR '_*#J' '_*N' _? ITS ENVIRONMENT CAT _/PROC_/LOADAVG _? #A_/#E_/#AE-MIN LOAD "6 RUNNING_/TOTAL CAT _/PROC_/MEMINFO _? THE MEMORY LEDGER CAT _/PROC_/CPUINFO _? THE ,C,P,US _/PROC_/,,PID_/FD_/ ALONE IS WORTH THE CHAPTER EVERY OPEN FILE1 SOCKET1 AND PIPE AS A SYMLINK4 LSOF AND LS -;L _/PROC_/"9_/FD SHOW WHO HOLDS WHAT4 ,PRIORITIES NICE AND THE SCHEDULER NICE -;N #AJ CMD _? RUN IT NICER " RENICE "6#E -;P ,,PID _? CHANGE A RUNNING ONE PS -O PID1NI1COMM _? SEE NICE VALUES ,HIGHER NICE "7 LOWER PRIORITY 8NICER0 TO OTHERS4 CHRT1 IONICE1 TASKSET REACH THE DEEPER SCHEDULERS "<,,RT CLASSES1 ,I_/,O PRIORITY1 ,,CPU PINNING">4 ,RESOURCE LIMITS ULIMIT -A _? THIS SHELL'S LIMITS FDS1 CORE SIZE1 PROCS1 MEMORY ULIMIT -;N #DJIF _? RAISE FD CEILING CAT _/PROC_/,,PID_/LIMITS _? A RUNNING PROCESS'S LIMITS TIMEOUT #CJ CMD _? KILL AFTER #CJ;S THE BLUNT INSTRUMENT SYSTEMD'S ,LIMIT,N,P,R,O,C_/,LIMIT,N,O,F,I,L,E THE SERVICE-LEVEL EQUIVALENTS ULIMIT -;N " IS THE ONE YOU'LL ACTUALLY HIT BUSY SERVERS BUMP IT CONSTANTLY4 ,SIGNALS THE COMPLETE TAXONOMY KILL -;L _? #FD TOTAL #A#CA STANDARD1 #CB#FD REALTIME "<,,SIGRTMIN44,,SIGRTMAX"> ,,SIGHUP "<#A"> _? HANGUP TERMINAL CLOSED DAEMONS USE AS 8RELOAD CONFIG0 ,,SIGINT "<#B"> _? ,CTRL-;,C INTERRUPT ,,SIGQUIT "<#C"> _? ,CTRL-_* QUIT "6 CORE DUMP ,,SIGILL "<#D"> _? ILLEGAL INSTRUCTION ,,SIGTRAP "<#E"> _? DEBUGGER BREAKPOINT ,,SIGABRT "<#F"> _? ABORT"<"> THE CORE DUMP ,,SIGFPE "<#H"> _? ARITHMETIC FAULT DIV BY ZERO ,,SIGKILL "<#I"> _? ,,UNCATCHABLE KERNEL KILLS IT NOW ,,SIGSEGV "<#AA"> _? SEGFAULT BAD MEMORY ACCESS ,,SIGPIPE "<#AC"> _? WROTE TO A CLOSED PIPE THE 8BROKEN PIPE0 ,,SIGALRM "<#AD"> _? ALARM"<"> TIMER ,,SIGTERM "<#AE"> _? POLITE TERMINATE THE DEFAULT KILL ,,SIGCHLD "<#AG"> _? A CHILD DIED PARENT REAPS VIA WAIT"<"> ,,SIGCONT "<#AH"> _? RESUME A STOPPED PROCESS ,,SIGSTOP "<#AI"> _? ,,UNCATCHABLE SUSPEND ,,SIGTSTP "<#BJ"> _? ,CTRL-;,Z TERMINAL STOP " ,,SIGUSR#A_/#B "<#AJ_/#AB"> _? USER-DEFINED DAEMONS' PRIVATE TRIGGERS ,THE SEMANTICS3 ,,SIGTERM ASKS "2 ,,SIGKILL EXECUTES "2 ,,SIGHUP IS OVERLOADED AS 8RELOAD YOUR CONFIG0 FOR DAEMONS2 ,,SIGSTOP_/,,SIGCONT ARE THE DEBUGGER'S AND JOB CONTROL'S PAUSE_/RESUME4 TRAP CATCHES THE CATCHABLES TRAP 'CLEANUP' ,,TERM ,,INT IS THE DAEMON-SHUTDOWN PATTERN4 ,NUMBERS #CB#FD ARE THE REALTIME RANGE3 #CB#CC ARE RESERVED FOR GLIBC'S THREADING INTERNALS1 AND #CD#FD ARE ,,SIGRTMIN44,,SIGRTMAX "<;,C3 ,,SIGRTMIN"6N"> SAME DELIVERY1 BUT THEY QUEUE " AND CAN CARRY AN INTEGER PAYLOAD ,,POSIX TIMERS1 ,,AIO1 AND MQ.-NOTIFY USE THEM4 KILL -,,RTMIN"6#E ,,PID SENDS ONE2 KILL -;L LISTS ALL #FD SLOTS4 ,SESSIONS1 GROUPS1 AND THE CONTROLLING TERMINAL PS -O PID1PPID1PGID1SID1TTY1CMD -;E _\ HEAD _? THE WHOLE HIERARCHY SETSID CMD _? NEW SESSION DETACH FROM THE TERMINAL DISOWN .0#A _? DROP FROM JOB TABLE SURVIVE EXIT NOHUP CMD @& _? ,,SIGHUP-IMMUNE ,A SESSION "7 A LOGIN OR DETACHED CONTEXT2 A PROCESS GROUP "7 A PIPELINE'S JOBS KILL -,,PID TARGETS A PID1 KILL -- -,,PGID A GROUP4 ,THE CONTROLLING TERMINAL "<,,TTY IN PS"> IS WHAT ,CTRL-;,C SIGNALS HIT DAEMONS SHED IT VIA SETSID4 CGROUPS "6 NAMESPACES THE CONTAINER PRIMITIVES LS _/SYS_/FS_/CGROUP_/ _? THE CGROUP HIERARCHY THE REAL CONTAINER LAYER CAT _/PROC_/,,PID_/CGROUP _? A PROCESS'S CGROUP SYSTEMD-CGLS _? SYSTEMD'S CGROUP TREE UNSHARE -;U -;N -;P BASH _? A BASH IN FRESH NAMESPACES A PROTO-CONTAINER LSNS _? LIST NAMESPACES ,NAMESPACES " ISOLATE2 CGROUPS METER "4 ,CONTAINERS ARE THESE TWO PRIMITIVES ARRANGED DOCKER RUN "7 NAMESPACES "6 CGROUPS "6 AN IMAGE1 NOTHING MORE4 ,THE SCHEDULER'S DEEPER KNOBS NICE -;N #AJ CMD _? -#BJ44#AI HIGHER "7 ,,LOWER PRIORITY RENICE -;N #E -;P ,,PID _? LIVE-TUNE CHRT -;F #EJ CMD _? ,,SCHED.-,,FIFO REALTIME CHRT -;R #EJ CMD _? ,,SCHED.-,,RR IONICE -C#C CMD _? IO PRIORITY IDLE-CLASS TASKSET -;C #J1A CMD _? ,,CPU PINNING ,REALTIME CLASSES BEAT NICE ,,SCHED.-,,FIFO_/,,RR PREEMPT EVERYTHING ELSE2 IONICE AND TASKSET SHAPE ,I_/,O AND ,,CPU AFFINITY4 ,MONITORING BEYOND TOP VMSTAT #A _? MEMORY "6 ,,IO "6 ,,CPU1 PER SECOND IOSTAT -;X #A _? DISK ,I_/,O MPSTAT -;,P ,,ALL #A _? PER-,,CPU DSTAT _? THE EVERYTHING-MONITOR FREE -;H _? MEMORY NOW WATCH -N#A 'PS AUX --SORT"7-.0CPU _\ HEAD' _? YOUR OWN LIVE MONITOR ,LOAD AVERAGE " "7 RUNNABLE PROCESSES AVERAGED OVER #A_/#E_/#AE MIN #D4JJ ON #D ,C,P,US "7 SATURATED2 ON #H ,C,P,US "7 HALF-LOADED4 ,INTERPRET RELATIVE TO CORE COUNT4 ,WHERE PROCESS WORK GOES WRONG ,SYMPTOM ,CAUSE KILL DIDN'T KILL IT ,PROCESS IGNORED ,,SIGTERM ESCALATE TO -#I KILL -#I DIDN'T KILL IT ;,D STATE MID-SYSCALL1 OFTEN DEAD ,,NFS_/DISK ,I_/,O ,ZOMBIE PILING UP ,PARENT NOT WAIT"<">ING A PARENT BUG2 KILL THE PARENT ,DAEMON DIES AT LOGOUT ,NEEDED NOHUP_/SETSID_/DISOWN ,,SIGHUP ,CTRL-;,C DOESN'T WORK ,PROCESS TRAPS OR IGNORES ,,SIGINT ,CAN'T FIND THE PROCESS PS -EF GREP NAME2 PGREP -;F FOR CMDLINE 8,TOO MANY OPEN FILES0 ULIMIT -;N THE FD CEILING ,EXERCISES #A4 SLEEP #CJJ @& JOB CONTROL3 JOBS1 FG1 ,CTRL-;,Z1 BG1 KILL4 #B4 LS -;L _/PROC_/@S@S_/FD INSPECT YOUR SHELL'S OWN OPEN FILES4 #C4 BASH -;C 'SLEEP #II' THEN PS -EJ,H FIND THE FORK RELATIONSHIP4 #D4 KILL -,,TSTP A PROCESS THEN -,,CONT FREEZE AND THAW IT4 #E4 WATCH -N#A CAT _/PROC_/LOADAVG WATCH LOAD BREATHE4 #F4 ULIMIT -;N YOUR FD CEILING2 THEN RAISE IT AND NOTE WHERE IT MATTERS4 ,TIMELINE ,CONCEPT ,MECHANISM ,BIRTH FORK"<"> EXEC"<"> ,DEATH EXIT"<"> ZOMBIE WAIT"<"> ,,PID #A ,THE ANCESTOR OF ALL SYSTEMD_/,SYS,VINIT ,SIGNALS ,INTER-PROCESS INTERRUPTS _/PROC ,THE KERNEL'S PROCESS WINDOW ,SEE ALSO - ,LINUX ,SHELL ,CORE ,UTILITIES ,,LFS ,KERNEL AND ,BOOT