Unlocking the Linux Kernel's Dependency Secrets: Why Your Header Changes Go Unnoticed

Tracking every file in a Linux kernel build seems trivial until you encounter disappearing dependency files and untracked header changes. Discover why the kernel deliberately deletes critical build artifacts and the clever workaround to reveal true dependencies.

Every kernel developer has faced this maddening scenario: You modify a header file buried deep in the source tree, rebuild, and... nothing happens. No recompilation. No error. Just silence. As one developer's quest to list all build files reveals, this frustration stems from the kernel's deliberate—and often misunderstood—handling of dependency tracking.

The Dependency Illusion

At first glance, solution seems straightforward:

Use .config for enabled options
Parse Makefiles for compiled .c files
Resolve #include directives recursively

Reality shatters this plan when encountering macros like:

#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)

Static parsing fails because TRACE_INCLUDE_FILE is evaluated during preprocessing. The compiler's -MMD flag generates .d dependency files with true header trees, but they vanish instantly after creation.

The Ninja Cleanup: .d Files vs. .cmd Files

The kernel build system executes a precise ritual:

Compiler generates .d file via -MMD
fixdep tool processes it
Outputs condensed .cmd file
Deletes .d file

This .cmd file (e.g., net/core/.sock.o.cmd) contains:

Full compile command
Direct headers only
CONFIG_* wildcards

cmd_and_fixdep = \\
    $(cmd); \\
    scripts/basic/fixdep $(depfile) $@ '$(make-cmd)' > $(dot-target).cmd;\\
    rm -f $(depfile)  # The critical deletion

The Recursion Gap

.cmd files sacrifice completeness for Kbuild integration:

Method	Pros	Cons
.d files	Accurate, compiler-generated	Deleted after use
.cmd files	Tracks CONFIG_*, Kbuild-friendly	No nested headers
strace build	Comprehensive	Extremely noisy
Static parsing	Theoretically possible	Fails on macros like `TRACE_INCLUDE`

This explains why changing deep headers doesn't trigger rebuilds—.cmd files lack recursive dependencies. The fix? Manually deleting object files, a hack every veteran developer knows.

The Extraction Workaround

The solution lies in repurposing existing artifacts:

# 1. Find all .cmd files in build directory
# 2. Extract compile commands
# 3. Re-run with -MMD (skip fixdep and deletion)

This regenerates full .d files with complete header trees. Though computationally expensive, it provides accurate, unaltered dependencies without modifying the build system.

Beyond the Build

This dependency opacity reveals a kernel design philosophy: Optimize for incremental builds, not transparency. The fixdep trade-off prioritizes build speed over exhaustive tracking—a reasonable choice for daily development, but a hurdle for auditing or reproducibility. As one developer wryly notes, sometimes the only way forward is to embrace the kernel's chaos and outsmart it with its own tools.

Source: Carmine T. Alessandro