Article illustration 1

Every kernel developer has faced this maddening scenario: You modify a header file buried deep in the source tree, rebuild, and... nothing happens. No recompilation. No error. Just silence. As one developer's quest to list all build files reveals, this frustration stems from the kernel's deliberate—and often misunderstood—handling of dependency tracking.

The Dependency Illusion

At first glance, solution seems straightforward:
1. Use .config for enabled options
2. Parse Makefiles for compiled .c files
3. Resolve #include directives recursively

Reality shatters this plan when encountering macros like:

#include TRACE_INCLUDE(TRACE_INCLUDE_FILE)

Static parsing fails because TRACE_INCLUDE_FILE is evaluated during preprocessing. The compiler's -MMD flag generates .d dependency files with true header trees, but they vanish instantly after creation.

The Ninja Cleanup: .d Files vs. .cmd Files

The kernel build system executes a precise ritual:
1. Compiler generates .d file via -MMD
2. fixdep tool processes it
3. Outputs condensed .cmd file
4. Deletes .d file

This .cmd file (e.g., net/core/.sock.o.cmd) contains:
- Full compile command
- Direct headers only
- CONFIG_* wildcards

cmd_and_fixdep = \\
    $(cmd); \\
    scripts/basic/fixdep $(depfile) $@ '$(make-cmd)' > $(dot-target).cmd;\\
    rm -f $(depfile)  # The critical deletion

The Recursion Gap

.cmd files sacrifice completeness for Kbuild integration:

Method Pros Cons
.d files Accurate, compiler-generated Deleted after use
.cmd files Tracks CONFIG_*, Kbuild-friendly No nested headers
strace build Comprehensive Extremely noisy
Static parsing Theoretically possible Fails on macros like TRACE_INCLUDE

This explains why changing deep headers doesn't trigger rebuilds—.cmd files lack recursive dependencies. The fix? Manually deleting object files, a hack every veteran developer knows.

The Extraction Workaround

The solution lies in repurposing existing artifacts:

# 1. Find all .cmd files in build directory
# 2. Extract compile commands
# 3. Re-run with -MMD (skip fixdep and deletion)

This regenerates full .d files with complete header trees. Though computationally expensive, it provides accurate, unaltered dependencies without modifying the build system.

Beyond the Build

This dependency opacity reveals a kernel design philosophy: Optimize for incremental builds, not transparency. The fixdep trade-off prioritizes build speed over exhaustive tracking—a reasonable choice for daily development, but a hurdle for auditing or reproducibility. As one developer wryly notes, sometimes the only way forward is to embrace the kernel's chaos and outsmart it with its own tools.

Source: Carmine T. Alessandro