The C-Shaped Hole in Package Management: Why Two Worlds Can't Talk

The fundamental problem with package management isn't that we have too many package managers—it's that we have two fundamentally different kinds that evolved to solve different problems, and they can't communicate about the one thing they both need: C libraries.

Two Worlds, One Problem

System package managers like apt, dnf, and pacman emerged because Linux distributions needed to assemble coherent systems from independently developed components. The kernel comes from one place, libc from another, coreutils from a third. These tools track what's installed and what depends on what, keeping only one version of each package to simplify dependency resolution. This "stop-the-world" model means getting a newer or older version requires upgrading the entire system.

Language package managers like npm, pip, cargo, and gem took the opposite approach. They're dependency assembly tools that help developers build projects, keeping every version around indefinitely so projects can pin exactly what they need. They're cross-platform by design—pip doesn't know if it's running on Debian or Fedora or macOS, so it can't shell out to the system package manager to install C dependencies even if it wanted to.

The difference shows up in how they handle the same library. A distro maintainer sees a new OpenSSL release and asks: will this break existing applications? Can we backport security fixes without changing behavior? A language package manager asks: does this satisfy version constraints? Can multiple versions coexist?

The C-Shaped Hole

C never developed a canonical package registry. It predates the model of "download dependencies from the internet," and by the time that model became standard, the ecosystem was too fragmented to converge. pkg-config exists as a partial vocabulary for discovering installed libraries, but it's a query mechanism for what's already on your system, not a way to declare or fetch dependencies.

System package managers filled this gap by default. If you need libcurl or OpenSSL or zlib, you install them through apt or dnf or brew. This makes your system package manager the de facto C package manager whether it was designed for that or not. And every distro names packages differently: libssl-dev on Debian, openssl-devel on Fedora, openssl on Alpine, openssl@3 on Homebrew. Same library, four names, no mapping between them.

Every language that needs C bindings solves distribution independently. Python has wheels that bundle compiled extensions. Node has node-gyp that compiles against system headers at install time. Rust has build.rs scripts that call pkg-config. Go has cgo with its own linking story. Ruby has native extensions that compile on gem install. None of these mechanisms really declare C dependencies in a machine-readable way.

The Human Layer

You end up with two dependency graphs that overlap on C libraries but can't communicate. pip knows your Python dependencies. apt knows your system dependencies. Neither knows what the other is doing, and the place where they meet is held together by humans who know which system packages to install before running pip install.

Or the language package just bundles the C library inside itself: Nokogiri ships libxml2, NumPy ships OpenBLAS, and the system package manager never sees them. This creates "phantom dependencies"—dependencies bundled into packages but not represented in metadata. If you want to know what NumPy actually depends on, you need tools like nm or readelf to inspect the binary symbols.

The pypackaging-native project documents this problem thoroughly: "The key problems are (1) not being able to express dependencies in metadata, and (2) the design of Python packaging and the wheel spec forcing vendoring dependencies."

The Bridge Attempts

Conda is probably the most successful attempt at bridging these worlds. It packages C libraries, Python, R, and other languages in a single dependency graph. You can declare dependencies on both libcurl and requests (or libcurl and an R package), and Conda understands what that means. This took off in scientific Python, where C dependencies are a nightmare.

But Conda never became universal. Part of the reason is that it bundles the hard problem (managing C dependencies) with a less compelling solution for the easy problem (pure Python packages). If you don't need compiled extensions, Conda is more than you need.

Nix and Guix come at it another way, with content-addressed storage and reproducibility as core design goals. They're still system package managers, still delivering applications, but they have a better model for mapping language packages into that world. They repackage what's on PyPI and npm, expressing C dependencies alongside Python ones in a single dependency graph.

Making the Invisible Visible

The gap gets filled by humans today. You run pip install, hit a compilation error about missing headers, google which apt package provides them. Dockerfiles accumulate RUN apt-get install lines that encode this knowledge.

But there's a deeper problem: sustainability. NumPy depends heavily on OpenBLAS, but that dependency doesn't appear in its package metadata. The wheel bundles a compiled libopenblas.so.0 inside the package, invisible to pip and invisible to any tool that only looks at Python dependency information. If you want to know what NumPy actually depends on, you need tools like nm or readelf to inspect the binary symbols.

The OpenBLAS maintainers don't get credit for the work that makes NumPy fast. Tracing these connections across package managers and languages is the first step toward fixing that. If software citation takes off in academia, the same problem applies: papers citing NumPy should propagate credit to BLAS, but only if we can trace the dependency.

The same data has security applications. If a C library has a CVE, nobody can currently tell which wheels or npm packages or gems bundle an affected version. Vulnerability scanners look at language package metadata and see nothing. With a symbol database, you could trace from CVE to upstream library to every package that vendors it, across ecosystems.

It also makes SBOMs more accurate: right now, generating an SBOM for a project misses all the C libraries bundled inside packages.

The Way Forward

Tools like Syft crawl container filesystems looking for known binaries and package manifests in well-known locations. It's a blunt instrument because it has to be: the metadata doesn't exist, so Syft reconstructs what it can from filesystem heuristics.

Vlad Harbuz and I have been investigating how to automate this through ecosyste.ms: mine symbols from binaries across ecosystems, build an index that maps symbols to system libraries to upstream projects. The output might look like: [email protected] (PyPI) depends on libopenblas.so.0 (bundled), which is OpenBLAS 0.3.21 upstream, packaged as libopenblas0 in apt, openblas in apk, openblas in rpm, and openblas in brew.

Ultimately I want to index these links into ecosyste.ms alongside the regular dependency data, so the cross-ecosystem connections become queryable like any other dependency. The motivation here is sustainability. Vlad works on the Open Source Pledge, which asks companies to pay maintainers of their dependencies. But dependencies on system libraries aren't surfaced anywhere, which means they're invisible to funding.

This isn't about proposing solutions—it's a wicked problem. The point is that any serious attempt at a protocol for package management needs to grapple with this. Cross-ecosystem dependencies aren't an edge case; they're everywhere, just invisible. Making these implicit dependencies first-class is the missing layer between the two worlds, and it needs to be part of the conversation when that protocol gets investigated further.

Vlad is presenting this at FOSDEM 2026 in the Package Management devroom, which I'm co-organizing. The conversation about package management protocols needs to include this C-shaped hole at its center, because until we can make these implicit dependencies visible, we're building systems on sand.