The container revolution owes much to Docker’s user-friendly approach, but its foundational design harbors a fundamental paradox. At its core, Docker’s OCI (Open Container Initiative) layers operate as a simple list—a linear sequence of filesystem changes. Yet, software dependencies form a Directed Acyclic Graph (DAG), where multiple components can depend on a shared resource. A list, even as a topological sort of a DAG, is a crude approximation of this reality. As one blunt critique puts it, using a list to represent a DAG is "fucking terrible"—a design choice born not of technical necessity, but of developer psychology and commercial incentives.

The story begins at dotCloud, Inc., a Platform-as-a-Service (PaaS) provider in the early 2010s. Customers struggled to deploy complex applications on their platform. dotCloud’s solution was radical: package entire OS images into a system mimicking version control history. Each Docker layer became a commit in a filesystem timeline. This paradigm resonated with developers fluent in Git but oblivious to build system intricacies. As the source notes, "anyone who knows a little more about installing operating systems... can tell you that a literally materialized history of changes that the system has undergone is a fucking terrible way to describe OS build instructions."

The consequences were immediate and measurable. A simple "Hello World" web server could balloon to 2GB of disk space due to duplicated layers and inefficient dependency tracking. Yet, this inefficiency served the PaaS provider’s interests. Larger storage requirements translated directly to higher infrastructure costs. Docker’s design, while "braindead" from a systems perspective, was brilliantly aligned with dotCloud’s business model. The more customers struggled with bloat, the more the provider profited.

This trade-off reshaped software deployment. By abstracting away the complexities of DAG-based dependency resolution, Docker democratized containerization. Developers could now version-control their entire application stack. But the technical debt mounted. Layer duplication wasted storage and bandwidth, while incremental builds became slow and opaque. The container ecosystem inherited this legacy, forcing tools like BuildKit and Kaniko to later work around the list-based model to achieve DAG-like efficiency.

Today, Docker’s layered design remains a cornerstone of containerization. Its success underscores a critical lesson in software engineering: user experience often trumps technical purity. Yet, as the industry pushes toward more granular, dependency-aware systems, the question lingers: was Docker’s list a pragmatic shortcut or a foundational flaw? The answer lies in the balance between accessibility and efficiency—a tension that continues to define the evolution of distributed systems.

"Docker was designed to allow even completely braindead idiots to deploy software. This is not a joke." — Source: schizo.cooking/effort/docker-layers.html