Cutting the Gordian Hairball: Intertwingler as a Frontal Assault on Link Rot

The author's reflection on completing a significant technical component of Intertwingler reveals a profound meditation on the state of the Web and its inherent limitations. What emerges is not merely a technical update but a philosophical stance on how information infrastructure ought to function in an increasingly complex digital landscape.

At its core, Intertwingler represents a deliberate attempt to solve what the author identifies as the Web's most fundamental flaw: link rot and content drift. These related phenomena, where referenced content disappears or changes unpredictably, undermine the very promise of hypertext by making cross-referencing unreliable. The author argues that this unreliability is why we "can't have nice things" on the Web—why we can't create truly dense hypermedia systems that would allow us to "understand more while having to read less" and "convey more while having to document less."

The technical solution being implemented—a custom caching mechanism—addresses this problem by creating continuity guarantees for addresses and content. This infrastructure enables the creation of atomized, reusable information chunks that can be transcluded across multiple documents and contexts. The author makes a provocative claim that treating Web servers merely as repositories for uploaded files from computers is fundamentally limiting, suggesting that the canonical interface for information storage should shift from file systems to live, networked resources.

The conceptual framework of "dense hypermedia" versus "sparse hypermedia" provides an interesting lens through which to view the evolution of Web technology. The modern Web, despite its success, remains in the author's view a "sparse" system where information is locked in monolithic documents with poor internal addressability. Dense hypermedia, by contrast, would leverage the Web's native capabilities for linking and embedding to create systems where information exists as discrete, addressable units that can be assembled and reassembled as needed.

Perhaps most compelling is the author's analysis of documentation as a process. Rather than documentation being "extra work" that chases after business processes, the ideal system would have documentation drive those processes. This inversion of the conventional relationship between documentation and action represents a significant shift in how we might think about knowledge management in organizational contexts.

The author's critique of current communication tools like Slack, Discord, and Zulip is particularly insightful. While these tools have improved upon older systems like IRC by assigning URLs to messages, they still maintain an artificial distinction between ephemeral messages and durable artifacts. The proposed alternative, embodied in the Sense Atlas product, would create an environment where elementary information objects are connected by semantic relationships rather than temporal sequence.

The technical challenges involved in implementing such a system are substantial, as evidenced by the year-long effort to complete the caching mechanism. The author acknowledges that the final product will likely break initially and require iterative fixes—a realistic assessment of complex software development.

Looking ahead, the planned separation of Intertwingler into two components—Intertwingler proper and the performance-focused Intermingler—suggests a strategy for both technical specialization and broader adoption. By isolating the anti-link-rot properties and creating mechanisms for "intelligent heterogeneity," the author aims to build infrastructure that can seamlessly integrate services from various vendors and programming languages.

The article implicitly raises several questions that warrant further consideration:

1. The File System Question: While the author makes a compelling case for moving beyond file systems as the primary interface for information storage, many organizations have deeply entrenched file-based workflows. The transition path would need careful consideration.

2. Adoption Challenges: Creating a new paradigm for information management requires not just technical solutions but changes in how people think about and work with information. The author acknowledges this indirectly when discussing the difficulty of finding language that communicates value to a general audience.

3. Scale and Performance: The proposed system of highly interconnected, addressable information chunks raises questions about performance at scale, though the caching mechanism being implemented appears designed to address this.

4. Security and Control: Centralizing information on Web servers with reliable addressing introduces new considerations for security, access control, and content governance that would need to be addressed.

The author's work represents a thoughtful response to what might be called the " satisficing trap"—the tendency to accept solutions that are "good enough" rather than pursuing better alternatives that require more effort. In a Web environment increasingly dominated by disposable content and ephemeral communication, Intertwingler offers a vision of what might be possible if we were to take the Web's original hypertext promises more seriously.

The completion of the caching mechanism marks not just a technical milestone but a philosophical statement about the kind of Web the author believes we should be building—one where information persists, connections remain reliable, and the act of creating and sharing knowledge becomes more efficient and meaningful.

For those interested in following this project, the author maintains multiple newsletters and offers consulting services related to these techniques. The separation of Intertwingler and the planned development of Intermingler suggest that we may see both specialized infrastructure and more accessible applications emerging from this work in the near future.