HMML – A Structured Alternative to Raster Images, but Does It Solve Real Problems?

What HMML Claims

• Composable media: Instead of a flat PNG/JPEG, an .hmml file stores each visual element (SVG, raster, 3D mesh, keyframe) as a separate node that can be addressed, diffed and remixed.
• Fine‑grained editing: Because the file retains the original primitives, a downstream developer can change a single node without re‑rendering the whole scene.
• Model‑friendly output: Large language models already generate HTML, CSS and SVG. HMML claims to be a thin wrapper that bundles those artifacts plus raw binary resources into a ~2 KB binary contract.
• Zero‑runtime dependencies: A tiny JavaScript loader (@eddocu/hmml) can unpack the blob in the browser, reconstruct the DOM and mount it with a single <div>.
• Version‑friendly format: Unknown chunks are ignored, allowing forward compatibility.

What Is Actually New?

1. A binary container with a simple header
   • The spec starts with the magic bytes 89 48 4D 4D 4C 0D 0A 1A 0A followed by version numbers and a codec flag. This is reminiscent of PNG’s chunk‑based layout, but the payload is a gzip‑compressed HTML/CSS/JS string plus a list of raw resources.
2. Resource handling without base64
   • Traditional data‑URI approaches embed binary data as base64 inside HTML, inflating size by ~33 %. HMML stores resources as raw byte streams in separate chunks, referenced from the markup via hmml:<id>. This reduces the on‑disk size of the examples (24‑49 % smaller than equivalent PNG/HTML bundles).
3. Two‑step API
   • pack(html) → Uint8Array and unpack(bytes) → doc with toHTML() or createObjectUrls(). The API mirrors existing bundlers (e.g., Webpack’s require() of assets) but is deliberately minimal.
4. Version‑tolerant parsing
   • Unknown chunks are skipped, similar to how browsers ignore unrecognised PNG ancillary chunks. This makes the format extensible without breaking older readers.

Where the Claims Meet Reality

1. Composability vs. Existing Web Standards

• The web already supports composable media through HTML + CSS + <picture> + <canvas> + WebGL. An SVG file can embed raster images, and a Web Component can encapsulate a 3D scene. HMML essentially packages those same standards into a binary envelope. The novelty is convenience, not a new rendering model.

2. Model Output Practicality

• Current text‑to‑image models (Stable Diffusion, DALL·E) output raster pixels because the training objective is pixel‑wise likelihood. Shifting to structured outputs would require a fundamentally different dataset and loss function (e.g., vector‑aware diffusion). While research on diffusion for SVG exists, production‑grade models that emit full HTML/CSS/JS bundles are still experimental. HMML may be ready, but the upstream models are not.

3. File Size vs. Network Cost

• The examples show a 2‑4 KB .hmml file, which is tiny compared to a 100 KB PNG. However, a realistic scene with high‑resolution textures, complex 3D meshes or video‑grade animation will quickly exceed that. The gzip‑compressed markup helps, but binary assets dominate size. In such cases, the benefit of a single file is offset by the need to stream large blobs anyway.

4. Tooling and Ecosystem

• The only reference implementation is a small NPM package (@eddocu/hmml). There is no integration with major browsers, build tools, or CMS platforms. Without a loader baked into browsers, developers must add a script tag and polyfill, which re‑introduces the “multiple files” problem the format tries to avoid.

5. Editability After Generation

• HMML preserves the original markup, so a developer could theoretically edit a node. In practice, the markup is often generated automatically and may lack meaningful identifiers or comments. Diffing two .hmml versions will show byte‑level changes, but the semantic meaning (e.g., “this node is the product image”) is still opaque without additional metadata.

6. Security Considerations

• Embedding arbitrary JavaScript inside a binary container raises the same CSP and XSS concerns as any HTML bundle. Because the container can hold raw JS, a malicious model could inject code that runs on unpack. The spec does not prescribe a sandboxing model, so developers must treat .hmml files like any untrusted HTML payload.

Limitations and Open Questions

Aspect	Limitation
Model readiness	No mainstream diffusion or LLM currently outputs full HTML/CSS/JS + raw assets. HMML is a downstream consumer waiting for upstream advances.
Performance	Unpacking requires decompressing the entire blob before any rendering can start; large scenes may introduce noticeable latency.
Interoperability	No native support in browsers; reliance on a JavaScript loader limits use in environments without a JS runtime (e.g., native mobile apps).
Versioning	While unknown chunks are ignored, there is no formal schema evolution strategy; adding new codecs could break older loaders.
Toolchain integration	No plugins for Webpack, Vite, or Rollup yet, meaning developers must manually pack/unpack during build.
Security	Same attack surface as HTML; no built‑in integrity checks or sandboxing.

Bottom Line

HMML is an interesting experiment in treating generated media as a structured document rather than a monolithic raster. Its binary wrapper is lightweight, its chunk model is sensible, and the API is deliberately simple. However, the format does not solve the core bottleneck: models still output pixels. Until generative architectures can reliably emit vector, layout and animation primitives, HMML will sit on the periphery, useful perhaps for niche pipelines that already produce HTML‑based assets.

For developers curious enough to try it, the reference repo is on GitHub: eddocu/hmml. The spec can be inspected in the spec.md file of that repo. If the community builds better model‑to‑HTML pipelines, HMML could become a convenient transport format; otherwise, it may remain a clever proof‑of‑concept that highlights the growing desire for editable AI‑generated media.