Announcing Zstandard in Rust – A Pure‑Rust Path to Safer, Portable Compression

The Trifecta Tech Foundation has just published the first public release of libzstd‑rs‑sys, a Rust‑native implementation of the Zstandard (zstd) compression format. After two earlier projects that provided Rust bindings for zlib and bzip2, this marks the foundation’s third foray into high‑performance data compression, and it arrives at a moment when the industry is increasingly looking to replace gzip with zstd for web traffic.

---

Why a New Rust Implementation?

The existing zstd crate works by compiling the original C source code of the reference implementation. While functional, that approach introduces two practical constraints:

1. Tool‑chain dependency – A C compiler must be present on the target platform. Setting up a reliable C tool‑chain on Windows, on embedded targets, or for WebAssembly can be cumbersome and sometimes impossible.
2. Mixed‑language ergonomics – Rust projects that depend on a crate which pulls in C code inherit the build‑time quirks of that code (linker flags, ABI mismatches, etc.). A pure‑Rust dependency eliminates those friction points and aligns with the Rust community’s preference for safe, Rust‑only libraries.

By re‑implementing the core of zstd in Rust, the foundation removes the need for a C tool‑chain, simplifies cross‑compilation, and offers a more idiomatic experience for Rust developers.

---

Core Technical Contributions

Portability and Drop‑in Compatibility

• Pure‑Rust core – All compression and decompression logic lives in safe Rust, with a thin libzstd‑rs‑sys shim that can be compiled into a static C library. Existing C applications can link against this library without code changes, making the project a viable alternative to the Meta‑maintained reference implementation.
• WebAssembly friendliness – Because no external C compiler is required, the library compiles cleanly to wasm32-unknown-unknown, opening the door for in‑browser compression utilities and edge‑function workloads.

Memory‑Safety Guarantees

The foundation emphasizes safety: the default build performs bounds checks on every array access that originates from untrusted input. This design choice yields a modest performance penalty—approximately 3 % slower than the C reference during decompression—but it provides strong protection against out‑of‑bounds reads that have historically been a source of security vulnerabilities in compression libraries.

For workloads where raw speed is paramount, the crate offers an unsafe-performance-experimental feature flag. Enabling it disables four specific bounds checks, bringing performance on par with the C implementation while still retaining the rest of the safety net. Users who enable this flag do so at their own risk, acknowledging the trade‑off between absolute safety and maximum throughput.

Rigorous Validation

• Reference test suite – The Rust implementation is exercised against the official zstd test suite, ensuring functional parity.
• Fuzz testing – Continuous fuzzing uncovers edge‑case inputs that could trigger undefined behavior.
• Miri verification – Running the code under the Miri interpreter validates memory safety at the interpreter level, catching subtle UB that normal testing might miss.

These layers of verification give confidence that the library behaves correctly across the full range of valid and malformed inputs.

---

Ecosystem Impact

The C reference implementation is maintained by Meta and requires a contributor agreement, which can be a barrier for independent developers. By providing an independent, high‑performance, and permissively‑licensed Rust implementation, the foundation diversifies the compression ecosystem and reduces reliance on a single corporate steward.

Additionally, the project has yielded side benefits:

• Miri improvements – Several limitations in the Miri interpreter were identified and resolved during development, benefiting the broader Rust community.
• Clippy contributions – The team submitted lint improvements that help Rust developers write cleaner, more idiomatic code.

---

Current State and Roadmap

The pre‑release (v0.0.1‑prerelease.2) is available on GitHub: libzstd‑rs‑sys releases. At this point, decompression and the dictionary builder are fully functional and pass the reference tests. Compression support remains a work in progress; the remaining tasks are outlined in Milestone 4 – Encoder implementation.

Future work includes:

• Completing the compression pipeline and polishing the API surface.
• Benchmarking against the reference implementation across a broader set of real‑world workloads.
• Expanding documentation and providing examples for both Rust and C consumers.

Funding is being sought to accelerate the compression work, and the foundation invites interested parties to contact them via trifectatech.org/support.

---

Integration with Existing Projects

The foundation maintains a fork of the official zstd repository that links against libzstd‑rs‑sys instead of the C library. The goal is to upstream this change, allowing downstream projects to opt into the Rust implementation with a simple Cargo feature flag. Most stable APIs map directly, but some experimental features require structural adjustments (e.g., converting an enum to a struct for FFI safety).

---

Community and Sponsor Acknowledgments

Development of the decompression side has been funded by Chainguard, Astral, and the NLnet Foundation. The dictionary builder received support from the Sovereign Tech Agency. Their contributions underscore the shared interest in having a secure, open‑source compression stack.

---

Implications for the Wider Software Landscape

If the Rust implementation reaches feature parity and performance parity with the C reference, it could become the default choice for new Rust projects that need high‑speed compression, especially those targeting platforms where a C tool‑chain is unavailable (e.g., embedded devices, WebAssembly, or CI environments with minimal tooling). Moreover, the existence of a fully Rust‑based zstd library may encourage other critical infrastructure projects—such as time synchronization or privilege‑boundary tools, also maintained by the foundation—to adopt a uniform, memory‑safe code base, thereby raising the overall security posture of the open‑source ecosystem.

---

Where to Find More Information

• GitHub repository – https://github.com/trifectatechfoundation/libzstd-rs-sys
• Detailed write‑up of contributions – https://trifectatech.org/blog/libzstd-rs-sys-details
• Support and contact – https://trifectatech.org/support (or email Erik Jonkers at [email protected])

---

The release of libzstd‑rs‑sys demonstrates how a focused, community‑driven effort can produce a high‑quality, safety‑first alternative to a long‑standing C library, and it sets the stage for broader adoption of Rust in performance‑critical system components.