Litestream Adds Writable VFS for SQLite Cold Starts

Litestream, the open-source SQLite replication tool developed by Fly.io, has added writable virtual filesystem (VFS) capabilities to solve cold-start bottlenecks in distributed systems. This enhancement directly supports Fly.io's recently launched Sprites – globally distributed applications that launch in under a second with persistent storage.

Sprites rely on SQLite databases managed by Litestream for their storage metadata. During cold starts, Sprites previously faced a critical limitation: Litestream's VFS operated in read-only mode, forcing applications to wait for full database downloads before processing writes. For latency-sensitive operations like handling incoming web requests during instance recovery, this delay proved problematic.

Solving the Write Barrier

The updated Litestream VFS introduces two key features:

1. Buffered Writes: When enabled via LITESTREAM_WRITE_ENABLED=true, writes are immediately accepted into a local buffer. Every second (or during clean shutdowns), this buffer syncs to object storage. Crucially, writes aren't durable until synced – an acceptable trade-off for Sprite's eventual-durability model.

The write path showing buffered operations syncing to object storage

1. Background Hydration: Enabled via LITESTREAM_HYDRATION_PATH, this feature fetches the full database in the background while immediately serving reads from object storage. Once hydration completes, the VFS transparently switches to the local copy for faster access.

Hydration timeline showing parallel read/write operations during download

Architectural Constraints

These features come with intentional limitations:

• Single-writer requirement (no distributed write coordination)
• Write durability depends on sync intervals
• Hydration files are treated as temporary and discarded on shutdown

As Litestream creator Ben Johnson notes: "These solutions are narrowly scoped for environments like Sprites. For conventional applications, running Litestream as a sidecar remains the simpler approach."

The VFS enhancements demonstrate Litestream's evolution beyond basic backup/restore into active infrastructure components. For developers building similar cold-start-sensitive systems, the source code offers implementation insights. Fly.io uses this functionality in production for Sprites' metadata management, where SQLite databases operate directly against S3 storage with local NVMe caching.

Documentation for the new VFS modes is available in the Litestream GitHub repository.