CI In a Box: Rethinking Continuous Integration with SSH Wrappers

The landscape of continuous integration has long been dominated by complex YAML configurations and proprietary runner systems. A new approach called "box" offers a refreshing alternative: a thin SSH wrapper that provides a unified interface for running commands across heterogeneous computing environments.

At its core, box is elegantly simple. It's essentially a wrapper around SSH that allows you to create and manage remote machines with specific operating systems and configurations. The interface is intuitive: you can create a Windows runner, a macOS runner, or a Linux runner with simple commands, then execute scripts on these machines as if they were local.

The CI workflow becomes remarkably straightforward. You specify your repository and commit, create runners for each target platform, and then execute your build and test commands across all environments. The beauty lies in its simplicity - the controlling CI machine runs a user-supplied script, and the script's exit code determines the overall CI result.

What makes this approach particularly interesting is how it handles cross-platform testing. Rather than trying to make everything work on a single platform, box embraces heterogeneity. Each runner can have the exact OS, CPU architecture, and environment needed for the software being tested. This is crucial for projects that need to support multiple platforms, as cross-platform compatibility issues often only surface when testing on the actual target systems.

However, the heterogeneous fleet problem is where things get complicated. Not all operating systems are created equal when it comes to CI infrastructure. Windows isn't UNIX, macOS has licensing and hardware constraints that make per-minute billed VMs challenging (though not impossible, as GitHub Actions demonstrates), and all platforms are moving targets that require ongoing maintenance and upgrades.

The SSH protocol itself presents interesting challenges. As Colin Watson noted, the SSH wire protocol only takes a single string as the command, expecting it to be passed to a shell on the remote end. This means that while you can use syntax like ssh $HOST cmd arg1 arg2, SSH simply intersperses all arguments with spaces. This shell injection vulnerability is both amusing and concerning when you consider that our entire cloud infrastructure relies on this behavior.

This limitation means you can't "just" use SSH for robust CI systems. You need to carefully handle quoting and ensure no processes are left behind after executing remote commands. The box approach addresses these issues by providing a more controlled interface while still leveraging SSH's ubiquity and reliability.

The broader implications for CI discourse are worth considering. The community often complains about YAML configuration files, and these complaints are valid - YAML is frequently verbose, error-prone, and difficult to debug. However, much of this complexity is avoidable. The solution isn't necessarily to eliminate YAML entirely, but to choose the appropriate level of abstraction for your needs.

You can position yourself anywhere on a spectrum: from writing simple bash scripts, to using scripts in your existing programming language, to adopting small build systems, medium-sized ones like Make or Zig build, or large comprehensive systems like Nix or Buck2. The key insight is that you don't need to write extensive configuration to get heterogeneous runners - that's where box shines.

For developers building cross-platform software, this approach offers compelling advantages. Instead of wrestling with complex container configurations or trying to emulate different operating systems, you can spin up actual machines with the target OS and run your tests in a genuine environment. This can catch platform-specific issues that might be missed by more abstracted approaches.

The box approach also aligns well with modern development practices. It's language-agnostic, meaning you can use it regardless of whether you're building in Rust, Go, Python, or any other language. It's also infrastructure-agnostic - you can use it with cloud providers, on-premises hardware, or even local development machines.

Security considerations are paramount when building CI systems, and the SSH-based approach has both advantages and challenges. SSH is well-understood, widely deployed, and has robust security features. However, the shell injection vulnerability mentioned earlier means that careful attention must be paid to how commands are constructed and executed.

Looking forward, approaches like box could influence how we think about CI infrastructure. As software becomes increasingly cross-platform and distributed, the ability to easily spin up and manage heterogeneous environments becomes more valuable. The simplicity of the box interface makes it accessible to developers who might be intimidated by more complex CI systems, while still providing the power needed for sophisticated workflows.

The future of CI might not be about building ever-more-complex configuration systems, but about creating simpler, more composable tools that let developers focus on what matters: building and testing their software. Box represents an interesting step in that direction, showing that sometimes the best solution is to wrap existing, proven technologies in a more user-friendly interface.

For teams considering their CI strategy, box offers an intriguing alternative to traditional approaches. It's particularly well-suited for projects that need to support multiple platforms, value simplicity and transparency, and want to avoid vendor lock-in. While it may not replace all CI use cases, it certainly provides a compelling option for many common scenarios.

The evolution of CI tools continues to be an exciting area of development in the software industry. As we move toward more distributed, cross-platform, and automated development workflows, tools that can bridge the gap between different environments while maintaining simplicity will become increasingly valuable. Box, with its SSH-based approach to heterogeneous runners, represents an interesting contribution to this ongoing evolution.