For teams drawn to the architectural promise of Neon – the serverless Postgres platform separating compute and storage – but needing control over their infrastructure, a new path has emerged. Molnett has open-sourced an early-stage Neon Operator for Kubernetes, aiming to simplify the deployment and management of self-hosted Neon clusters.

Article illustration 1

Democratizing Neon's Architecture

Neon's design, inspired by systems like Amazon Aurora, leverages disaggregated components: Safekeepers (for write-ahead log persistence), Pageservers (handling storage and serving page requests), and ephemeral Compute nodes. While powerful, orchestrating these components reliably in a self-managed environment has been a significant hurdle. Molnett's operator directly addresses this complexity.

"We're excited to share our findings with the community," the Molnett team stated, highlighting their goal of fostering a community of "platform builders" interested in disaggregated database architectures. They specifically praised Neon's foundational work: "None of this would be possible without the relentless effort of the team at Neon. They have done amazing work making an OSS product that is feasible to self-host at scale." This architecture, they argue, creates a "win-win" where self-hosting complements rather than competes with Neon Cloud.

How the Operator Unlocks Self-Hosting

The Neon Operator provides Kubernetes-native management through three Custom Resource Definitions (CRDs):

  1. Cluster: Defines the core infrastructure requirements: a S3-compatible object store, a foundational Postgres instance, and locally-attached disks (NVMe recommended). Deploys Safekeepers, Pageservers, and auxiliary services.
  2. Project: Equivalent to a Neon Cloud Project, representing a tenant.
  3. Branch: Equivalent to a Neon Cloud Branch (internally a Timeline). Creating this resource triggers the provisioning of a fully functional Postgres compute instance.

An integrated API communicates with Neon's Storage Controller to orchestrate timeline (branch) placement across Pageservers and Safekeepers. Crucially, the operator handles node failures automatically:

"In case of node failures, compute nodes are automatically configured to communicate with a potentially new set of pageservers and safekeepers. This allows the operator to manage sharding and failovers without requiring human intervention."

Technical Choices and Challenges: Betting on Rust

The operator is built using Rust and the kube-rs library. This choice prioritizes performance and safety but comes with trade-offs. The team acknowledges that kube-rs is less feature-rich than alternatives like Kubebuilder (commonly used with Go), and the lack of a built-in integration testing framework (envtest equivalent) is a hurdle. Furthermore, they face a dependency challenge:

"The intention is for the operator to directly depend on the neondatabase/neon repository for API clients, protobufs and other types. This is not possible today as Neon does not publish their crates... We're intending to contribute a solution to this upstream."

Despite the challenges, Molnett is committed to Rust for now, valuing the potential long-term benefits of shared core libraries with the Neon project itself.

Addressing the Skepticism and the Path Forward

The announcement candidly addresses the vocal criticism Neon Cloud has faced regarding reliability: "My personal opinion is that it's quite tiring to even observe the negativity... But at what cost?" Molnett asserts their confidence stems from technical analysis: "We are confident this is related to Neon Cloud's unique load patterns rather than a fundamental architectural or code quality issue." Their commitment remains strong due to Neon's unique architecture: "There is no other option that is built with the same type of architecture, so we've got to make it work either way!"

Molnett is now focused on achieving a production-ready release for their own internal use. Key milestones include:

  • Management APIs for databases and users
  • Pageserver "drain and fill" operations for maintenance
  • Integrated PGBouncer deployment with Compute nodes
  • Enhanced observability for monitoring and alerting

An Invitation to Build

Molnett explicitly invites collaboration: "Are you looking to deploy Neon locally? Or to collaborate...?" Developers and organizations interested in self-hosting Neon or contributing to the operator's development are encouraged to reach out via X/Discord (@bittermandel) or the GitHub repository. This release represents a significant step towards making Neon's innovative architecture accessible beyond the managed cloud, powered by the Kubernetes ecosystem.

Source: Molnett Blog