Netfence: eBPF-Based Network Filtering with Envoy xDS Patterns

A new project called Netfence is attempting to bridge the gap between modern service mesh control planes and low-level kernel networking. It takes inspiration from Envoy's xDS protocol but applies it to eBPF filters, creating a system for dynamic, host-level network security that operates with minimal performance penalty.

The core idea is to run a daemon on each VM or container host that injects eBPF filter programs into cgroups and network interfaces. This daemon also runs a built-in DNS server for each attached interface or cgroup. When a container or VM makes a DNS query, Netfence intercepts it, resolves the domain, and automatically populates an IP allowlist in the eBPF filter. Traffic to any IP not on that list is dropped before it even leaves the host. The entire system is controlled by a central gRPC-based control plane that you implement, which pushes rules like ALLOW *.pypi.org or ALLOW 10.0.0.0/16.

This architecture mirrors the separation of concerns found in Envoy's data plane and control plane. The Netfence daemon on each host is the data plane, managing the eBPF programs and local DNS resolution. Your control plane is the brain, deciding what traffic should be allowed based on your application's needs, tenant information, or security policies. The daemon connects to the control plane via a bidirectional gRPC stream, subscribing to updates and sending heartbeats.

How It Works in Practice

The workflow is designed for orchestration systems. When a new container or VM starts, your orchestration layer (like Kubernetes or a custom scheduler) calls the daemon's local gRPC API to attach a filter. This could be to a specific network interface (using TC eBPF filters) or to a cgroup (for container-level filtering). You can attach metadata like vm_id=abc or tenant=acme to this attachment.

Once attached, the daemon sends a Subscribed message to your control plane and waits for a SubscribedAck with the initial configuration. This handshake ensures the attachment is fully configured before any traffic flows. The control plane uses the metadata to look up the appropriate rules for that specific VM or tenant and sends back the initial allow/deny lists and DNS rules.

The DNS server is a key component. Each attachment gets its own unique DNS address and port. Containers and VMs are configured to use this local DNS server. When a query comes in, Netfence checks its domain rules. These rules support subdomains with specificity-based matching, so a rule for *.api.example.com would be overridden by a more specific rule for api.example.com. If the domain is allowed, Netfence resolves it, adds the resulting IPs to the eBPF filter (with an optional TTL), and the traffic flows. If a domain is denied or unknown, the DNS query fails or the resulting IPs are blocked.

This approach has several implications:

1. Performance: eBPF filters operate in the kernel, so the performance overhead is negligible compared to userspace proxies. The filtering happens before packets traverse the network stack, reducing unnecessary processing.
2. Security: It provides a host-level firewall that is aware of application-level domains, not just IP addresses. This is harder to achieve with traditional iptables rules, which are typically IP-based.
3. Dynamic Control: Rules can be updated in real-time from the control plane. If a service's allowed domains change, the control plane pushes an update, and the eBPF filter is modified accordingly.

Trade-offs and Considerations

While the concept is powerful, there are trade-offs to consider. The system relies on a custom control plane implementation, which adds development overhead. You need to build the logic for deciding which rules apply to which attachments, likely integrating with your existing service discovery or policy engine.

The DNS interception model also introduces a specific behavior. It assumes that all outbound traffic is initiated via DNS resolution. This is true for most HTTP/gRPC traffic but may not cover all protocols or direct IP connections. The project's denylist and block-all modes offer alternatives, but the primary strength is in the allowlist model driven by DNS.

Another consideration is the operational complexity of managing eBPF programs across a fleet of hosts. While Netfence handles the loading and management, ensuring compatibility with different kernel versions and handling program updates requires careful testing.

Broader Pattern: The xDS-ification of Everything

Netfence fits into a larger trend of applying the xDS pattern beyond service meshes. xDS (discovery service) is a generic protocol for dynamically configuring data plane proxies. Its success with Envoy has inspired similar approaches for other infrastructure components, from load balancers to network filters.

The pattern's appeal is clear: decouple the control logic (which can be complex and centralized) from the data plane (which should be simple, fast, and distributed). By using gRPC streams, Netfence provides a reliable, real-time channel for policy distribution, similar to how Envoy receives updates from its control plane.

For teams already using Envoy or similar service meshes, this pattern is familiar. However, Netfence operates at a lower level, securing the host itself rather than just the application's network namespace. This could be particularly useful for multi-tenant environments or for securing legacy applications that don't have built-in service mesh integration.

Getting Started

The project provides a CLI and daemon (netfenced) for managing attachments. You can start the daemon with a configuration file and then use commands like netfenced attach to link a network interface or cgroup to the filter. The control plane side is left as an exercise for the user, but the gRPC protocol is defined, giving you a clear interface to implement.

For those interested in the technical details, the GitHub repository contains the protocol definitions and implementation. It's a niche tool, but for environments where dynamic, kernel-level network filtering is a requirement, it offers a compelling alternative to traditional firewalls or userspace proxies.

The project is still in its early stages, and its adoption will depend on how well it integrates with existing orchestration platforms and how robust the control plane implementation proves to be. But as a concept, it demonstrates how eBPF and modern control plane patterns are converging to solve long-standing networking challenges.