SCION Association Builds 25 Gbit/s Workstation to Advance Next-Generation Internet Routing

The SCION Association has completed construction of a specialized workstation designed to push the boundaries of its open-source border router performance. Built at a cost of approximately CHF 3,741 (≈$4,700 USD), this custom system aims to enable development of a new AF_XDP-based underlay capable of handling 25 Gbit/s traffic streams – a fivefold improvement over current capabilities.

SCION (Scalability, Control, and Isolation On Next-Generation Networks) presents a modern alternative to the decades-old Border Gateway Protocol (BGP) that underpins today's internet routing. Unlike BGP, SCION provides mathematically verifiable path authenticity, explicit path control, deterministic failover, and consistent latency – attributes critical for financial infrastructure. Switzerland's banking system already relies on SCION through Anapaya Systems' commercial implementation.

Schematic of packet flow through Linux networking stack by Jan Engelhardt

Despite significant improvements in recent years, the open-source SCION border router (hosted on GitHub) tops out at approximately 5-6 Gbit/s per stream due to Linux kernel networking overhead. This limitation stems from the traditional packet processing path: packets pass through multiple kernel subsystems (filtering, routing, queueing) before reaching userspace, creating bottlenecks at high throughput.

To bypass this, the team turned to AF_XDP – a Linux kernel mechanism allowing direct packet access via shared memory rings between NICs and userspace. "Typical VM offerings don't expose the access needed for XDP/AF_XDP," noted the project lead. "You need bare metal with specific NIC support." This necessitated custom hardware capable of zero-copy operation.

Juniper MX series routers exemplify the scale SCION aims to approach

Hardware selection proved critical. After evaluating options, the team chose:

• NICs: Three Mellanox NVIDIA BlueField-2 DPUs (dual 25G ports) operating in NIC mode (€289.92 each)
• Motherboard: ASUS Pro WS W790E-SAGE SE for its seven PCIe Gen5 slots
• CPU: Intel Xeon W5-2455X (12-core, 64 PCIe Gen5 lanes) for IO bandwidth
• Cooling: Noctua NH-U14S cooler and optimized fan array for silent office operation

The build process revealed unexpected challenges: firmware updates requiring physical card reflashing, BMC password bugs locking out management interfaces, and PCIe slot compatibility issues causing days of troubleshooting. Thermal management was particularly crucial, as the BlueField-2 cards – designed for rack cooling – required custom airflow solutions in a tower case.

Mellanox BlueField-2 NICs before installation

Initial benchmarks validated the effort: AF_XDP tests achieved 24.6 Gbit/s line rate (24,607 Mbps) with zero packet loss across 1.5 TB of transferred data. "This proves we can push packets at wire speed without kernel overhead," the builder confirmed.

The workstation now serves as a development platform for SCION's AF_XDP underlay, funded by the NLnet Foundation. Future work will focus on integrating this capability into the main SCION OSS router. Complete hardware specifications:

Component	Quantity	Price
ASUS Pro WS W790E-SAGE SE	1	CHF 962.90
Intel Xeon W5-2455X	1	CHF 1,106.00
Mellanox BlueField-2 NICs	3	CHF 318.09
Corsair DDR5 RDIMM 64GB Kit	1	CHF 536.00
Cooling/Case/Peripherals	-	CHF 1,818.35
Total		CHF 3,741.34

Technical details and benchmark code are available in the project's GitHub repository.