Nvidia's Groq 3 LPX: 256 LPUs in a Single Rack for AI Inference

Nvidia has unveiled the Groq 3 LPX, a specialized inference server rack designed to deliver massive AI processing power in a single chassis. The system packs 256 Groq 3 LPUs (Language Processing Units) and features 128GB of on-chip SRAM, positioning it as a high-performance solution for AI inference workloads.

What's New The Groq 3 LPX represents Nvidia's latest push into specialized AI hardware, focusing specifically on inference rather than training. The rack's architecture emphasizes memory bandwidth and processing density, with the 128GB of on-chip SRAM providing extremely low-latency access to model weights and intermediate computations.

The system is scheduled for release in the second half of 2026, giving developers and enterprises time to prepare for deployment. This timing aligns with Nvidia's broader roadmap for AI infrastructure, as the industry continues to scale up inference capabilities to match growing demand for AI services.

Technical Architecture The 256 LPUs configuration suggests a highly parallel processing approach, likely designed to handle multiple simultaneous inference requests or to process large models across many cores. The on-chip SRAM is particularly noteworthy - traditional GPU architectures rely more heavily on HBM (High Bandwidth Memory), but the Groq 3 LPX appears to prioritize ultra-fast on-chip memory for reduced latency.

This design choice indicates Nvidia is targeting workloads where inference speed is critical, such as real-time AI applications, autonomous systems, or high-frequency trading scenarios where every microsecond matters.

Market Context The announcement comes amid intense competition in AI hardware, with companies like Groq (the original LPU company), Cerebras, and various cloud providers developing specialized inference solutions. Nvidia's entry into this space with the Groq 3 LPX shows the company's strategy of covering both ends of the AI spectrum - from massive training clusters to specialized inference hardware.

Availability and Pricing While specific pricing hasn't been disclosed, enterprise-grade AI hardware of this caliber typically runs into the hundreds of thousands of dollars per rack. The H2 2026 availability window suggests Nvidia is still finalizing production and working through supply chain considerations for these specialized components.

The Groq 3 LPX appears designed for data centers and cloud providers that need to maximize inference throughput per rack unit, potentially offering better power efficiency and performance density than traditional GPU-based solutions for certain workloads.

For more information, visit Nvidia's GTC 2026 announcements.