vLLM

#vLLM

6 articles tagged
Intel Releases llm-scaler-vllm 0.14.0-b8, Talks Up 1.49x Performance With BMG-G31
AI

Intel Releases llm-scaler-vllm 0.14.0-b8, Talks Up 1.49x Performance With BMG-G31

3/2/2026
From 24 to 216: A Systematic Performance Engineering Journey for LLM-based TTS
LLMs

From 24 to 216: A Systematic Performance Engineering Journey for LLM-based TTS

1/25/2026
vLLM Achieves 2.2k Tokens/Second per H200 GPU with Wide-EP Architecture
LLMs

vLLM Achieves 2.2k Tokens/Second per H200 GPU with Wide-EP Architecture

1/14/2026
A Memcached for Attention: Inside the Cross-GPU KV Cache Marketplace for LLM Inference
AI

A Memcached for Attention: Inside the Cross-GPU KV Cache Marketplace for LLM Inference

11/12/2025
AI

Cascade's Predicted Outputs Turbocharges VLLM: Skip Regeneration, Not Tokens

10/10/2025
Building the AI Cathedral: How Google Cloud Scales Inference for Billions of Agents and Users
AI

Building the AI Cathedral: How Google Cloud Scales Inference for Billions of Agents and Users

7/26/2025

Popular Tags

#AI (2026)#Security (667)#Open Source (525)#privacy (496)#Apple (493)#API (471)#Hardware (455)#Reddit (413)#regulation (368)#Trends (350)#Infrastructure (331)#Cybersecurity (323)#Business (291)#Microsoft (289)#Linux (284)#Cloud (280)#Gaming (271)#Remote Code Execution (267)#OpenAI (266)#Windows (263)#Performance (246)#Vulnerabilities (243)#Rust (220)#Developer Tools (220)#Supply Chain (201)#Automation (199)#Android (190)#Battery (189)#Samsung (188)#China (183)#LLMs (180)#Machine Learning (180)#Developer (177)#Authentication (175)#Anthropic (174)#Azure (173)#Nvidia (172)#Productivity (167)#Funding (162)#DevOps (155)#AMD (150)#Intel (145)#mobile (144)#GPU (140)#iOS (137)#Camera (136)#Linux kernel (130)#rate-limiting (129)#CISA (128)#LLM (124)