LLM inference

#LLM inference

11 articles tagged
Real‑time LLM Inference on Standard GPUs Hits 3,000 tokens/s per Request
LLMs

Real‑time LLM Inference on Standard GPUs Hits 3,000 tokens/s per Request

5/29/2026
DeepSeek V4 Completes Full Adaptation to Huawei Ascend, Marking a Milestone for China AI Stack
Chips

DeepSeek V4 Completes Full Adaptation to Huawei Ascend, Marking a Milestone for China AI Stack

5/25/2026
Apple Silicon vs. OpenRouter: A Cost‑and‑Speed Reality Check
LLMs

Apple Silicon vs. OpenRouter: A Cost‑and‑Speed Reality Check

5/17/2026
Orthrus: Dual-View Diffusion Decoding Claims to Accelerate LLM Inference While Maintaining Exact Generation Fidelity
LLMs

Orthrus: Dual-View Diffusion Decoding Claims to Accelerate LLM Inference While Maintaining Exact Generation Fidelity

5/16/2026
Zero-Copy GPU Inference from WebAssembly on Apple Silicon
Hardware

Zero-Copy GPU Inference from WebAssembly on Apple Silicon

4/19/2026
ZSE: A Memory-Efficient LLM Inference Engine with Smart Resource Orchestration
LLMs

ZSE: A Memory-Efficient LLM Inference Engine with Smart Resource Orchestration

2/26/2026
nTransformer Enables Llama 70B Inference on Single Consumer GPU with Novel Streaming Architecture
Hardware

nTransformer Enables Llama 70B Inference on Single Consumer GPU with Novel Streaming Architecture

2/22/2026
Beyond the Data Center: Taalas and the Path to Ubiquitous AI
Chips

Beyond the Data Center: Taalas and the Path to Ubiquitous AI

2/20/2026
Continuous Batching: Optimizing LLM Inference Throughput from First Principles
LLMs

Continuous Batching: Optimizing LLM Inference Throughput from First Principles

2/16/2026
Memory Walls and Interconnect Bottlenecks: New Research Charts Path for Efficient LLM Inference Hardware
Hardware

Memory Walls and Interconnect Bottlenecks: New Research Charts Path for Efficient LLM Inference Hardware

1/25/2026
The End of One-Size-Fits-All LLM APIs: Why Workload-Specific Inference Is Taking Over
LLMs

The End of One-Size-Fits-All LLM APIs: Why Workload-Specific Inference Is Taking Over

1/22/2026

Popular Tags

#AI (3691)#Security (1346)#Open Source (980)#privacy (869)#Hardware (866)#Remote Code Execution (769)#Apple (769)#API (748)#Microsoft (695)#Windows (694)#Infrastructure (666)#regulation (662)#Reddit (627)#Cloud (589)#Trends (580)#Cybersecurity (560)#Linux (548)#Gaming (508)#Business (490)#Performance (477)#Vulnerabilities (452)#Supply Chain (431)#OpenAI (394)#Android (374)#Machine Learning (373)#Developer Tools (371)#Automation (368)#Rust (367)#Samsung (366)#Nvidia (341)#Battery (340)#LLMs (336)#Azure (336)#Authentication (321)#DevOps (311)#China (300)#Patch (300)#Anthropic (297)#Developer (295)#AMD (293)#Productivity (291)#GPU (283)#LLM (280)#Camera (271)#iOS (268)#Intel (268)#Linux kernel (265)#Cloudflare (259)#Patch Management (250)#mobile (250)