Supermicro launches shared KV cache with IBM and NVIDIA
The new solution aims to reuse computed context in shared storage outside GPUs for long-context and agentic inference.
AT A GLANCE
Key points
- The architecture combines Supermicro Petascale, IBM Storage Scale, NVIDIA Dynamo, and Spectrum-X.
- A joint test reported up to 56x faster time to first token for 130,000-token prompts.
- Shared storage is designed to keep KV state outside GPU memory and reuse it across nodes.
What is changing?
Supermicro announced a joint KV cache storage solution with IBM and NVIDIA for long-context AI inference. The system combines Supermicro Petascale all-flash servers, IBM Storage Scale, NVIDIA Dynamo, and Spectrum-X Ethernet in one architecture.
KV cache stores intermediate context states that a model has already computed, reducing repeated processing on later requests. In a joint test using a 120-billion-parameter model, the companies reported up to 56x faster time to first token for 130,000-token prompts than a setup without KV cache. This is a provider-run result and has not been independently verified across models.
The solution moves cache capacity beyond local GPU memory and shares it across nodes. Supermicro also says rack integration and on-site deployment services can reduce deployment time from weeks to days.
Explore the original sources
Source date: 5 November 2026.
Published by AI Radar: 5 Nov 2026, 18:09 (Istanbul, UTC+3). Updated: 9 Nov 2026, 01:03.
FOLLOW THE TOPIC
Topics and tags
Share your discovery.
Send this story to a friend.
Reader comments
What do you think about this development? Share your experience or question. Comments are published after editorial approval.
Your name/nickname and comment become public if approved. Read: Privacy notice.
Loading comments…