ATOM Meets Mooncake: A Shared KV Pool for Agentic Serving on AMD Instinct GPUs
Prefill/decode disaggregation splits host memory in two. Mooncake Store pools it back into one shared KV pool for ATOM on AMD Instinct GPUs: about a hundred concurrent agent sessions on one MI355X server at about 44K tokens/s per GPU with more than 95% of prompt tokens reused, and up to about twice the throughput of a split pool on four MI350X servers.
Nov 8, 2026
