Explore our latest articles on LLM inference, optimization techniques, and system architecture.
When Prefix Cache Meets KDA: How Mooncake Enabled Day-0 Support for Kimi K3
On July 27, Moonshot AI officially open-sourced Kimi K3. Having been part of the journey across multiple Kimi generations, Mooncake, together with SGLang, vLLM, and TokenSpeed, delivered Day-0 support to enable efficient distributed inference for Kimi K3.
AgentENV: When LLMs Learn to Get the Job Done, We’re Open-Sourcing the Infrastructure Behind Them
AgentENV is an open-source execution environment for large-scale Agentic RL that combines strong microVM isolation with fast snapshots, copy-on-write forks, and efficient resource reuse.
Scaling KV Cache Beyond Memory with Mooncake SSD Offloading
Mooncake extends KV cache beyond expensive memory by pooling local NVMe SSDs into a distributed, persistent cache tier that preserves long-context reuse and reduces TTFT for agentic workloads.
How Much KV Cache Budget Do We Need for LLM Serving?
Estimate the KV Cache capacity for LLM inference workloads by analyzing hit rate and prefill speedup under different cache budgets, with the help of KV Cache Hit Rate Simulator.
KT-FT v0.6.1: Closing the Loop from MoE Fine-Tuning to Local Serving
KT-FT v0.6.1 connects MoE SFT and local SGLang serving into an end-to-end loop; split LoRA serving bridges KT expert and SGLang non-expert adapters for Qwen3.5 MoE.
OpenClaw + Mooncake: A Stability Upgrade for Real-World Multi-Session Inference
By integrating Mooncake into OpenClaw's real inference path, we not only improved fast-path latency, but also sharply reduced TTFT tail latency in multi-session, long-context workloads, turning a system that was usually fast but occasionally slow into one that feels consistently smooth.
Mooncake Joins the PyTorch Ecosystem
Mooncake is now part of the PyTorch Ecosystem, complementing PyTorch-native LLM serving with high-performance disaggregated data transfer and storage.
KTransformers + LLaMA-Factory + SGLang: Low-Cost Local Fine-Tuning and Inference
A low-cost, low-memory end-to-end fine-tuning and inference workflow for large MoE models with KTransformers, LLaMA-Factory, and SGLang.