KV Cache Size Calculator Model family Model Tokens per sequence Sequences KV precision Indexer precision Include draft KV cache Include linear-attention state Recurrent state precision Include SConv state State Checkpoint Policy: Prompt-End State Fixed Interval Checkpoints State checkpoint interval Want to analyze KV cache hit rates of your traces? Try our KV Cache Hit Rate Simulator. Want to scale KV cache storage across your inference cluster? Try Mooncake. Found an issue or have feedback? Open an issue and let us know. Total cache size -- = -- GB -- -- = -- Source: --