KV Cache Size Calculator Model family Model Tokens per sequence Sequences KV precision Indexer precision Include draft KV cache Include linear-attention state Want to analyze KV cache hit rates of your traces? Try our KV Cache Hit Rate Simulator. Want to scale KV cache storage across your inference cluster? Try Mooncake. Found an issue or have feedback? Open an issue and let us know. Total cache size -- = -- GB -- -- = -- Source: --