KV bytes/token = 2 × layers × kv_heads × head_dim × bytes_per_element. Assumes 90% usable memory and ~2 GiB/GPU overhead.
Configs from each model's public config.json. Estimates only. Made by Inference Engineering Academy, which has a free Foundations course on LLM inference.