For a 7B parameter FP16 model with 4096 token context, approximately how large is the KV cache?

Question

MLQuiz · Accepted Answer

The text states: 'For a 4096‑token context, KV cache consumes ~800 MB'. The model weights are 14 GB, attention mask 16 MB, and current activation ~6 KB.