What is the size of the causal attention mask for a 4096‑token sequence (stored as booleans)?

Question

Accepted Answer

The text explicitly gives: 'Attention Mask: 16 MB (cached triangular) and Example: 4096² ≈ 16 MB'.