Attention mechanisms (e.g. DeepSeek NSA) that look at only a learned subset of past tokens, turning the KV-bandwidth cost from linear in context length into roughly square-root.
Tensions
Quality vs. cost is still empirical territory, but the math is decisive: dense attention caps practical context at the current 100-200K plateau, while sparse attention is the only known route to million-plus token windows without exploding serving cost.
Related Concepts
KV cache | context length | memory bandwidth | inference economics