Attention Sinks: Stable Streaming LLMs
See how preserving initial attention sinks stabilizes a rolling KV cache, why it costs recent slots, and why stable streaming does not mean infinite recall.
7 min readConcept
Explore machine learning concepts related to Streaming. Clear explanations and practical insights.
See how preserving initial attention sinks stabilizes a rolling KV cache, why it costs recent slots, and why stable streaming does not mean infinite recall.
Explore how LSH uses probabilistic hash functions to find similar vectors in sub-linear time, perfect for streaming and high-dimensional data.