Paper
LLMs & Texto
KVpop -- Key-Value Cache Compression with Predictive Online Pruning
KVpop learns optimal key-value cache eviction by directly supervising keep-or-drop decisions using future-attention targets, achieving high performance with reduced memory usage.
Hugging Face · Daily Papers
·Lukas Hauzenberger, Niklas Schmidinger
·
·▲ 10 upvotes
Este artigo está em destaque na seleção diária de papers do Hugging Face, curada pela comunidade de pesquisa em IA.
Autores: Lukas Hauzenberger, Niklas Schmidinger, Anamaria-Roberta Hartl, David Stap, Thomas Schmied, Sebastian Böck
- 10 upvotes da comunidade
- Temas: KV cache, autoregressive decoding, KV eviction, future-attention target, delayed memory-based scorer, attention maps
Resumo
Resumo original (em inglês), extraído do paper:
KVpop learns optimal key-value cache eviction by directly supervising keep-or-drop decisions using future-attention targets, achieving high performance with reduced memory usage.