Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE
A novel zero-shot method called Jet-Long enables efficient long-context processing for large language models by dynamically adapting rescaling factors and utilizing a bifocal atten…
Hugging Face · Daily Papers
·Haozhan Tang, Zerui Wang
·
·▲ 16 upvotes
Este artigo está em destaque na seleção diária de papers do Hugging Face, curada pela comunidade de pesquisa em IA.
Autores: Haozhan Tang, Zerui Wang, Yuxian Gu, Song Han, Han Cai
- 16 upvotes da comunidade
- Temas: LLMs, long-context applications, retrieval-augmented generation, agentic workflows, zero-shot context extension, RoPE-faithful window
Resumo
Resumo original (em inglês), extraído do paper:
A novel zero-shot method called Jet-Long enables efficient long-context processing for large language models by dynamically adapting rescaling factors and utilizing a bifocal attention mechanism that maintains high performance across varying sequence lengths.