Paper
Dados & Embeddings
OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers
OmniOpt presents a unified framework for optimizer selection in large-scale model training by combining meta-pipeline transformations, norm-constrained linear minimization oracles,…
Hugging Face · Daily Papers
·Siyuan Li, Jiabao Pan
·
·▲ 30 upvotes
Este artigo está em destaque na seleção diária de papers do Hugging Face, curada pela comunidade de pesquisa em IA.
Autores: Siyuan Li, Jiabao Pan, Yumou Liu, Zhuoli Ouyang, Xin Jin, Xinglong Xu
- 30 upvotes da comunidade
- Temas: optimizer selection, large-scale model training, meta-pipeline, norm-constrained linear minimization oracles, cross-domain benchmark, optimizer families
Resumo
Resumo original (em inglês), extraído do paper:
OmniOpt presents a unified framework for optimizer selection in large-scale model training by combining meta-pipeline transformations, norm-constrained linear minimization oracles, and a cross-domain benchmark to systematically analyze optimizer families and their trade-offs across different training objectives and model scales.Onde ler
// relacionados
Leia também
Blog
Flight attendants freaked out that Google is buying tons of Spirit employee data
Blog
Anthropic says any lab can now let a language model agent run the whole protein design stack
Blog
Valid Per-Field Selective Risk Control for Document Extraction: Three Failure Modes, a Validity Ladder, and When Conditioning Pays
Blog