Holistic Data Scheduler for LLM Pre-training via Multi-Objective Reinforcement Learning
A novel online data mixing framework called Holistic Data Scheduler uses reinforcement learning with a multi-objective reward function to optimize large language model pre-training…
Hugging Face · Daily Papers
·Chenhao Dang, Jing Ma
·
·▲ 3 upvotes
Este artigo está em destaque na seleção diária de papers do Hugging Face, curada pela comunidade de pesquisa em IA.
Autores: Chenhao Dang, Jing Ma, Mingjie Liao
- 3 upvotes da comunidade
- Temas: Large Language Model, online data mixing, reinforcement learning, Soft Actor-Critic, multi-objective reward function, data-driven reward
Resumo
Resumo original (em inglês), extraído do paper:
A novel online data mixing framework called Holistic Data Scheduler uses reinforcement learning with a multi-objective reward function to optimize large language model pre-training efficiency and performance.