Accuracy and Order Sensitivity Diverge Under Label-Free Strategies
Preventing models from seeing option labels during answering does not reliably reduce positional bias or improve multiple-choice accuracy, and only showing all options with an LLM…
Hugging Face · Daily Papers
·Karl Hanna, Chen Feng
·
·▲ 1 upvotes
Este artigo está em destaque na seleção diária de papers do Hugging Face, curada pela comunidade de pesquisa em IA.
Autores: Karl Hanna, Chen Feng
- 1 upvotes da comunidade
- Temas: multiple-choice benchmarks, option order, positional influence, generation-then-matching, LLM matcher, cyclic permutation
Resumo
Resumo original (em inglês), extraído do paper:
Preventing models from seeing option labels during answering does not reliably reduce positional bias or improve multiple-choice accuracy, and only showing all options with an LLM matcher preserves baseline performance.