AxiomicLabs/Arithmark-3.0
Dataset com 1 mil – 10 mil exemplos — 251 downloads no Hugging Face. ArithMark 3.0 ArithMark 3.0 is our 3rd gen benchmark for evaluating arithmetic ability in language models. As a changeup from our previous two entries…
O dataset AxiomicLabs/Arithmark-3.0 está entre os destaques do Hugging Face — dados que alimentam o treinamento e a avaliação dos modelos do momento.
Ficha do dataset
- Tamanho: 1 mil – 10 mil exemplos
- Tarefas: perguntas e respostas
- Idiomas: inglês
- Licença: Apache 2.0
- Downloads: 251 · Curtidas: 9
Sobre o dataset
ArithMark 3.0 ArithMark 3.0 is our 3rd gen benchmark for evaluating arithmetic ability in language models. As a changeup from our previous two entries, problems are expressed as continuation style short English word problems rather than bare equations. The benchmark is designed primarily for base-model continuation log-likelihood scoring. It does not require instruction following, chain-of-thought, or generated explanations. Random-choice accuracy is 25%. The included… See the full description on the dataset page: https://huggingface.co/datasets/AxiomicLabs/Arithmark-3.0.
Como carregar
Use a biblioteca datasets do Hugging Face:
pip install -U datasets
from datasets import load_dataset
ds = load_dataset("AxiomicLabs/Arithmark-3.0")
print(ds)
print(ds["train"][0])Tags
question-answering arithmetic multiple-choice synthetic language-model-evaluation