HuggingFaceCode/stack-v3-train

O dataset HuggingFaceCode/stack-v3-train está entre os destaques do Hugging Face — dados que alimentam o treinamento e a avaliação dos modelos do momento.

Ficha do dataset

Sobre o dataset

🥞 The Stack v3 What is it? What is being released How to download and use it Dataset statistics Dataset structure Dataset creation Considerations for using the data Additional information What is it? The Stack v3 is the largest, most up-to-date open dataset of source code, crawled directly from GitHub and built to pre-train code LLMs with full-repository context.

Como carregar

Use a biblioteca datasets do Hugging Face:

pip install -U datasets

Como é um dataset grande, vale carregar em modo streaming (sem baixar tudo):

from datasets import load_dataset

ds = load_dataset("HuggingFaceCode/stack-v3-train", split="train", streaming=True)
for exemplo in ds.take(3):
    print(exemplo)

Tags

text-generation code

Explorar o dataset no Hugging Face →