
dolma
by allenai (organization)
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research (excerpt from the dataset card on Hugging Face)
Dataset facts
- Hub id
- allenai/dolma
- Likes
- 1,107
- Downloads (last 30 days)
- 2,859
- Tasks
- Text generation
- Modality
- Not given
- Size
- More than 1T examples
- License
- ODC-By (odc-by)
- Access
- Open download
- Author
- allenai · Organization
- Languages
- en
- Created
- Jun 30, 2023
- Last modified
- Apr 17, 2024
More datasets from allenai
Rank 52: allenai/c4
A colossal, cleaned version of Common Crawl's web crawl corpus. Based on Common Crawl dataset: " This is the processed version of Google's C4 dataset We prepared five va…
688 likes773,370 downloads in the last 30 daysText generationODC-By
Rank 97: allenai/objaverse
Objaverse is a Massive Dataset with 800K+ Annotated 3D Objects. More documentation is coming soon. In the meantime, please see our paper and website for additional detai…
477 likes477,055 downloads in the last 30 daysODC-By
Rank 101: allenai/WildChat-1M
Interactive Search Tool: (paper) License: ODC-BY Language(s) (NLP): multi-lingual Point of Contact: Yuntian Deng Dataset Summary WildChat is a collection of 1 million co…
474 likes36,853 downloads in the last 30 daysText generationODC-By
Rank 121: allenai/ai2_arc
A new dataset of 7,787 genuine grade-school level, multiple-choice science questions, assembled to encourage research in advanced question-answering. The dataset is part…
424 likes601,170 downloads in the last 30 daysQuestion answeringCC BY-SA 4.0
Rank 190: allenai/olmOCR-bench
olmOCR-bench is a dataset of 1,403 PDF files, plus 7,010 unit test cases that capture properties of the output that a good OCR system should have. This benchmark evaluat…
309 likes34,698 downloads in the last 30 daysODC-By
Rank 225: allenai/tulu-3-sft-mixture
Note that this collection is licensed under ODC-BY-1.0 license; different licenses apply to subsets of the data. Some portions of the dataset are non-commercial. We pres…
275 likes30,570 downloads in the last 30 daysOtherODC-By