DatasetRank #419 of 836 178 likes

MADLAD-400

Name: MADLAD-400
Creator: allenai
License: https://spdx.org/licenses/ODC-By-1.0.html
Keywords: Text generation

by allenai (organization)

MADLAD-400 (Multilingual Audited Dataset: Low-resource And Document-level) is a document-level multilingual dataset based on Common Crawl, covering 419 languages in total. This uses all snapshots of CommonCrawl available as of August 1, 2022. The primary advantage of this… (excerpt from the dataset card on Hugging Face)

View on Hugging Face

Dataset facts

Hub id: allenai/MADLAD-400
Likes: 178
Downloads (last 30 days): 40,883
Tasks: Text generation
Modality: Not given
Size: More than 1T examples
License: ODC-By (odc-by)
Access: Open download
Author: allenai · Organization
Languages: Not given
Created: Sep 1, 2023
Last modified: Sep 9, 2024

Rank #419 of 836 datasets in AI on Hugging Face, by likes. Data from the Hugging Face Hub API, fetched Oct 18, 2026. Likes measure popularity, not quality or safety. Downloads are for the last 30 days, as the Hub reports them; counts change daily. A listing is not an endorsement. How this list is made

More datasets from allenai

Rank 20: allenai/dolma
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
1,107 likes2,859 downloads in the last 30 daysText generationODC-By
Rank 52: allenai/c4
A colossal, cleaned version of Common Crawl's web crawl corpus. Based on Common Crawl dataset: " This is the processed version of Google's C4 dataset We prepared five va…
688 likes773,370 downloads in the last 30 daysText generationODC-By
Rank 97: allenai/objaverse
Objaverse is a Massive Dataset with 800K+ Annotated 3D Objects. More documentation is coming soon. In the meantime, please see our paper and website for additional detai…
477 likes477,055 downloads in the last 30 daysODC-By
Rank 101: allenai/WildChat-1M
Interactive Search Tool: (paper) License: ODC-BY Language(s) (NLP): multi-lingual Point of Contact: Yuntian Deng Dataset Summary WildChat is a collection of 1 million co…
474 likes36,853 downloads in the last 30 daysText generationODC-By
Rank 121: allenai/ai2_arc
A new dataset of 7,787 genuine grade-school level, multiple-choice science questions, assembled to encourage research in advanced question-answering. The dataset is part…
424 likes601,170 downloads in the last 30 daysQuestion answeringCC BY-SA 4.0
Rank 190: allenai/olmOCR-bench
olmOCR-bench is a dataset of 1,403 PDF files, plus 7,010 unit test cases that capture properties of the output that a good OCR system should have. This benchmark evaluat…
309 likes34,698 downloads in the last 30 daysODC-By

Dataset facts

More datasets from allenai

Rank 20: allenai/dolma

Rank 52: allenai/c4

Rank 97: allenai/objaverse

Rank 101: allenai/WildChat-1M

Rank 121: allenai/ai2_arc

Rank 190: allenai/olmOCR-bench

Support Us