the_pile_deduplicated

Dataset facts

Hub id

EleutherAI/the_pile_deduplicated

Likes

121

Downloads (last 30 days)

13,476

Tasks

Not given

Modality

text

Size

100M to 1B examples

License

No license given

Access

Open download

Author

EleutherAI · Organization

Languages

Not given

Created

Dec 2, 2022

Last modified

Dec 2, 2022

Rank #639 of 836 datasets in AI on Hugging Face, by likes. Data from the Hugging Face Hub API, fetched Oct 18, 2026. Likes measure popularity, not quality or safety. Downloads are for the last 30 days, as the Hub reports them; counts change daily. A listing is not an endorsement. How this list is made

More datasets from EleutherAI

• 83

  Rank 83: EleutherAI/pile

  The Pile is a 825 GiB diverse, open source language modelling data set that consists of 22 smaller, high-quality datasets combined together.

  512 likes8,740 downloads in the last 30 daysText generationOther license

• 268

  Rank 268: EleutherAI/proof-pile-2

  A dataset of high quality mathematical text.

  242 likes30,341 downloads in the last 30 daysText generationNo license given

• 673

  Rank 673: EleutherAI/hendrycks_math

  MATH dataset from Citation Information @article{hendrycksmath2021, title={Measuring Mathematical Problem Solving With the MATH Dataset}, author={Dan Hendrycks and Collin…

  116 likes141,372 downloads in the last 30 daysMIT

• 829

  Rank 829: EleutherAI/pile-standard-pythia-preshuffled

  5 likes4,740 downloads in the last 30 daysNo license given

• 836

  Rank 836: EleutherAI/pile_val_test

  The Pile: Validation and Test Splits This repo contains the validation and test splits of The Pile, an 825 GiB English text dataset designed for training large language…

  0 likes503 downloads in the last 30 daysText generationMIT