Independent project. Not a U.S. government website.Independent project. Not a United States government website.
Support our projectSupport usSupport our project · Buy me a coffee (opens Buy Me a Coffee)This is a processed version of the HuggingFaceH4/stack-exchange-preferences. The following steps were applied: Parse HTML to Markdown with markdownify Create pairs (resp…
153 likes1,875 downloads in the last 30 daysText generationNo license given
SWE-bench is a dataset that tests systems’ ability to solve GitHub issues automatically. The dataset collects 2,294 Issue-Pull Request pairs from 12 popular Python repos…
152 likes95,198 downloads in the last 30 daysNo license given
Action100M: A Large-scale Video Action Dataset Paper | GitHub Action100M is a large-scale dataset constructed from 1.2M Internet instructional videos (14.6 years of dura…
151 likes832 downloads in the last 30 daysVideo classificationFAIR NONCOMMERCIAL RESEARCH LICENSE
Open-SWE-Traces: Advancing Distillation for Software Engineering Agents 🚨 What's New [09/26] Release v1.2: Added new agent trajectories generated by Qwen3.8-27B for min…
151 likes23,594 downloads in the last 30 daysCC BY 4.0
This dataset contains 3.3K expert-level pairwise human preferences for model responses generated by 6 models in response to 80 MT-bench questions. The 6 models are GPT-4…
150 likes3,812 downloads in the last 30 daysQuestion answeringCC BY 4.0
Terminal-Corpus: Large-Scale SFT Dataset for Terminal Agents Terminal-Corpus is a large-scale Supervised Fine-Tuning (SFT) dataset designed to scale the terminal interac…
150 likes20,522 downloads in the last 30 daysQuestion answeringCC BY 4.0
Egocentric-100K is the largest dataset of manual labor. You can visualize the dataset here. Egocentric-100K is state-of-the-art in hand visibility and active manipulatio…
149 likes186,854 downloads in the last 30 daysApache 2.0Gated
SWE-rebench-OpenHands-Trajectories is a dataset of multi-turn agent trajectories for software engineering tasks, collected using Qwen/Qwen3-Coder-480B-A35B-Instruct with…
149 likes9,466 downloads in the last 30 daysCC BY 4.0
This pretraining dataset, for generative AI model training, preserves high-value math and code while enriching it with diverse multilingual Q&A, fueling the next generat…
149 likes40,415 downloads in the last 30 daysText generationOther licenseGated
The EarthView Dataset is a comprehensive collection of multispectral earth imagery. The dataset is divided into four distinct subsets sourced from Satellogic, Sentinel-1…
149 likes1,720 downloads in the last 30 daysCC BY 4.0Gated
Common Voice is Mozilla's initiative to help teach machines how real people speak. The dataset currently consists of 7,335 validated hours of speech in 60 languages, but…
148 likes1,529 downloads in the last 30 daysSpeech recognitionCC0 1.0
Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, Alham Fikri Aji, Dataset Description We distill the knowledge from large language models by performing sent…
148 likes381 downloads in the last 30 daysCC BY-NC 4.0
This dataset provides the entire corpus of arXiv's LaTeX source files, pre-parsed, formatted, and aligned with official metadata in ready-to-query Parquet files. Why I B…
148 likes15,714 downloads in the last 30 daysText generationDual license
CL-bench: A Benchmark for Context Learning Dataset Description CL-bench is a benchmark for evaluating language models' context learning abilities. Resolving tasks in CL-…
148 likes1,290 downloads in the last 30 daysText generationOther license
We released PMC-Patients-V2 (in JSON format with the same keys), which is based on 2024 PMC baseline and contains 250,294 patients. The data collection pipeline remains…
148 likes1,606 downloads in the last 30 daysCC BY-NC-SA 4.0
This dataset is the WizardLM dataset victor123/evol_instruct_70k, removing instances of blatant alignment. 54974 instructions remain. inspired by All credit to anon82314…
147 likes205 downloads in the last 30 daysNo license given
注意：此数据集是由ChatGPT产生的，未经过严格校验，内容可能包含错误。使用过程中请注意这一点。 instruction中包含多轮对话的上文内容，以Human:和Assistant:区分，output中包含当前助手角色的回答。 样例 { "instruction"…
146 likes980 downloads in the last 30 daysGPL 3.0
The GitHub Code clean dataset in a more filtered version of codeparrot/github-code dataset, it consists of 115M code files from GitHub in 32 programming languages with 6…
146 likes40,455 downloads in the last 30 daysApache 2.0
主に，英語で構築されたLLMモデルなどに対して，チャット(Instruction)応答タスクに関してLoRAなどでチューニングするために使用できます． ※様々な公開言語資源を利用させていただきました．関係各位にはこの場を借りて御礼申し上げます． updates 2023/5/15にAlpaca datasetがNCにライセンス変更された…
146 likes380 downloads in the last 30 daysCC BY-SA 4.0
146 likes187 downloads in the last 30 daysOpenRAIL
本数据集是参考Alpaca方法基于GPT4得到的self-instruct数据，约5万条。 Dataset from It is the chinese dataset from Usage and License Notices The data is intended and licensed for research use on…
146 likes2,782 downloads in the last 30 daysText generationCC BY 4.0
Training Data for Text Embedding Models [!NOTE] This repository contains raw datasets, all of which have also been formatted for easy training in the Embedding Model Dat…
145 likes2,531 downloads in the last 30 daysFeature extraction (embeddings)No license given
TACO is a benchmark for Python code generation, it includes 25443 problems and 1000 problems for train and test splits.
144 likes26,690 downloads in the last 30 daysText generationApache 2.0
The Nemotron-Pre-Training-Dataset-v2.1 extends the previously released Nemotron pretraining datasets with refreshed, higher-quality, and more diverse data across math, c…
144 likes6,420 downloads in the last 30 daysText generationOther licenseGated
AI Village is an ongoing experiment by AI Digest in which a group of AI agents — built on frontier models from Anthropic, OpenAI, and Google — live together in a long-ru…
143 likes2,181 downloads in the last 30 daysAi village research termsGated
The dataset consists of 4 versions: bigcode/the-stack-v2: the full "The Stack v2" dataset bigcode/the-stack-v2-dedup: based on the bigcode/the-stack-v2 but further near-…
143 likes2,531 downloads in the last 30 daysText generationOther licenseGated
Abstract from the paper introducing this dataset: High-quality conversational datasets are essential for developing AI models that can communicate with users. One way to…
143 likes3,550 downloads in the last 30 daysCC BY 4.0
ChatMed-Dataset is a dataset of 110,113 medical query-response pairs (in Chinese) generated by OpenAI's GPT-3.5 engine. The queries are crawled from several online medic…
143 likes197 downloads in the last 30 daysText generationCC BY 4.0
FinanceBench is a first-of-its-kind test suite for evaluating the performance of LLMs on open book financial question answering (QA). This is an open source sample of 15…
143 likes7,093 downloads in the last 30 daysCC BY-NC 4.0
Samantha has been trained in philosophy, psychology, and personal relationships. She is an Assistant - but unlike other Assistants, she also wants to be your friend and…
143 likes1,101 downloads in the last 30 daysApache 2.0
✨ (New!) Access Tahoe-100M directly from your coding agents using the Tara plugin The Tara plugin is the new agent-native route for exploring Tahoe datasets including Ta…
143 likes120,244 downloads in the last 30 daysCC0 1.0
The Financial data from Yahoo! *** Key Points to Note *** All financial data is sourced from Yahoo!Ⓡ Finance, Nasdaq!Ⓡ, and the U.S. Department of the Treasury via publi…
142 likes143,407 downloads in the last 30 daysODC-By
The Belebele Benchmark for Massively Multilingual NLU Evaluation Belebele is a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants…
142 likes41,225 downloads in the last 30 daysQuestion answeringCC BY-SA 4.0
RAGBEnch is a large-scale RAG benchmark dataset of 100k RAG examples. It covers five unique industry-specific domains and various RAG task types. RAGBench examples are s…
142 likes3,566 downloads in the last 30 daysCC BY 4.0
Dear researchers and engineers, you're accessing a dataset that would cost millions of dollars to build and took millions of nerves to negotiate favorable terms for its…
142 likes184 downloads in the last 30 daysCC BY-NC 4.0Gated
MegaMath: Pushing the Limits of Open Math Copora Megamath is part of TxT360, curated by LLM360 Team. We introduce MegaMath, an open math pretraining dataset curated from…
142 likes49,346 downloads in the last 30 daysText generationODC-By
This is a dataset designed to enhance the overall truthfulness of LLMs, without sacrificing immersion when roleplaying as a human. For example, in normal AI assistant mo…
142 likes655 downloads in the last 30 daysCC BY 4.0
Everyday conversations for Smol LLMs finetunings This dataset contains 2.2k multi-turn conversations generated by Llama-3.1-70B-Instruct. We ask the LLM to generate a si…
141 likes1,668 downloads in the last 30 daysApache 2.0
Leaderboard: the panel at the top of this page has two views. Use the Task menu in its top-right corner to switch between Lhtb2 Solved (LHTB 2.0, out of 77 tasks) and Lh…
141 likes33,325 downloads in the last 30 daysText generationApache 2.0
An Open Robustness Benchmark for Jailbreaking Language Models NeurIPS 2024 Datasets and Benchmarks Track Paper | Leaderboard | Benchmark code What is JailbreakBench? Jai…
141 likes27,407 downloads in the last 30 daysMIT
The Mathematics Aptitude Test of Heuristics (MATH) dataset consists of problems from mathematics competitions, including the AMC 10, AMC 12, AIME, and more. Each problem…
141 likes15,104 downloads in the last 30 daysMIT
IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages Updates [23 December 2025] We now have 11,200 hours of transcribed data! 🎉 O…
140 likes21,748 downloads in the last 30 daysCC BY 4.0Gated
Note the ODC-BY license, indicating that different licenses apply to subsets of the data. This means that some portions of the dataset are non-commercial. We present the…
140 likes1,010 downloads in the last 30 daysQuestion answeringODC-By
Dataset for summarization of long documents.Adapted from this repo.Note that original data are pre-tokenized so this dataset returns " ".join(text) and add "\n" for para…
140 likes7,399 downloads in the last 30 daysSummarizationNo license given
Mathematics database. This dataset code generates mathematical question and answer pairs, from a range of question types at roughly school-level difficulty. This is desi…
140 likes3,070 downloads in the last 30 daysNo license given
Quick links: [🌐 Website] [📜 Paper] [💻 Code] ParseBench is a benchmark for evaluating document parsing systems on real-world enterprise documents, with the following c…
140 likes27,195 downloads in the last 30 daysApache 2.0
ChartVerse-SFT-1800K is an extended large-scale chart reasoning dataset with Chain-of-Thought (CoT) annotations, developed as part of the opendatalab/ChartVerse project.…
140 likes2,963 downloads in the last 30 daysVisual question answeringApache 2.0
The data fields are the same among all splits. conversations: a List of string . head -n 1 sharegpt_gpt4.jsonl {"conversations":[ {'from': 'human', 'value': '採用優雅現代中文，用中…
139 likes1,032 downloads in the last 30 daysText classificationCC BY 4.0
(TBD) Automated ArXiv question answering via large language models Github | Homepage | Simple QA - Hugging Face Space Automated Question Answering with ArXiv Papers Late…
139 likes240 downloads in the last 30 daysQuestion answeringCC BY 4.0
A dataset of 1.7 million arXiv articles for applications like trend analysis, paper recommender engines, category prediction, co-citation networks, knowledge graph const…
138 likes910 downloads in the last 30 daysTranslationCC0 1.0
This dataset contains questions and answers from the Stack Overflow Data Dump for the purpose of preference model training. Importantly, the questions have been filtered…
138 likes10,508 downloads in the last 30 daysQuestion answeringCC BY-SA 4.0
GraphWalks: a multi hop reasoning long context benchmark In Graphwalks, the model is given a graph represented by its edge list and asked to perform an operation. Exampl…
138 likes5,332 downloads in the last 30 daysMIT
Adopted by NVIDIA's Nemotron family of models! 🤗 HuggingFace ｜ Slack | WeChat Overview OpenResearcher is a fully open agentic large language model (30B-A3B) designed fo…
138 likes3,315 downloads in the last 30 daysMIT
Trendyol Cybersecurity Defense Instruction-Tuning Dataset (v2.0) 🚀 TL;DR 53,202 meticulously curated system/user/assistant instruction-tuning examples covering 200+ spe…
138 likes3,640 downloads in the last 30 daysText generationApache 2.0
PyTorch original implementation of Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset
137 likes2,074 downloads in the last 30 daysQuestion answeringCC BY-NC 4.0
This dataset by AbacusAI was crafted by Eric Hartford This is a synthetic dataset, generated mainly with Mistral-Medium and dolphin-2.7-mixtral-8x7b The purpose of this…
136 likes107 downloads in the last 30 daysApache 2.0
This is a dataset of Spotify tracks over a range of 125 different genres. Each track has some audio features associated with it. The data is in CSV format which is tabul…
136 likes5,046 downloads in the last 30 daysFeature extraction (embeddings)BSD
MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning Code: Project Page: MegaScience is a large-scale mixture of high-quality open-source d…
136 likes5,936 downloads in the last 30 daysText generationCC BY-NC-SA 4.0
Multi-modal Bi-lingual Instruction Dataset for Vision Language Models
136 likes25,428 downloads in the last 30 daysImage to textOther license
The Nemotron-Pre-Training-Dataset-v2.1 extends the previously released Nemotron pretraining datasets with refreshed, higher-quality, and more diverse data across math, c…
136 likes3,874 downloads in the last 30 daysText generationOther licenseGated
🚀 V2 is out! This dataset has a successor: fable-5-premium-v2 — new users should start there. A rigorously cleaned, high-quality supervised fine-tuning (SFT) dataset bu…
136 likes2,832 downloads in the last 30 daysText generationMIT
In this paper, we present OMNI-EDIT, which is an omnipotent editor to handle seven different image editing tasks with any aspect ratio seamlessly. Our contribution is in…
136 likes59,750 downloads in the last 30 daysMIT
The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The dataset…
136 likes270,756 downloads in the last 30 daysImage classificationUnknown license
One million executable, interpretable CAD construction sequences synthesized entirely without real-world data. Zero-to-CAD: Agentic Synthesis of Interpretable CAD Progra…
135 likes12,535 downloads in the last 30 daysText to 3DApache 2.0
The Berkeley function calling leaderboard is a live leaderboard to evaluate the ability of different LLMs to call functions (also referred to as tools). We built this da…
135 likes137,586 downloads in the last 30 daysApache 2.0
Post-OCR correction is a large corpus of 1 billion words containing original texts with a varying number of OCR mistakes and an experimental multilingual post-OCR correc…
135 likes1,076 downloads in the last 30 daysCC0 1.0
Dataset for O1 Replication Journey: A Strategic Progress Report Usage from datasets import load_dataset dataset = load_dataset("GAIR/o1-journey", split="train") Citation…
134 likes319 downloads in the last 30 daysNo license given
SMOL (Set for Maximal Overall Leverage) is a collection professional translations into 221 Low-Resource Languages, for the purpose of training translation models, and ot…
134 likes5,033 downloads in the last 30 daysTranslationCC BY 4.0
"A Question-Entailment Approach to Question Answering". Asma Ben Abacha and Dina Demner-Fushman. BMC Bioinformatics, 2019.
134 likes7,319 downloads in the last 30 daysQuestion answeringNo license given
134 likes15,200 downloads in the last 30 daysCC BY 4.0
The Open Super-large Crawled Aggregated coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the Ungol…
134 likes1,414 downloads in the last 30 daysFill maskCC0 1.0Gated
This repository contains the RL training datasets used in the paper Deep But Reliable: Advancing Multi-turn Reasoning for Thinking with Images
134 likes251 downloads in the last 30 daysApache 2.0
A 900-prompt factuality benchmark from Google DeepMind, designed to evaluate agents on difficult multi-step information-seeking tasks across 17 different fields. ▶ Googl…
133 likes19,432 downloads in the last 30 daysQuestion answeringApache 2.0
Recent advancements in AI, particularly in large language models (LLMs), have led to significant breakthroughs in mathematical reasoning capabilities. However, existing…
133 likes25,440 downloads in the last 30 daysApache 2.0
The Nemotron-Instruction-Following-Chat-v1 dataset is designed to broadly strengthen the model’s interactive capabilities, spanning open-ended chat, precise instruction…
133 likes2,847 downloads in the last 30 daysCC BY 4.0
Mind2Web is a dataset for developing and evaluating generalist agents for the web that can follow language instructions to complete complex tasks on any website. Existin…
133 likes12,380 downloads in the last 30 daysCC BY 4.0
ABC-130k is the largest open-source robot teleoperation dataset. It contains bimanual manipulation trajectories collected on two-arm YAM stations. Episodes are distribut…
133 likes242,451 downloads in the last 30 daysRoboticsApache 2.0Gated
🚀 Creating the highest-quality pre-training datasets for LLMs 🌟 📄 PAPER 🤗 CLIMBLAB 🤗 CLIMBMIX 🏠 HOMEPAGE Figure 1: Continuously training a 1B model yields a 2.0% i…
132 likes8,161 downloads in the last 30 daysText generationCC BY-NC 4.0
Nemotron-Personas-Japan は、日本における人口の多様性と豊かさを捉えることを目的とし、実世界の人口統計、地理的分布、性格特性の分布に基づいて合成的に生成されたペルソナのオープンソースデータセットです。名前、性別、年齢、背景、婚姻状況、学歴、職業、居住地などの統計に基づいて生成した初のデータセットされた Nemotr…
132 likes2,837 downloads in the last 30 daysText generationCC BY 4.0
This dataset contains a diverse set of natural Japanese speech, collected from terrestrial television streams. It contains more than 35000 hours of audio. Paper: ReazonS…
132 likes800 downloads in the last 30 daysSpeech recognitionOther licenseGated
This is a subset of SmolTalk dataset adapted for smol models with less than 1B parameters. We used it to build SmolLM2-360M-Instruct and SmolLM2-135M-Instruct. We do SFT…
131 likes12,843 downloads in the last 30 daysApache 2.0
🍃 MINT-1T:Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens 🍃 MINT-1T is an open-source Multimodal INTerleaved dataset with 1 t…
131 likes848,113 downloads in the last 30 daysImage to textCC BY 4.0
Atlas Map: Map of Cleaned Data
131 likes105 downloads in the last 30 daysText generationApache 2.0
Please see paper & website for more information: Overview Cambrian-10M is a comprehensive dataset designed for instruction tuning, particularly in multimodal settings in…
131 likes21,719 downloads in the last 30 daysVisual question answeringApache 2.0
UltraInteract is a large-scale, high-quality alignment dataset specifically designed for complex reasoning tasks. For each instruction, it includes a preference tree con…
131 likes1,061 downloads in the last 30 daysMIT
👉 Looking for the newest release? The current flagship is ai4privacy/pii-masking-openpii-1.5m. 1.6M samples, 30 languages, 19 PII classes, Asia Pacific extension.?** Th…
130 likes3,811 downloads in the last 30 daysText classificationNo license given
The largest agentic-coding trace dataset to date: 112 B tokens of execution-free agentic trajectories covering 122 K pull requests, 3 K repositories, and 16 programming…
130 likes3,272 downloads in the last 30 daysText generationApache 2.0
The NQ corpus contains questions from real users, and it requires QA systems to read and comprehend an entire Wikipedia article that may or may not contain the answer to…
130 likes16,612 downloads in the last 30 daysQuestion answeringCC BY-SA 3.0
SWE-chat: Coding Agent Interactions From Real Users in the Wild 📄 COLM 2026 Paper: arxiv.org/abs/2604.20779 🌐 Website: swe-chat.com Dataset Summary SWE-chat captures r…
130 likes12,260 downloads in the last 30 daysText generationODC-ByGated
本项目由交易猫基金会支持（交易猫基金会 CA：0x8a99b8d53eff6bc331af529af74ad267f3167777）。 精简版币安期货历史数据 - 只包含 4 个主要币种，适合快速下载和研究使用。 📊 数据概览 文件 记录数 压缩大小 时间范围 说明 candles_1m_main4_*.bin.zst 998 万 3…
130 likes495 downloads in the last 30 daysTime-series forecastingMIT
包含从12560本网文提取的约21.7M条可用于训练小说生成的中文指令数据(novel_json_tokens512.zip)。下载链接： 提取码：jlh3 以及从中提取出的包含50k条数据的子集(novel_cn_token512_50k.json)。其中输入和输出都不多于 512 tokens。 样例 在原有小说文本基础上，依据下列…
130 likes584 downloads in the last 30 daysMIT
Note: This dataset is part of the lineup CodeX by Modotte. You can get lots of datasets in this same lineup, with the main focus on providing very high-quality datasets…
129 likes686 downloads in the last 30 daysText generationApache 2.0
Paper: Nemotron-Math: Efficient Long-Context Distillation of Mathematical Reasoning from Multi-Mode SupervisionCode: Nemotron-MathProofs-v1 documentation Dataset Descrip…
129 likes1,144 downloads in the last 30 daysText generationCC BY-SA 4.0
This repository hosts data for Scalable 3D Captioning with Pretrained Models and View Selection for 3D Captioning via Diffusion Ranking, including descriptive captions f…
129 likes22,864 downloads in the last 30 daysText to 3DODC-By
OmniScience: A Large-scale Multi-modal Dataset for Scientific Image Understanding 🚀 2026-05-17: This work was accepted by the KDD 2026 Dataset & Benchmark Track. 🚀 202…
129 likes971 downloads in the last 30 daysImage to textCC BY-NC-SA 4.0Gated
RealWorldQA is a benchmark designed for real-world understanding. The dataset consists of anonymized images taken from vehicles, in addition to other real-world images.…
129 likes4,552 downloads in the last 30 daysCC BY ND 4.0
Movie Review Dataset. This is a dataset of containing 5,331 positive and 5,331 negative processed sentences from Rotten Tomatoes movie reviews. This data was first used…
128 likes59,053 downloads in the last 30 daysText classificationUnknown license
Qwen3.5-reasoning-700x is a high-quality distilled dataset. This dataset uses the high-quality instructions constructed by Alibaba-Superior-Reasoning-Stage2 as the seed…
128 likes266 downloads in the last 30 daysQuestion answeringApache 2.0
Paper | Code | Project Page Dataset Description This dataset is specifically designed for training the agentic capabilities of Large Language Models (LLMs). The dataset…
128 likes1,387 downloads in the last 30 daysText generationODC-By
A large crowd-sourced dataset for developing natural language interfaces for relational databases
128 likes1,532 downloads in the last 30 daysUnknown license
Help keep USASI useful.
Optional. No USASI account required. Payment takes place on the linked provider’s website (Buy Me a Coffee).
About supporting this project