Craw4LLM

Repository facts

Stars

666

Forks

60

Open issues and pull requests

4

Primary language

Python

License

MIT License (MIT)

Owner

cxcscmu · Organization

Created

Feb 17, 2025

Last push

Feb 24, 2025

Topics

• crawler
• crawling
• large-language-models
• llm
• pre-training
• pretraining
• web-crawler
• web-crawling

Rank #7,965 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 854

  Rank 854: apify/crawlee-python

  Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files…

  9,597 starsPythonApache-2.0Last push Oct 16, 2026

• 313

  Rank 313: apify/crawlee

  Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML,…

  26,183 starsTypeScriptApache-2.0Last push Oct 16, 2026

• 52

  Rank 52: unclecode/crawl4ai

  Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.

  85,765 starsPythonApache-2.0Last push Oct 15, 2026

• 239

  Rank 239: ScrapeGraphAI/Scrapegraph-ai

  Python scraper based on AI

  31,902 starsPythonMITLast push Oct 6, 2026

• 658

  Rank 658: RUCAIBox/LLMSurvey

  The official GitHub page for the survey paper "A Survey of Large Language Models".

  12,233 starsPythonNo license file detectedLast push Mar 11, 2025

• 1,137

  Rank 1,137: datajuicer/data-juicer

  Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

  7,154 starsPythonApache-2.0Last push Oct 16, 2026