promptfoo
by promptfoo (organization)
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic. (description from the project’s GitHub page)
Repository facts
- Stars
- 26,109
- Forks
- 2,508
- Open issues and pull requests
- 775
- Primary language
- TypeScript
- License
- MIT License (MIT)
- Owner
- promptfoo · Organization
- Created
- Apr 28, 2023
- Last push
- Oct 18, 2026
Topics
Related projects
Projects that share this one’s GitHub topics.
Rank 198: langfuse/langfuse
🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
35,996 starsTypeScriptOther licenseLast push Oct 18, 2026
Rank 1,300: Helicone/helicone
🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓
6,225 starsTypeScriptApache-2.0Last push Oct 10, 2026
Rank 361: comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
22,708 starsPythonApache-2.0Last push Oct 18, 2026
Rank 680: Arize-ai/phoenix
AI Observability & Evaluation
11,897 starsPythonOther licenseLast push Oct 17, 2026
Rank 3,199: uptrain-ai/uptrain
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding u…
2,366 starsPythonApache-2.0Last push Aug 18, 2024
Rank 9,325: relari-ai/continuous-eval
Data-Driven Evaluation for LLM-Powered Applications
519 starsPythonApache-2.0Last push Aug 10, 2026