prometheus-eval

Repository facts

Stars

1,121

Forks

68

Open issues and pull requests

13

Primary language

Python

License

Apache License 2.0 (Apache-2.0)

Owner

prometheus-eval · Organization

Created

Apr 18, 2024

Last push

Apr 25, 2025

Topics

• evaluation
• gpt4
• litellm
• llm
• llm-as-a-judge
• llm-as-evaluator
• llmops
• python
• vllm

Rank #5,660 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 89

  Rank 89: BerriAI/litellm

  The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Az…

  61,490 starsPythonOther licenseLast push Oct 18, 2026

• 1,801

  Rank 1,801: open-compass/VLMEvalKit

  Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

  4,445 starsPythonApache-2.0Last push Oct 16, 2026

• 7,389

  Rank 7,389: ModelTC/LightCompress

  [EMNLP 2024 & AAAI 2026] A powerful toolkit for compressing large models including LLMs, VLMs, and video generative models.

  751 starsPythonApache-2.0Last push May 14, 2026

• 2,619

  Rank 2,619: vllm-project/vllm-ascend

  Community maintained hardware plugin for vLLM on Huawei Ascend

  2,996 starsPythonApache-2.0Last push Oct 18, 2026

• 361

  Rank 361: comet-ml/opik

  Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

  22,708 starsPythonApache-2.0Last push Oct 18, 2026

• 502

  Rank 502: vibrantlabsai/ragas

  Supercharge Your LLM Application Evaluations 🚀

  16,056 starsPythonApache-2.0Last push Feb 24, 2026