sarathi-serve

Repository facts

Stars

534

Forks

68

Open issues and pull requests

17

Primary language

Python

License

Apache License 2.0 (Apache-2.0)

Owner

microsoft · Organization

Created

Nov 2, 2023

Last push

Jan 8, 2026

Topics

• llama
• llm-inference
• pytorch
• transformer

Rank #9,171 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 43

  Rank 43: vllm-project/vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  94,068 starsPythonApache-2.0Last push Oct 18, 2026

• 768

  Rank 768: bigscience-workshop/petals

  🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

  10,639 starsPythonMITLast push Sep 7, 2024

• 2,609

  Rank 2,609: pytorch/ao

  PyTorch native quantization for training and inference

  3,005 starsPythonOther licenseLast push Oct 16, 2026

• 3,021

  Rank 3,021: X-PLUG/mPLUG-Owl

  mPLUG-Owl: The Powerful Multi-modal Large Language Model Family

  2,539 starsPythonMITLast push Apr 2, 2025

• 4,833

  Rank 4,833: GradientHQ/parallax

  Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere

  1,397 starsPythonApache-2.0Last push Jul 1, 2026

• 6,577

  Rank 6,577: explosion/curated-transformers

  🤖 A PyTorch library of curated Transformer models and their composable components

  890 starsPythonMITLast push Apr 17, 2024