rtp-llm

Repository facts

Stars

1,362

Forks

284

Open issues and pull requests

238

Primary language

Python

License

Apache License 2.0 (Apache-2.0)

Owner

alibaba · Organization

Created

Dec 27, 2023

Last push

Oct 18, 2026

Topics

• gpt
• inference
• llama
• llm
• llm-serving
• llmops
• model-serving

Rank #4,923 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 43

  Rank 43: vllm-project/vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  94,068 starsPythonApache-2.0Last push Oct 18, 2026

• 2,619

  Rank 2,619: vllm-project/vllm-ascend

  Community maintained hardware plugin for vLLM on Huawei Ascend

  2,996 starsPythonApache-2.0Last push Oct 18, 2026

• 3,552

  Rank 3,552: microsoft/aici

  AICI: Prompts as (Wasm) Programs

  2,076 starsRustMITLast push Jan 22, 2025

• 6,501

  Rank 6,501: zhihu/ZhiLight

  A highly optimized LLM inference acceleration engine for Llama and its variants.

  907 starsC++Apache-2.0Last push Mar 18, 2026

• 922

  Rank 922: bentoml/BentoML

  The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

  8,906 starsPythonApache-2.0Last push Oct 5, 2026

• 645

  Rank 645: bentoml/OpenLLM

  Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

  12,559 starsPythonApache-2.0Last push Oct 12, 2026