PowerInfer

Repository facts

Stars

9,837

Forks

623

Open issues and pull requests

133

Primary language

C++

License

MIT License (MIT)

Owner

Tiiny-AI · Organization

Created

Dec 15, 2023

Last push

May 11, 2026

Topics

• large-language-models
• llama
• llm
• llm-inference
• local-inference

Rank #832 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 1,372

  Rank 1,372: lemonade-sdk/lemonade

  Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

  5,927 starsC++Apache-2.0Last push Oct 18, 2026

• 6,501

  Rank 6,501: zhihu/ZhiLight

  A highly optimized LLM inference acceleration engine for Llama and its variants.

  907 starsC++Apache-2.0Last push Mar 18, 2026

• 8,484

  Rank 8,484: andrewkchan/yalm

  Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O

  604 starsC++No license file detectedLast push Sep 13, 2025

• 645

  Rank 645: bentoml/OpenLLM

  Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

  12,559 starsPythonApache-2.0Last push Oct 12, 2026

• 1,005

  Rank 1,005: InternLM/lmdeploy

  LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

  8,108 starsPythonApache-2.0Last push Oct 16, 2026

• 1,389

  Rank 1,389: gpustack/gpustack

  A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

  5,844 starsPythonApache-2.0Last push Oct 16, 2026