ZhiLight

Repository facts

Stars

907

Forks

109

Open issues and pull requests

6

Primary language

C++

License

Apache License 2.0 (Apache-2.0)

Owner

zhihu · Organization

Created

Dec 6, 2024

Last push

Mar 18, 2026

Topics

• cuda
• deepseek-r1
• gpt
• inference-engine
• llama
• llm
• llm-inference
• llm-serving
• model-serving
• pytorch

Rank #6,501 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 43

  Rank 43: vllm-project/vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  94,068 starsPythonApache-2.0Last push Oct 18, 2026

• 7,564

  Rank 7,564: pegainfer-project/pegainfer

  Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

  724 starsRustApache-2.0Last push Oct 17, 2026

• 8,484

  Rank 8,484: andrewkchan/yalm

  Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O

  604 starsC++No license file detectedLast push Sep 13, 2025

• 4,923

  Rank 4,923: alibaba/rtp-llm

  RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

  1,362 starsPythonApache-2.0Last push Oct 18, 2026

• 1,389

  Rank 1,389: gpustack/gpustack

  A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

  5,844 starsPythonApache-2.0Last push Oct 16, 2026

• 3,967

  Rank 3,967: ovg-project/kvcached

  Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

  1,807 starsPythonApache-2.0Last push Oct 18, 2026