kvcached

Repository facts

Stars

1,807

Forks

216

Open issues and pull requests

103

Primary language

Python

License

Apache License 2.0 (Apache-2.0)

Owner

ovg-project · Organization

Created

May 27, 2025

Last push

Oct 18, 2026

Topics

• elastic-kvcache
• gpu-mutiplexing
• gpu-sharing
• inference-engine
• kvcache
• kvcache-optimization
• kvcached
• llm
• llm-framework
• llm-inference
• llm-serving
• ollama
• online-offline-coserve
• serverless
• sglang
• vllm

Rank #3,967 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 1,389

  Rank 1,389: gpustack/gpustack

  A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

  5,844 starsPythonApache-2.0Last push Oct 16, 2026

• 7,564

  Rank 7,564: pegainfer-project/pegainfer

  Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

  724 starsRustApache-2.0Last push Oct 17, 2026

• 3,852

  Rank 3,852: SemiAnalysisAI/InferenceX

  Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

  1,874 starsPythonApache-2.0Last push Oct 18, 2026

• 1,183

  Rank 1,183: kvcache-ai/Mooncake

  Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

  6,826 starsC++Apache-2.0Last push Oct 18, 2026

• 3,552

  Rank 3,552: microsoft/aici

  AICI: Prompts as (Wasm) Programs

  2,076 starsRustMITLast push Jan 22, 2025

• 6,501

  Rank 6,501: zhihu/ZhiLight

  A highly optimized LLM inference acceleration engine for Llama and its variants.

  907 starsC++Apache-2.0Last push Mar 18, 2026