aici

Repository facts

Stars

2,076

Forks

86

Open issues and pull requests

41

Primary language

Rust

License

MIT License (MIT)

Owner

microsoft · Organization

Created

Sep 26, 2023

Last push

Jan 22, 2025

Topics

• ai
• inference
• language-model
• llm
• llm-framework
• llm-inference
• llm-serving
• llmops
• model-serving
• rust
• transformer
• wasm
• wasmtime

Rank #3,552 of 9,507 in AI on GitHub, by stars. Data from the GitHub API, fetched Oct 18, 2026; star counts change daily. Stars measure popularity, not quality or safety, and a listing is not an endorsement. How this list is made

Related projects

Projects that share this one’s GitHub topics.

• 7,564

  Rank 7,564: pegainfer-project/pegainfer

  Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

  724 starsRustApache-2.0Last push Oct 17, 2026

• 2,619

  Rank 2,619: vllm-project/vllm-ascend

  Community maintained hardware plugin for vLLM on Huawei Ascend

  2,996 starsPythonApache-2.0Last push Oct 18, 2026

• 922

  Rank 922: bentoml/BentoML

  The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

  8,906 starsPythonApache-2.0Last push Oct 5, 2026

• 4,923

  Rank 4,923: alibaba/rtp-llm

  RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

  1,362 starsPythonApache-2.0Last push Oct 18, 2026

• 43

  Rank 43: vllm-project/vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  94,068 starsPythonApache-2.0Last push Oct 18, 2026

• 1,506

  Rank 1,506: superduper-io/superduper

  Superduper: End-to-end framework for building custom AI applications and agents.

  5,325 starsPythonApache-2.0Last push Sep 1, 2025