inference
by xorbitsai (organization)
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API. (description from the project’s GitHub page)
Repository facts
- Stars
- 9,616
- Forks
- 884
- Open issues and pull requests
- 31
- Primary language
- Python
- License
- Apache License 2.0 (Apache-2.0)
- Owner
- xorbitsai · Organization
- Created
- Jun 14, 2023
- Last push
- Oct 18, 2026
Topics
Related projects
Projects that share this one’s GitHub topics.
Rank 3,852: SemiAnalysisAI/InferenceX
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
1,874 starsPythonApache-2.0Last push Oct 18, 2026
Rank 8,611: zhongkaifu/TensorSharp
A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-comp…
592 starsC#BSD-3-ClauseLast push Oct 18, 2026
Rank 7,564: pegainfer-project/pegainfer
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
724 starsRustApache-2.0Last push Oct 17, 2026
Rank 1,389: gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
5,844 starsPythonApache-2.0Last push Oct 16, 2026
Rank 7,018: XHToken/Spark-X2.5
Spark-x2.5 open model series. Pushing the Limits of Agentic Capabilities in On-Device Models
812 starsApache-2.0Last push Sep 20, 2026
Rank 7,913: StartLuxLabs/StartLux-Decision
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
676 starsPythonApache-2.0Last push Oct 17, 2026