TensorRT-LLM
by NVIDIA (organization)
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference… (description from the project’s GitHub page)
Repository facts
- Stars
- 14,849
- Forks
- 2,832
- Open issues and pull requests
- 1,707
- Primary language
- Python
- License
- Other license
- Owner
- NVIDIA · Organization
- Created
- Aug 16, 2023
- Last push
- Oct 18, 2026
Topics
Related projects
Projects that share this one’s GitHub topics.
Rank 43: vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
94,068 starsPythonApache-2.0Last push Oct 18, 2026
Rank 185: sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
37,276 starsPythonApache-2.0Last push Oct 18, 2026
Rank 7,564: pegainfer-project/pegainfer
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
724 starsRustApache-2.0Last push Oct 17, 2026
Rank 4,833: GradientHQ/parallax
Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere
1,397 starsPythonApache-2.0Last push Jul 1, 2026
Rank 3,402: MoonshotAI/MoBA
MoBA: Mixture of Block Attention for Long-Context LLMs
2,194 starsPythonMITLast push Apr 3, 2025
Rank 3,852: SemiAnalysisAI/InferenceX
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
1,874 starsPythonApache-2.0Last push Oct 18, 2026