TensorRT-LLM
Project record
TensorRT-LLM is NVIDIA's open-source library for running large language model inference on NVIDIA GPUs. It provides a Python LLM API, optimized kernels, Python and C++ runtimes, and an online serving command (trtllm-serve).24
- Repository: GitHub repository (external site: github.com)
- Documentation: Documentation (external site: nvidia.github.io)
- License: License (external site: raw.githubusercontent.com)
Availability and license
Overall availability
Source code is public on GitHub; prebuilt wheels are installable with pip and development containers are available from NVIDIA NGC.25
Availability is separate from permission: read the license before using or redistributing.
The license file states the project is under Apache 2.0 but contains portions derived from other open-source projects that may carry different licenses, listed in the same file; individual file headers give specific terms.1
Public materials checklist
| Item | Status | Notes and evidence |
|---|---|---|
| Source codeIs the source code publicly readable? | Public | 2 |
| DocumentationIs user documentation published? | Public | 4 |
| InstallationAre installation instructions or packages publicly available? | Public | Installation via a PyPI wheel, NGC container images, or building from source on Linux.5 |
| Supported platformsAre supported operating systems or hardware documented? | Public | The support matrix lists Linux x86_64 and aarch64 and NVIDIA Ampere through Blackwell GPU architectures.6 |
| Release statusAre versioned releases published? | Public | Versioned releases are published on GitHub; v1.2.1 (April 2026) was the latest non-prerelease when checked.3 |
What it is useful for
The documentation covers offline inference through the LLM API, online serving, quantization, speculative decoding, KV cache management, LoRA adapters, and parallelism strategies for deploying models on NVIDIA GPUs.4
Organization context
U.S. eligibility
Eligible · basis: U.S.-governed project
The repository is published under the NVIDIA GitHub organization and its license file names NVIDIA Corporation & Affiliates as copyright holder. NVIDIA's principal executive offices are in Santa Clara, California, per its Form 10-Q for the quarter ended July 26, 2026.127
Sources
This listing is not an endorsement, a safety assessment, or a federal approval.