Sign in
← Back to search
github Active AI dev

Repository profile

NVIDIA/TensorRT-LLM

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Python NOASSERTION main Stack scanned README.md
Sign in to like NVIDIA/TensorRT-LLM
Open website Open GitHub
Stars
14,118
Forks
2,561
Watchers
118
Issues
1,507
Commits
7,877
Awesome lists
2

Repository updates

Follow repository updates

Get generated NVIDIA/TensorRT-LLM development summaries by email, or follow the weekly and monthly RSS feeds.

Weekly RSS Monthly RSS

Sign in to subscribe by email. RSS feeds are public.

Sign in to subscribe

Activity and growth

Tracked growth, recent movement, and commit velocity from stored repository snapshots.

Latest capture 2026-07-15 03:16

Star growth, last 7 days
0 0.0%
Commit velocity, last 7 days
0 0.0%
Stars since baseline
+393
Snapshot coverage
5

Tracked growth

5 captures since 2026-05-25

Stars from baseline +393

Time horizon

All tracked data

Stars history

Total stars

Commits history

Default branch commits

Detected stack

Frameworks, package managers, ecosystems, and dependency manifests found during catalog scans.

Scanned 2026-07-15 03:16

Stack signals
4
Package managers
6
Manifest files
80
Dependencies
684

Frameworks and tools

  • FastAPI web framework · high confidence
  • Jupyter notebook · high confidence
  • pytest test framework · high confidence
  • Starlette web framework · high confidence
CMake Conan PEP 517 pip Poetry uv c-cpp python

Dependency files

80 manifests
  • constraints.txt python ecosystem, 5 dependencies
  • pyproject.toml python ecosystem, 2 dependencies
  • requirements-dev.txt python ecosystem, 50 dependencies
  • requirements.txt python ecosystem, 90 dependencies
  • setup.py python ecosystem, 0 dependencies
  • 3rdparty/CMakeLists.txt c-cpp ecosystem, 2 dependencies
  • cpp/CMakeLists.txt c-cpp ecosystem, 9 dependencies
  • cpp/conanfile.py c-cpp ecosystem, 0 dependencies
  • 72 more files

Classification

Searchable topics, generated tags, and stack labels that explain where this repository fits.

Topics
5
Tags
0
Stacks
4

Topics

#blackwell #cuda #llm-serving #moe #pytorch

Generated tags

No generated tags yet.

Stack labels

FastAPI Jupyter pytest Starlette

AI development signals

Agent instructions and tool configuration paths found in the repository tree.

200 paths

AI agent config detected

200 config paths 145 files 55 directories
Agent instructions 3 Claude Code 196 CodeRabbit

Key config paths

1 more config path detected.

Review config paths

Showing the first 24 paths. 176 more detected.

Similar repositories

Nearest indexed repositories by embedding similarity.

NVIDIA/Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

3,232 stars
Python 1 awesome list

llm-d/llm-d

Achieve state of the art inference performance with modern accelerators on Kubernetes

3,800 stars
Shell 1 awesome list

ikawrakow/ik_llama.cpp

llama.cpp fork with additional SOTA quants and improved performance

2,893 stars
C++ 1 awesome list

tensorzero/tensorzero

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

11,695 stars
Rust 4 awesome lists

InternLM/lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

7,954 stars
Python 2 awesome lists

OptimalScale/LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

8,484 stars
Python 2 awesome lists

Metadata

Language
Python
License
NOASSERTION
Default branch
main
Created
2023-08-16
First commit
2023-09-20
Last pushed
2026-07-15
GitHub updated
2026-07-15
Last synced
2026-07-15 03:16
Stack detected
2026-07-15 03:16
Archived
no

Links and files

GitHub Website

https://nvidia.github.io/TensorRT-LLM

README

Appears in

2