TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
A unified library of SOTA model optimization techniques like quantization, pruning, distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
Achieve state of the art inference performance with modern accelerators on Kubernetes
TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
No description.
Universal LLM Deployment Engine with ML Compilation
1 capture since 2026-05-25
AI agent config detected
Key config paths
.claude
.coderabbit.yaml
.codex/AGENTS.md
AGENTS.md
CLAUDE.md
tensorrt_llm/runtime/kv_cache_manager_v2/AGENTS.md
1 more config path detected.
.claude
.claude/agent-tests
.claude/agent-tests/perf-test-sync
.claude/agent-tests/perf-test-sync/build_prompt.py
.claude/agent-tests/perf-test-sync/promptfooconfig.yaml
.claude/agent-tests/perf-test-sync/render_report.py
.claude/agent-tests/perf-test-sync/run.sh
.claude/agents
.claude/agents/ad-conf-check-update.md
.claude/agents/ad-debug-agent.md
.claude/agents/ad-onboard-reviewer.md
.claude/agents/ad-run-agent.md
.claude/agents/exec-compile-specialist.md
.claude/agents/kernel-cuda-specialist.md
.claude/agents/kernel-cute-specialist.md
.claude/agents/kernel-tileir-specialist.md
.claude/agents/kernel-triton-specialist.md
.claude/agents/perf-profiling-specialist.md
.claude/agents/perf-test-sync.md
.claude/agents/perf-torch-cuda-graph-specialist.md
.claude/README.md
.claude/skills
.claude/skills/ad-accuracy-debug
.claude/skills/ad-accuracy-debug/SKILL.md
Showing the first 24 paths. 176 more detected.