Search every repository hiding inside awesome lists.

A high-throughput and memory-efficient inference and serving engine for LLMs

The open-source AI voice studio. Clone, dictate, create.

SGLang is a high-performance serving framework for large language models and multimodal models.

Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Open3D: A Modern Library for 3D Data Processing

📚LeetCUDA: Modern CUDA Learn Notes with PyTorch for Beginners🐑, 200+ CUDA Kernels, Tensor Cores, HGEMM, FA-2 MMA.🎉

NumPy aware dynamic Python compiler using LLVM

NumPy & SciPy for GPU

cuDF - GPU DataFrame Library

Containers for machine learning

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Supercharge Your LLM with the Fastest KV Cache Layer

FlashInfer: Kernel Library for LLM Serving

cuML - RAPIDS Machine Learning Library

A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI model deployment.

Fast inference engine for Transformer models

A retargetable MLIR-based machine learning compiler and runtime toolkit.

Self-hosted, local only NVR and AI Computer Vision software. With features such as object detection, motion detection, face recognition and more, it gives you the power to keep an eye on your home, office or any other place you want to monitor.

Train, inspect, edit, automate, and export 3D Gaussian Splatting scenes from a single native application.

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

PyTorch native quantization and sparsity for training and inference

cuGraph - RAPIDS Graph Analytics Library

Large-scale LLM inference engine

ThunderSVM: A Fast SVM Library on GPUs and CPUs

Self-host the powerful Chatterbox TTS model. This server offers a user-friendly Web UI, flexible API endpoints (incl. OpenAI compatible), predefined voices, voice cloning, and large audiobook-scale text processing. Runs accelerated on NVIDIA (CUDA), AMD (ROCm), and CPU.

ThunderGBM: Fast GBDTs and Random Forests on GPUs

A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

Optimized vLLM deployment for NVIDIA Blackwell (RTX 5090) on Linux Kernel 6.14. Resolves SM_120 kernel incompatibilities, P2P deadlocks, and memory fragmentation for high-performance LLM inference.