Search awesome repositories

A high-throughput and memory-efficient inference and serving engine for LLMs

The open-source AI voice studio. Clone, dictate, create.

SGLang is a high-performance serving framework for large language models and multimodal models.

Build and run Docker containers leveraging NVIDIA GPUs

Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Open3D: A Modern Library for 3D Data Processing

NumPy & SciPy for GPU

📚LeetCUDA: Modern CUDA Learn Notes with PyTorch for Beginners🐑, 200+ CUDA Kernels, Tensor Cores, HGEMM, FA-2 MMA.🎉

NumPy aware dynamic Python compiler using LLVM

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

cuDF - GPU DataFrame Library

Containers for machine learning

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

FlashInfer: Kernel Library for LLM Serving

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

cuML - RAPIDS Machine Learning Library

Fast inference engine for Transformer models

A retargetable MLIR-based machine learning compiler and runtime toolkit.

Train, inspect, edit, automate, and export 3D Gaussian Splatting scenes from a single native application.

Single C file, Realtime CPU/GPU Profiler with Remote Web Viewer

Self-hosted, local only NVR and AI Computer Vision software. With features such as object detection, motion detection, face recognition and more, it gives you the power to keep an eye on your home, office or any other place you want to monitor.

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

PyTorch native quantization and sparsity for training and inference

cuGraph - RAPIDS Graph Analytics Library

Large-scale LLM inference engine

Large-scale LLM inference engine

ThunderSVM: A Fast SVM Library on GPUs and CPUs

Self-host the powerful Chatterbox TTS model. This server offers a user-friendly Web UI, flexible API endpoints (incl. OpenAI compatible), predefined voices, voice cloning, and large audiobook-scale text processing. Runs accelerated on NVIDIA (CUDA), AMD (ROCm), and CPU.

The FFmpeg build script provides an easy way to build a static FFmpeg on OSX and Linux with non-free codecs included.