Awesome Opensource Ai

A high-throughput and memory-efficient inference and serving engine for LLMs

The open-source AI voice studio. Clone, dictate, create.

SGLang is a high-performance serving framework for large language models and multimodal models.

Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

Open3D: A Modern Library for 3D Data Processing

NumPy & SciPy for GPU

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

cuDF - GPU DataFrame Library

Containers for machine learning

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

FlashInfer: Kernel Library for LLM Serving

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

cuML - RAPIDS Machine Learning Library

Fast inference engine for Transformer models

A retargetable MLIR-based machine learning compiler and runtime toolkit.

Train, inspect, edit, automate, and export 3D Gaussian Splatting scenes from a single native application.

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

PyTorch native quantization and sparsity for training and inference

cuGraph - RAPIDS Graph Analytics Library

Large-scale LLM inference engine