Awesome Opensource Ai

Python bindings for llama.cpp

cuDF - GPU DataFrame Library

High-speed Large Language Model Serving for Local Deployment

Easy-to-use image segmentation library with awesome pre-trained model zoo, supporting wide-range of practical tasks in Semantic Segmentation, Interactive Segmentation, Panoptic Segmentation, Image Matting, 3D Segmentation, etc.

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀

Accessible large language models via k-bit quantization for PyTorch.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

An app that brings language models directly to your phone.

Flax is a neural network library for JAX that is designed for flexibility.

lightweight, standalone C++ inference engine for Google's Gemma models.

the portable Python dataframe library

A query and indexing engine for Redis, providing secondary indexing, full-text search, vector similarity search and aggregations.

LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

A treasure chest for visual classification and recognition powered by PaddlePaddle

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.

OpenSpiel is a collection of environments and algorithms for research in general reinforcement learning and search/planning in games.

Header-only C++/python library for fast approximate nearest neighbors

cuML - RAPIDS Machine Learning Library

A distributed approximate nearest neighborhood search (ANN) library which provides a high quality vector index build, search and distributed online serving toolkits for large scale vector search scenario.

An Engine-Agnostic Deep Learning Framework in Java

On-device AI across mobile, embedded and edge for PyTorch

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

Fast inference engine for Transformer models

Pre-trained Deep Learning models and demos (high quality and extremely fast)

RTAB-Map library and standalone application

A retargetable MLIR-based machine learning compiler and runtime toolkit.

A flexible, high-performance 3D simulator for Embodied AI research.

Non-Metric Space Library (NMSLIB): An efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.

Train, inspect, edit, automate, and export 3D Gaussian Splatting scenes from a single native application.

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Open source, local, and self-hosted Amazon Echo/Google Home competitive Voice Assistant alternative

PyTorch native quantization and sparsity for training and inference

llama.cpp fork with additional SOTA quants and improved performance

Data manipulation and transformation for audio signal processing, powered by PyTorch

The pretty much "official" DSPy framework for Typescript

nanoflann: a C++11 header-only library for Nearest Neighbor (NN) search with KD-trees

cuGraph - RAPIDS Graph Analytics Library

Production-grade 3D gaussian splatting with CPU/GPU support for Windows, Mac and Linux 🚀

:robot: MoveIt for ROS 2

Fast, Flexible and Portable Structured Generation

Large-scale LLM inference engine

OpenMLDB is an open-source machine learning database that provides a feature platform computing consistent features for training and inference.

Reference implementations of MLPerf® inference benchmarks

🛰️ An approximate nearest-neighbor search library for Python and Java with a focus on ease of use, simplicity, and deployability.

Fast Multimodal LLM on Mobile Devices

The good ol' Forge WebUI, now updated with new features~

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.