Awesome Opensource Ai

Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.

Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

An Open Standard for lineage metadata collection

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Machine learning metrics for distributed, scalable PyTorch applications.

SDG is a specialized framework designed to generate high-quality structured tabular data.

Data Contracts engine for the modern data stack. https://www.soda.io

Inspect: A framework for large language model evaluations

Fully open data curation for reasoning models

A Python package to assess and improve fairness of machine learning models.

Feature engineering and selection open-source Python library compatible with sklearn.

cuGraph - RAPIDS Graph Analytics Library

TFX is an end-to-end platform for deploying production ML pipelines

Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal.

VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo

Enforce the output format (JSON Schema, Regex etc) of a language model

[CVPR 2025] Magma: A Foundation Model for Multimodal AI Agents

Exploratory analysis of Bayesian models with Python

Scalable toolkit for efficient model reinforcement

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

Fast, Flexible and Portable Structured Generation

Anti-Detect Browser that passes every bot detection test. Drop-in Playwright replacement.

Developer Asset Hub for NVIDIA Nemotron — A one-stop resource for training recipes, usage cookbooks, datasets, and full end-to-end reference examples to build with Nemotron models

Automated Machine Learning on Kubernetes

MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.

Lightweight and extensible compatibility layer between dataframe libraries!

Draw datasets from within Python notebooks.

Agentic RL Training at Scale

A fast and lightweight framework for creating decentralized agents with ease.

Machine learning with dataframes

MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering

A lightweight, low-dependency, unified API to use all common reranking and cross-encoder models.

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

Reference implementations of MLPerf® inference benchmarks

🛰️ An approximate nearest-neighbor search library for Python and Java with a focus on ease of use, simplicity, and deployability.

Fast Multimodal LLM on Mobile Devices

A Data Streaming Library for Efficient Neural Network Training

The good ol' Forge WebUI, now updated with new features~

The LLM Anti-Framework

Training Sparse Autoencoders on Language Models

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.

Scalable and memory-optimized training of diffusion models

The official API server for Exllama. OAI compatible, lightweight, and fast.

PinchBench is a benchmarking system for evaluating LLM models as OpenClaw coding agents. Made with 🦀 by the humans at https://kilo.ai

🌎💪 BrowserGym, a Gym environment for web task automation

Trained models & code to predict toxic comments on all 3 Jigsaw Toxic Comment Challenges. Built using ⚡ Pytorch Lightning and 🤗 Transformers. For access to our API, please email us at contact@unitary.ai.

XAI - An eXplainability toolbox for machine learning

LiveBench: A Challenging, Contamination-Free LLM Benchmark