Awesome Opensource Ai

Inspect: A framework for large language model evaluations

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Machine learning metrics for distributed, scalable PyTorch applications.

SDG is a specialized framework designed to generate high-quality structured tabular data.

Python & JS/TS SDK for running AI-generated code/code interpreting in your AI app

Fully open data curation for reasoning models

A Python package to assess and improve fairness of machine learning models.

Feature engineering and selection open-source Python library compatible with sklearn.

An AI-powered agentic red team framework that automates offensive security operations, from reconnaissance to exploitation to post-exploitation, with zero human intervention.

Collect, aggregate, and visualize a data ecosystem's metadata

Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal.

TFX is an end-to-end platform for deploying production ML pipelines

VeOmni: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo

Helios: Real Real-Time Long Video Generation Model

Toward High-Accuracy Open-Source Biomolecular Structure Prediction.

[ICLR & NeurIPS 2025] Repository for Show-o series, One Single Transformer to Unify Multimodal Understanding and Generation.

:robot: MoveIt for ROS 2

[CVPR 2025] Magma: A Foundation Model for Multimodal AI Agents

Manages Unified Access to Generative AI Services built on Envoy Gateway

Scalable toolkit for efficient model reinforcement

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

Fast, Flexible and Portable Structured Generation

Developer Asset Hub for NVIDIA Nemotron — A one-stop resource for training recipes, usage cookbooks, datasets, and full end-to-end reference examples to build with Nemotron models

Interpretability and explainability of data and machine learning models

Large-scale LLM inference engine

Reference implementations of MLPerf® training benchmarks

Synthetic data curation for post-training and structured data extraction

OpenMLDB is an open-source machine learning database that provides a feature platform computing consistent features for training and inference.

Automated Machine Learning on Kubernetes

MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.

MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering

MMaDA - Open-Sourced Multimodal Large Diffusion Language Models (dLLMs with block diffusion, mixed-CoT, unified RL)

Apache Solr open-source search software

Machine learning with dataframes

A fast and lightweight framework for creating decentralized agents with ease.

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

Reference implementations of MLPerf® inference benchmarks

The good ol' Forge WebUI, now updated with new features~

Fast Multimodal LLM on Mobile Devices

Code repo for "WebArena: A Realistic Web Environment for Building Autonomous Agents"

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.

A Data Streaming Library for Efficient Neural Network Training

The open document intelligence platform for builders and hackers - DMS for the agentic world

Newelle - Your Ultimate Virtual Assistant

Interface for OuteTTS models.

NASA Astrobee Robot Software

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.

Scalable and memory-optimized training of diffusion models