Awesome Opensource Ai

TencentDB Agent Memory delivers fully local long-term memory for AI Agents via a 4-tier progressive pipeline, with zero external API dependencies.

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Build effective agents using Model Context Protocol and simple workflow patterns

Accessible large language models via k-bit quantization for PyTorch.

Coding Agent Harness

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

⚙️🦀 Build modular and scalable LLM Applications in Rust

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

Fast, flexible LLM inference

🔥 Open Source Browser API for AI Agents & Apps. Steel Browser is a batteries-included browser sandbox that lets you automate the web without worrying about infrastructure.

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

Tools for merging pretrained large language models.

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

Adding guardrails to large language models.

Dynamic, resilient AI orchestration. Coordinate data, models, and compute as you build AI workflows.

✨ AI Coding, Vim Style

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

Superagent protects your AI applications against prompt injections, data leaks, and harmful outputs. Embed safety directly into your app and prove compliance to your customers.

TypeScript AI agent orchestration framework with dynamic workflows. Describe the goal, not the graph: a coordinator plans the task DAG at runtime and runs it on any LLM (Claude, ChatGPT, Gemini, DeepSeek, or local models).

Julep — durable, composable AI agents. Flows that crash and resume, retry safely, and explain every step.

Open-source framework for building AI-powered apps in JavaScript, Go, and Python, built and used in production by Google

an ambient intelligence library

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

A model-driven approach to building AI agents in just a few lines of code.

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines

Robust recipes to align language models with human and AI preferences

Chronos: Pretrained Models for Time Series Forecasting

🐢 Open-Source Evaluation & Testing library for LLM Agents

⚡️next-generation personal AI assistant powered by LLM, RAG and agent loops, supporting computer-use, browser-use and coding agent, demo: https://demo.openagentai.org

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

A Next-Generation Training Engine Built for Ultra-Large MoE Models

Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/

A blazing fast inference solution for text embeddings models

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

A fast multimodal LLM for real-time voice

LLM training code for Databricks foundation models

Your agent in your terminal, equipped with local tools: writes code, uses the terminal, browses the web. Make your own persistent autonomous agent on top!

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

One delightful Ruby framework for every major AI provider. Build AI agents, chatbots, RAG apps, and multimodal workflows in beautiful, expressive code.

AdalFlow: The library to build & auto-optimize LLM applications.

Optimizing inference proxy for LLMs

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

A full-stack AI Red Teaming platform securing AI ecosystems via OpenClaw Security Scan, Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

Harness LLMs with Multi-Agent Programming

OpenAgents - AI Agent Networks for Open Collaboration

A system for agentic LLM-powered data processing and ETL

Achieve state of the art inference performance with modern accelerators on Kubernetes