Search every repository hiding inside awesome lists.

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.

SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]

#1 Persistent memory for AI coding agents based on real-world benchmarks

AI Agent Framework, the Pydantic way

Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full desktops (macOS, Linux, Windows).

A collection of projects designed to help developers quickly get started with building deployable applications using the Claude API

The LLM Evaluation Framework

Context window optimization for AI coding agents. Sandboxes tool output, 98% reduction. 15 platforms

A format specification for describing a visual identity to coding agents. DESIGN.md gives agents a persistent, structured understanding of a design system.

DeepSeek-native AI coding agent for your terminal. Engineered around prefix-cache stability — leave it running.

AG-UI: the Agent-User Interaction Protocol. Bring Agents into Frontend Applications.

Structured Outputs

Browser Harness | Self-healing harness that enables LLMs to complete any task.

"OpenHarness: Open Agent Harness with a Built-in Personal Agent--Ohmo!"

Fully autonomous & self-evolving research from idea to paper. Chat an Idea. Get a Paper. 🦞

Open Source framework for voice and multimodal conversational AI

Open-source, secure environment with real-world tools for enterprise-grade agents.

Secure, Fast, and Extensible Sandbox runtime for AI agents.

Multi-Agent Harness for Production AI

Visual testing tool for MCP servers

AI Observability & Evaluation

The best agent harness.

Build effective agents using Model Context Protocol and simple workflow patterns

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

From a goal to a task DAG, automatically. TypeScript-native multi-agent orchestration.

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

OpenShell is the safe, private runtime for autonomous AI agents.

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

The first "code-first" agent framework for seamlessly planning and executing data analytics tasks.