Search awesome repositories

A command-line benchmarking tool

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

SWE-bench: Can Language Models Resolve Real-world Github Issues?

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

A series of large language models developed by Baichuan Intelligent Technology

⚡FlashRAG: A Python Toolkit for Efficient RAG Research (WWW2025 Resource)

MTEB: State-of-the-art evaluation of embeddings across languages and modalities

A 13B large language model developed by Baichuan Intelligent Technology

benchmark tooling that loves you ❤️

Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024

Reference implementations of MLPerf® training benchmarks

Reference implementations of MLPerf® inference benchmarks

τ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

Microbenchmarking app for Swift with nice log-log plots

A High Performance HTTP Server for Ruby

🐰 Bencher - Continuous Benchmarking

DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents

An agent benchmark with tasks in a simulated software company.

[ICLR'25] BigCodeBench: Benchmarking Code Generation Towards AGI

Open-source benchmark for browser AI agents on daily tasks.

Ultra-fast websocket client and server for asyncio

📊 zig benchmark

[NeurIPS'25 D&B] Mind2Web-2 Benchmark: Evaluating Agentic Search with Agent-as-a-Judge

Benchmark, evaluate, and optimize skills to ensure reliable performance across all LLMs

Wide NoSQL benchmark for RocksDB, LevelDB, Redis, WiredTiger and MongoDB extending the Yahoo Cloud Serving Benchmark

Benchmark the RTP video streaming performance of various libraries

BenchClaw — Multi-dimensional AI agent evaluation with 17-judge AI Tribunal, 10 scoring dimensions, radar charts, and deception detection. Benchmark any LLM agent.

A microbenchmark support library (uses zig build)