Awesome Llm

Production-ready platform for agentic workflow development.

No description.

A high-throughput and memory-efficient inference and serving engine for LLMs

Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

Making large AI models cheaper, faster and more accessible

DSPy: The framework for programming—not prompting—language models

SGLang is a high-performance serving framework for large language models and multimodal models.

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Ongoing research training transformer models at scale

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

A framework for few-shot evaluation of language models.

Minimal reproduction of DeepSeek R1-Zero

Go ahead and axolotl questions

Access large language models from the command-line

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Efficient Triton Kernels for LLM Training

A PyTorch native platform for training generative AI models

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

A blazing fast inference solution for text embeddings models

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

AdalFlow: The library to build & auto-optimize LLM applications.

Harness LLMs with Multi-Agent Programming

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).

The hub for EleutherAI's work on interpretability and learning dynamics

Benchmarking large language models' complex reasoning ability with chain-of-thought prompting

Minimalistic large language model 3D-parallelism training

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

OpenAGI: When LLM Meets Domain Experts

Mesh TensorFlow: Model Parallelism Made Easier

Build production-ready LLM applications and advanced agents using Python, LangChain, and LangGraph. This is the companion repository for the book on generative AI with LangChain.

A security scanner for your LLM agentic workflows

Make your LLM agent and chat with it simple and fast!