Awesome Llm

A high-throughput and memory-efficient inference and serving engine for LLMs

Unsloth Studio is a web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

Making large AI models cheaper, faster and more accessible

Langchain-Chatchat（原Langchain-ChatGLM）基于 Langchain 与 ChatGLM, Qwen 与 Llama 等语言模型的 RAG 与 Agent 应用 | Langchain-Chatchat (formerly langchain-ChatGLM), local knowledge based LLM (like ChatGLM, Qwen and Llama) RAG and Agent app with langchain

DSPy: The framework for programming—not prompting—language models

SGLang is a high-performance serving framework for large language models and multimodal models.

Integrate cutting-edge LLM technology quickly and easily into your apps

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

A framework for few-shot evaluation of language models.

Minimal reproduction of DeepSeek R1-Zero

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Go ahead and axolotl questions

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.

A PyTorch native platform for training generative AI models

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

A blazing fast inference solution for text embeddings models

AdalFlow: The library to build & auto-optimize LLM applications.

Harness LLMs with Multi-Agent Programming

The platform for LLM evaluations and AI agent testing

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Seamlessly integrate LLMs as Python functions

OpenAGI: When LLM Meets Domain Experts

Mesh TensorFlow: Model Parallelism Made Easier

A security scanner for your LLM agentic workflows