Awesome Llm

Production-ready platform for agentic workflow development.

A high-throughput and memory-efficient inference and serving engine for LLMs

DSPy: The framework for programming—not prompting—language models

SGLang is a high-performance serving framework for large language models and multimodal models.

Integrate cutting-edge LLM technology quickly and easily into your apps

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

Ongoing research training transformer models at scale

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

A framework for few-shot evaluation of language models.

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Go ahead and axolotl questions

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

A blazing fast inference solution for text embeddings models

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

Harness LLMs with Multi-Agent Programming

The platform for LLM evaluations and AI agent testing

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

Holistic Evaluation of Language Models (HELM) is an open source Python framework created by the Center for Research on Foundation Models (CRFM) at Stanford for holistic, reproducible and transparent evaluation of foundation models, including large language models (LLMs) and multimodal models.

Seamlessly integrate LLMs as Python functions