Awesome Opensource Ai

Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

A high-throughput and memory-efficient inference and serving engine for LLMs

Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek-V4, MiniMax-H3, Gemma 4, FLUX and more.

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

aider is AI pair programming in your terminal

LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.

AI Agent Assistant & development framework that integrates lots of IM platforms, LLMs, plugins and AI feature, and can be your openclaw alternative. ✨

SOTA Open Source TTS

SGLang is a high-performance serving framework for large language models and multimodal models.

AirLLM 70B inference with single 4GB GPU

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

LangChain4j is an idiomatic, open-source Java library for building LLM-powered applications on the JVM. It offers a unified API over popular LLM providers and vector stores, and makes implementing tool calling (including MCP support), agents and RAG easy. It integrates seamlessly with enterprise Java frameworks like Quarkus and Spring Boot.

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Low-code framework for building custom LLMs, neural networks, and other AI models

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

High-speed Large Language Model Serving for Local Deployment

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Easily fine-tune, evaluate and deploy Qwen, Gemma, or any open weight LLM!

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Tools for merging pretrained large language models.

Efficient Triton Kernels for LLM Training

A model-driven approach to building AI agents in just a few lines of code.

An automated document analyzer for Paperless-ngx using OpenAI API, Ollama, Deepseek-r1, Azure and all OpenAI API compatible Services to automatically analyze and tag your documents.

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc

H2O LLM Studio - a framework and no-code GUI for fine-tuning LLMs. Documentation: https://docs.h2o.ai/h2o-llmstudio/

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

Harness LLMs with Multi-Agent Programming

PyTorch native quantization for training and inference

Maid is a free and open source application for interfacing with llama.cpp models locally, and with Anthropic, DeepSeek, Ollama, Mistral and OpenAI models remotely.

Vim plugin for LLM-assisted code/text completion

The leading open-source AI copilot for JetBrains. Connect to any model in any environment, and customize your coding experience in any way you like.

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

Fast Multimodal LLM on Mobile Devices

Interface for OuteTTS models.

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.