Awesome

GitHub projects from awesome lists

Search awesome repositories

Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.

Continue with GitHub Browse awesome lists Request a list

Repos indexed: 18,423
Awesome lists tracked: 132
Current results: 46

Find repositories

Start broad, then narrow by ecosystem, freshness, health, and growth.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Awesome list

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

46 repos shown

Topic: inference

Browse

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

pytorch/ao

PyTorch native quantization for training and inference

AI dev

Stack

Python CMake PEP 517 pip

GitHub topics

#brrr #cuda #dtypes #float8 #inference #llama

Updated: 2026-09-15
Lists: 1 list mention
First commit: 2023-11-03
History: 108 history points
License: NOASSERTION
Issues: 789 open

2,977

stars

Forks: 621
Commits: 2,796 commits
Star growth, last 7 days: +10 +0.3%
Commit velocity, last 7 days: +3 +0.1%

Website GitHub

vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

AI dev

Stack

C++ FastAPI pytest CMake PEP 517 pip

GitHub topics

#ascend #inference #llm #llm-serving #llmops #mlops

Updated: 2026-09-11
Lists: 2 list mentions
First commit: 2025-01-29
History: 1 history point
License: Apache-2.0
Issues: 3,178 open

2,800

stars

Forks: 2,234
Commits: 5,179 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

roboflow/inference

Turn any computer or edge device into a command center for your computer vision projects.

AI dev

Stack

Python Celery FastAPI Flask Next.js npm PEP 517 pip

GitHub topics

#agents #classification #computer-vision #deployment #docker #inference

Updated: 2026-09-01
Lists: 1 list mention
First commit: 2023-07-31
History: 2 history points
License: NOASSERTION
Issues: 156 open

2,431

stars

Forks: 312
Commits: 11,480 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

adithya-s-k/AI-Engineering.academy

Mastering Applied AI, One Concept at a Time

Stack

Jupyter Notebook FastAPI LangChain LlamaIndex React npm pip

GitHub topics

#fine-tuning #finetuning #finetuning-llms #inference #large-language-models #llm

Updated: 2026-02-27
Lists: 1 list mention
First commit: 2023-10-05
History: 1 history point
License: MIT
Issues: 7 open

2,378

stars

Forks: 277
Commits: 429 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

dstackai/dstack

Vendor-agnostic orchestration for training, inference and agentic workloads across NVIDIA, AMD, TPU, and Tenstorrent on clouds, Kubernetes, and bare metal.

AI dev

Stack

Python FastAPI gRPC Go pytest React Go modules npm PEP 517

GitHub topics

#agent-skills #agentic-orchestration #amd #cloud #containers #docker

Updated: 2026-09-11
Lists: 1 list mention
First commit: 2022-01-13
History: 7 history points
License: MPL-2.0
Issues: 69 open

2,244

stars

Forks: 261
Commits: 3,852 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

envoyproxy/ai-gateway

Manages Unified Access to Generative AI Services built on Envoy Gateway

AI dev

Stack

Go Cobra FastAPI gRPC Go React Go modules npm pip

GitHub topics

#ai #ai-gateway #api-gateway #cncf #inference #kubernetes

Updated: 2026-09-07
Lists: 1 list mention
First commit: 2024-10-21
History: 7 history points
License: Apache-2.0
Issues: 282 open

2,003

stars

Forks: 363
Commits: 1,570 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

Human-Agent-Society/reef

Continual learning infra for self-improving agents

AI dev

Stack

Python Next.js pytest React npm PEP 517 uv

GitHub topics

#agent-infrastructure #ai-agents #continual-learning #inference #llm #llm-training

Updated: 2026-09-15
Lists: 1 list mention
First commit: 2026-08-31
History: 1 history point
License: Apache-2.0
Issues: 58 open

1,698

stars

Forks: 108
Commits: 286 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

xLLM-AI/xllm

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.

AI dev

Stack

C++ Astro Vite CMake npm PEP 517

GitHub topics

#deepseek #glm #inference #inference-engine #large-language-models #llm-inference

Updated: 2026-09-15
Lists: 0 list mentions
First commit: 2025-08-13
History: 56 history points
License: Apache-2.0
Issues: 212 open

1,571

stars

Forks: 298
Commits: 1,534 commits
Star growth, last 7 days: +4 +0.3%
Commit velocity, last 7 days: +32 +2.1%

Website GitHub

jd-opensource/xllm

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.

AI dev

Stack

C++ CMake PEP 517 pip

GitHub topics

#deepseek #glm #inference #inference-engine #large-language-models #llm-inference

Updated: 2026-06-27
Lists: 1 list mention
First commit: 2025-08-13
History: 4 history points
License: Apache-2.0
Issues: 162 open

1,370

stars

Forks: 240
Commits: 1,115 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

alibaba/rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

AI dev

Stack

Cuda React Spring Boot Vite Bundler CMake Composer

GitHub topics

#gpt #inference #llama #llm #llm-serving #llmops

Updated: 2026-09-07
Lists: 1 list mention
First commit: 2023-12-27
History: 7 history points
License: Apache-2.0
Issues: 216 open

1,326

stars

Forks: 270
Commits: 4,752 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

kaito-project/aikit

🏗️ Fine-tune, build, and deploy open-source LLMs easily!

AI dev

Stack

Go gRPC Go React Go modules npm Yarn

GitHub topics

#ai #buildkit #chatgpt #docker #fine-tuning #finetuning

Updated: 2026-08-11
Lists: 2 list mentions
First commit: 2023-09-20
History: 6 history points
License: MIT
Issues: 35 open

535

stars

Forks: 57
Commits: 736 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

defilantech/LLMKube

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

AI dev

Stack

Go Cobra gRPC Go Go modules

GitHub topics

#ai #apple-silicon #autoscaling #edge-computing #gguf #gpu

Updated: 2026-08-23
Lists: 1 list mention
First commit: 2025-11-17
History: 7 history points
License: Apache-2.0
Issues: 109 open

196

stars

Forks: 30
Commits: 976 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

pyadav/gateway

[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications

Stack

Go Cobra gRPC Go Next.js React Bun Go modules npm

GitHub topics

#ai #aistudio #api #gateway #generative-ai #grpc

Updated: 2024-04-08
Lists: 0 list mentions
First commit: 2024-01-18
History: 4 history points
License: Apache-2.0
Issues: 10 open

161

stars

Forks: 16
Commits: 141 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

missingstudio/gateway

[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications

Stack

Go Bun Go modules npm

GitHub topics

#ai #aistudio #api #gateway #generative-ai #grpc

Updated: 2024-04-08
Lists: 0 list mentions
First commit: 2024-01-18
History: 41 history points
License: Apache-2.0
Issues: 10 open

160

stars

Forks: 17
Commits: 141 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

CogitatorTech/infera

A DuckDB extension for in-database inference

AI dev

Stack

Rust pytest Cargo CMake PEP 517

GitHub topics

#duckdb #duckdb-extension #inference #machine-learning #onnx #rust

Updated: 2026-07-22
Lists: 1 list mention
First commit: 2025-09-26
History: 2 history points
License: Apache-2.0
Issues: 5 open

136

stars

Forks: 6
Commits: 40 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

cognisoc/zigllm

Learn how LLMs work by building one in Zig -- from tensors to text generation.

Stack

Zig pip

GitHub topics

#cognisoc #educational #from-scratch #inference #learn-by-doing #llm

Updated: 2026-07-02
Lists: 1 list mention
First commit: 2026-03-15
History: 5 history points
License: MIT
Issues: 1 open

stars

Forks: 0
Commits: 14 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

Search awesome repositories

Find repositories

Put your repository first

How it works

Pricing

How it works

Pricing