Sign in
← Back to search

alibaba/rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

ReactSpring BootVite #gpt#inference#llama#llm#llm-serving#llmops#model-serving
Sign in to like alibaba/rtp-llm Open GitHub
Stars
1,172
Forks
204
Commits
4589
Language
Cuda
Awesome lists
1

Similar repositories

jd-opensource/xllm

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.

1300 stars
C++ 1 awesome list

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

81308 stars
Python 3 awesome lists

OptimalScale/LMFlow

An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

8486 stars
Python 2 awesome lists

llm-d/llm-d

Achieve state of the art inference performance with modern accelerators on Kubernetes

3250 stars
Shell 1 awesome list

jianzhnie/LLamaTuner

Easy and Efficient Finetuning LLMs. (Supported LLama, LLama2, LLama3, Qwen, Baichuan, GLM , Falcon) 大模型高效量化训练+部署.

620 stars
Python 1 awesome list

RUCAIBox/R1-Searcher

R1-searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning

713 stars
Python 1 awesome list

Tracked growth

2 captures since 2026-05-25

Latest capture 2026-06-03 03:05

Stars history

Total stars

Commits history

Default branch commits

Detected stack

Frameworks and tools

  • React · frontend framework · high confidence
  • Spring Boot · web framework · high confidence
  • Vite · build tool · high confidence
Bundler CMake Composer Dart Pub .NET SDK Maven npm pip

Dependency files

  • docs/requirements.txt · python · 21 dependencies
  • 3rdparty/protobuf/composer.json · php · 1 dependencies
  • 3rdparty/trt_fused_multihead_attention/CMakeLists.txt · c-cpp · 0 dependencies
  • docs/index/package.json · javascript · 7 dependencies
  • rtp_llm/flexlb/pom.xml · java · 68 dependencies
  • 3rdparty/protobuf/global.json · dotnet · 0 dependencies
  • 3rdparty/protobuf/cmake/CMakeLists.txt · c-cpp · 2 dependencies
  • 3rdparty/protobuf/examples/CMakeLists.txt · c-cpp · 1 dependencies
  • 3rdparty/protobuf/examples/pubspec.yaml · dart · 1 dependencies
  • 3rdparty/protobuf/java/pom.xml · java · 12 dependencies

Metadata

  • Created: 2023-12-27
  • First commit: 2023-12-27
  • Last pushed: 2026-06-03
  • Archived: no
  • Stack detected: 2026-06-03 03:05
  • License: Apache-2.0

AI development signals

AI agent config detected

2 config paths 2 files 0 directories
Claude Code 2

Key config paths

  • file rtp_llm/flexlb/CLAUDE.md
  • file rtp_llm/flexlb/flexlb-sync/CLAUDE.md