aniketmaurya/llm-inference
Large Language Model (LLM) Inference API and Chatbot
Repository profile
Seamlessly integrate LLMs into scikit-learn.
Repository updates
Get generated BeastByteAI/scikit-llm development summaries by email, or follow the weekly and monthly RSS feeds.
Sign in to subscribe by email. RSS feeds are public.
Sign in to subscribeTracked growth, recent movement, and commit velocity from stored repository snapshots.
Latest capture 2026-07-02 03:11
5 captures since 2026-05-27
Stars from baseline +27
All tracked data
Frameworks, package managers, ecosystems, and dependency manifests found during catalog scans.
Scanned 2026-07-02 03:11
Searchable topics, generated tags, and stack labels that explain where this repository fits.
Agent instructions and tool configuration paths found in the repository tree.
Nearest indexed repositories by embedding similarity.
Large Language Model (LLM) Inference API and Chatbot
A high-throughput and memory-efficient inference and serving engine for LLMs
The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.
Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A
Easy and Efficient Finetuning LLMs. (Supported LLama, LLama2, LLama3, Qwen, Baichuan, GLM , Falcon) 大模型高效量化训练+部署.
LlamaIndex is the leading document agent and OCR platform