🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
Appears on
Quick read
Latest capture 2026-08-11 04:36
0 paths
Agent instructions and tool configuration found in this repository.
No config files detected.
60 observed captures since 2026-06-02. Observed captures are shown by default.
Stars from first capture +15021
Observed captures only
All tracked data
Observed snapshots
Observed snapshots
Nearest indexed repositories by embedding similarity.
The context API to search, scrape, and interact with the web at scale. 🔥
🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
The unified web layer for AI agents. Search (8 engines), stealth browse, auth, and act on 24 platforms. One npm install, self-hosted.
MCP server with multi-tier fallback chain for fetching web content as clean markdown
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
Lightpanda: the headless browser designed for AI and automation
