Search every repository hiding inside awesome lists.

The API to search, scrape, and interact with the web at scale. 🔥

Scrapy, a fast high-level web crawling & scraping framework for Python.

The all-in-one AI productivity accelerator. On device and privacy first with no annoying setup or configuration.

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring—all for free or enjoy our SaaS plan!

Python scraper based on AI

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

Lighter web automation with Python

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

Self-hosted webscraper.

Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.

Anti-Detect Browser that passes every bot detection test. Drop-in Playwright replacement.

❤️ Fredy - [F]ind [R]eal [E]state [D]amn Eas[y] - Fredy keeps searching for new apartments, houses, and flats in Germany on platforms like ImmoScout24, Immowelt, Immonet, eBay Kleinanzeigen, and WG-Gesucht and instantly delivers the results to you via Slack, Telegram, Email, Discord or ntfy, so you can focus on the more important things in life ;)

ChatGPT Team(Business) 促销码自动扫描工具 — 批量发现/验证/价格收集，支持 17 国 34 个码，最高折扣 71% | ChatGPT Business promo code scanner — batch discovery, validation, price collection, 34 codes across 17 countries, up to 71% off

Model Context Protocol (MCP) Server for Graphlit Platform

The unix-way web crawler

An unofficial API for Quora.

The open-source execution engine for AI agents. 412 modules, MCP-native, triggers, queue, versioning, metering.

A Python library (and CLI) for Amazon order history, line items, and transactions

Open-source, local-first AI visibility intelligence dashboard. Track your brand across 6 AI models.

The unified web layer for AI agents. Search (8 engines), stealth browse, auth, and act on 24 platforms. One npm install, self-hosted.

Unleash the power of AI with SuperSummarizeAI! Effortlessly extract, condense, and clip content from webpages and YouTube videos using ChatGPT. Turning endless streams of content into digestible summaries.

MCP server for fetching web URLs with token estimation, caching, and intelligent routing. Built for AI agents.