Pyx-Corp/spectrawl
The unified web layer for AI agents. Search (8 engines), stealth browse, auth, and act on 24 platforms. One npm install, self-hosted.
24 stars
JavaScript
1 awesome list
D4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
- Stars
- 58,415
- Forks
- 5,660
- Commits
- 1447
- Language
- Python
- Awesome lists
- 0
Similar repositories
unclecode/crawl4ai
🚀🤖 Crawl4AI: Open-source LLM Friendly Web Crawler & Scraper. Don't be shy, join here: https://discord.gg/jP8KfhDhyN
67581 stars
Python
2 awesome lists
AIMLPM/markcrawl
Fast Python web crawler for RAG and AI ingestion. Extracts clean Markdown from any site for LLMs and vector stores.
2 stars
Python
1 awesome list
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
9134 stars
Python
1 awesome list
goodreasonai/ScrapeServ
A self-hosted API that takes a URL and returns a file with browser screenshots.
1185 stars
Python
0 awesome lists
firecrawl/firecrawl
The API to search, scrape, and interact with the web at scale. 🔥
127429 stars
TypeScript
1 awesome list
Tracked growth
1 capture since 2026-06-02
Latest capture 2026-06-02 07:11
Stars history
Total starsCommits history
Default branch commitsDetected stack
Frameworks and tools
- pytest · test framework · high confidence
PEP 517
pip
Dependency files
-
pyproject.toml· python · 21 dependencies -
setup.cfg· python · 0 dependencies -
docs/requirements.txt· python · 8 dependencies -
tests/requirements.txt· python · 8 dependencies
Metadata
- Created: 2024-10-13
- First commit: 2024-10-13
- Last pushed: 2026-06-01
- Website: https://scrapling.readthedocs.io/en/latest/
- Archived: no
- Stack detected: 2026-06-02 07:11
- License: BSD-3-Clause
AI development signals
No AI development config files detected.
Appears in
- No awesome list links recorded.