Highlighted
Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome
GitHub projects from awesome lists
Search awesome repositories
Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.
- Repos indexed
- 17,373
- Awesome lists tracked
- 125
- Current results
- 7
7 repos shown
D4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!
GitHub topics
- Updated
- 2026-07-15
- Lists
- 1 list mention
- First commit
- 2024-10-13
- History
- 42 history points
- License
- BSD-3-Clause
- Issues
- 4 open
69,810
stars
- Forks
- 6,915
- Commits
- 1,532 commits
- Star growth, last 7 days
- +750 +1.1%
- Commit velocity, last 7 days
- +17 +1.1%
scrapy/scrapy
Scrapy, a fast high-level web crawling & scraping framework for Python.
GitHub topics
- Updated
- 2026-07-13
- Lists
- 2 list mentions
- First commit
- 2009-05-04
- History
- 28 history points
- License
- BSD-3-Clause
- Issues
- 587 open
63,189
stars
- Forks
- 11,789
- Commits
- 11,204 commits
- Star growth, last 7 days
- +116 +0.2%
- Commit velocity, last 7 days
- +2 0.0%
apify/crawlee-python
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
AI dev
GitHub topics
- Updated
- 2026-07-16
- Lists
- 1 list mention
- First commit
- 2024-01-10
- History
- 25 history points
- License
- Apache-2.0
- Issues
- 80 open
9,330
stars
- Forks
- 777
- Commits
- 1,707 commits
- Star growth, last 7 days
- +33 +0.4%
- Commit velocity, last 7 days
- +26 +1.5%
exa-labs/exa-mcp-server
Exa MCP for web search and web crawling!
Stack
GitHub topics
- Updated
- 2026-07-16
- Lists
- 1 list mention
- First commit
- 2024-11-27
- History
- 7 history points
- License
- MIT
- Issues
- 34 open
4,732
stars
- Forks
- 359
- Commits
- 414 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
webrecorder/browsertrix-crawler
Run a high-fidelity browser-based web archiving crawler in a single Docker container
Stack
GitHub topics
- Updated
- 2026-07-11
- Lists
- 1 list mention
- First commit
- 2020-10-31
- History
- 5 history points
- License
- AGPL-3.0
- Issues
- 137 open
1,079
stars
- Forks
- 147
- Commits
- 696 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
Florents-Tselai/WarcDB
WarcDB: Web crawl data as SQLite databases.
- Updated
- 2024-07-13
- Lists
- 1 list mention
- First commit
- 2022-05-29
- History
- 5 history points
- License
- Apache-2.0
- Issues
- 9 open
406
stars
- Forks
- 10
- Commits
- 73 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
N0taN3rd/Squidwarc
Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head
Stack
GitHub topics
- Updated
- 2020-05-19
- Lists
- 1 list mention
- First commit
- 2017-07-20
- History
- 5 history points
- License
- Apache-2.0
- Issues
- 11 open
178
stars
- Forks
- 25
- Commits
- 119 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
