Awesome

GitHub projects from awesome lists

Search awesome repositories

Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.

Continue with GitHub Browse awesome lists Request a list

Repos indexed: 18,423
Awesome lists tracked: 132
Current results: 15

Find repositories

Start broad, then narrow by ecosystem, freshness, health, and growth.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Awesome list

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

15 repos shown

Topic: scraper

Browse

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

firecrawl/firecrawl

The context API to search, scrape, and interact with the web at scale. 🔥

AI dev

Stack

TypeScript Astro Express Next.js pytest Bundler Cargo Composer

GitHub topics

#ai #ai-agents #ai-crawler #ai-scraping #ai-search #crawler

Updated: 2026-09-16
Lists: 2 list mentions
First commit: 2024-04-15
History: 146 history points
License: AGPL-3.0
Issues: 626 open

180,946

stars

Forks: 9,811
Commits: 6,303 commits
Star growth, last 7 days: +2,482 +1.4%
Commit velocity, last 7 days: +21 +0.3%

Website GitHub

huginn/huginn

Create agents that monitor and act on your behalf. Your agents are standing by!

Stack

Ruby Ruby on Rails Svelte Bundler npm

GitHub topics

#agent #automation #feed #feedgenerator #huginn #monitoring

Updated: 2026-08-15
Lists: 3 list mentions
First commit: 2013-03-10
History: 8 history points
License: MIT
Issues: 696 open

49,820

stars

Forks: 4,291
Commits: 4,112 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

iawia002/lux

👾 Fast and simple video download library and CLI tool written in Go

Stack

Go Go modules

GitHub topics

#bilibili #crawler #download #downloader #go #golang

Updated: 2026-03-29
Lists: 1 list mention
First commit: 2018-02-06
History: 4 history points
License: MIT
Issues: 545 open

31,584

stars

Forks: 3,315
Commits: 869 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

AI dev

Stack

TypeScript Express React npm pnpm Yarn

GitHub topics

#apify #automation #crawler #crawling #headless #headless-chrome

Updated: 2026-09-04
Lists: 1 list mention
First commit: 2016-08-26
History: 1 history point
License: Apache-2.0
Issues: 146 open

25,646

stars

Forks: 1,658
Commits: 5,689 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

apify/crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.

AI dev

Stack

Python pytest React npm PEP 517 pip

GitHub topics

#apify #automation #beautifulsoup #crawler #crawling #headless

Updated: 2026-09-10
Lists: 2 list mentions
First commit: 2024-01-10
History: 43 history points
License: Apache-2.0
Issues: 96 open

9,508

stars

Forks: 805
Commits: 1,836 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

mishushakov/llm-scraper

Turn any webpage into structured data using LLMs

Stack

TypeScript Vite npm

GitHub topics

#ai #artificial-intelligence #browser #browser-automation #gpt #gpt-4

Updated: 2026-08-02
Lists: 2 list mentions
First commit: 2024-04-20
History: 1 history point
License: MIT
Issues: 7 open

6,915

stars

Forks: 453
Commits: 108 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

d60/twikit

Stack

Python pip

GitHub topics

#bot #client #python #python-web-scraper #python3 #scrape

Updated: 2026-03-10
Lists: 0 list mentions
First commit: 2024-12-01
History: 43 history points
License: MIT
Issues: 161 open

4,609

stars

Forks: 561
Commits: 39 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

myreader-io/myGPTReader

A community-driven way to read and chat with AI bots - powered by chatGPT.

Stack

Python Flask LangChain LlamaIndex React npm pip

GitHub topics

#ai #chatgpt #crawler #daily-news #embedding #gpt-35-turbo

Updated: 2026-02-05
Lists: 1 list mention
First commit: 2023-03-08
History: 7 history points
License: MIT
Issues: 15 open

4,419

stars

Forks: 438
Commits: 305 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

itsOwen/CyberScraper-2077

A Powerful web scraper powered by LLM | OpenAI, Gemini & Ollama

Stack

Python LangChain Streamlit pip

GitHub topics

#ai-scraping #gemini-api #llm #llm-scraper #openai #scraper

Updated: 2026-08-20
Lists: 1 list mention
First commit: 2024-08-17
History: 1 history point
License: MIT
Issues: 3 open

3,252

stars

Forks: 356
Commits: 107 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

felipecsl/wombat

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Stack

Ruby Bundler

GitHub topics

#crawler #dsl #ruby #scraper

Updated: 2026-04-07
Lists: 1 list mention
First commit: 2011-11-14
History: 5 history points
License: MIT
Issues: 24 open

1,361

stars

Forks: 128
Commits: 300 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

0x676e67/wreq

An ergonomic, privacy-aware Rust HTTP Client

Stack

Rust Cargo

GitHub topics

#akamai #censorship-resistant #client #crawler #fingerprint #http

Updated: 2026-09-02
Lists: 1 list mention
First commit: 2016-07-01
History: 7 history points
License: Apache-2.0
Issues: 20 open

1,009

stars

Forks: 120
Commits: 2,909 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

postmodern/spidr

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Stack

Ruby Sinatra Bundler

GitHub topics

#crawler #ruby #scraper #spider #spider-links #web

Updated: 2026-01-12
Lists: 1 list mention
First commit: 2008-05-19
History: 5 history points
License: MIT
Issues: 16 open

835

stars

Forks: 106
Commits: 898 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

openzim/zimit

Make a ZIM file from any Web site and surf offline!

Stack

Python pytest PEP 517

GitHub topics

#docker #scraper #webscraping #zim

Updated: 2026-07-31
Lists: 1 list mention
First commit: 2016-01-09
History: 6 history points
License: GPL-3.0
Issues: 82 open

827

stars

Forks: 49
Commits: 518 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

Cybrarist/Discount-Bandit

A Multi-User Selfhosted price tracker for Amazon, Aliexpress, ebay and many more along with custom stores. Get notified when price matches one or more criterias set by the user

Stack

PHP Laravel Tailwind CSS Vite Composer npm

GitHub topics

#amazon-price-tracker #automation #ebay #ecommerce #filamentphp #laravel

Updated: 2026-06-01
Lists: 1 list mention
First commit: 2023-07-06
History: 7 history points
License: Unknown
Issues: 14 open

733

stars

Forks: 42
Commits: 221 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

geeks-of-data/knowledge-gpt

Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.

Stack

Python FastAPI pip

GitHub topics

#context #embedding #embedding-vectors #gpt #gpt3-turbo #gpt4

Updated: 2023-04-25
Lists: 1 list mention
First commit: 2023-03-11
History: 6 history points
License: MIT
Issues: 8 open

291

stars

Forks: 52
Commits: 163 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

Search awesome repositories

Find repositories

Put your repository first

How it works

Pricing

How it works

Pricing