Awesome

GitHub projects from awesome lists

Search awesome repositories

Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.

Continue with GitHub Browse awesome lists Request a list

Repos indexed: 17,379
Awesome lists tracked: 125
Current results: 103

Find repositories

Start broad, then narrow by ecosystem, freshness, health, and growth.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Awesome list

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

103 repos shown

List: awesome-web-archiving

Browse

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

commoncrawl/whirlwind-python

A whirlwind tour of Common Crawl's data using Python

Stack

Python pip

GitHub topics

#archive #python #tutorial #warc

Updated: 2026-06-15
Lists: 1 list mention
First commit: 2024-06-23
History: 5 history points
License: Apache-2.0
Issues: 0 open

stars

Forks: 9
Commits: 41 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

PromyLOPh/crocoite

Web archiving using Google Chrome

Archived

Stack

Python pytest pip

GitHub topics

#archiving #chrome-browser #devtools #warc

Updated: 2019-12-30
Lists: 1 list mention
First commit: 2017-11-17
History: 5 history points
License: MIT
Issues: 12 open

stars

Forks: 7
Commits: 282 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

nla/outbackcdx

Web archive index server based on RocksDB

Stack

Java Maven

GitHub topics

#wayback #web-archiving

Updated: 2026-07-09
Lists: 1 list mention
First commit: 2015-01-15
History: 5 history points
License: Apache-2.0
Issues: 22 open

stars

Forks: 25
Commits: 615 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

ukwa/shine

Prototype SOLR-powered web archive exploration UI.

Archived

Stack

JavaScript React Composer npm

Updated: 2020-06-03
Lists: 1 list mention
First commit: 2013-07-03
History: 5 history points
License: Apache-2.0
Issues: 50 open

stars

Forks: 7
Commits: 748 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

harvard-lil/bag-nabit

Download and attach provenance to public datasets

Stack

Python pytest uv

Updated: 2025-03-31
Lists: 1 list mention
First commit: 2024-11-28
History: 5 history points
License: Unknown
Issues: 1 open

stars

Forks: 1
Commits: 39 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

nla/httrack2warc

Converts HTTrack crawls to WARC files

Stack

Java Maven

GitHub topics

#web-archiving

Updated: 2024-08-06
Lists: 1 list mention
First commit: 2017-10-23
History: 5 history points
License: Apache-2.0
Issues: 2 open

stars

Forks: 6
Commits: 117 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

chfoo/warcat-rs

Command-line tool and Rust library for handling Web ARChive (WARC) files

Stack

Rust Cargo .NET SDK pip

Updated: 2025-06-02
Lists: 1 list mention
First commit: 2024-09-15
History: 5 history points
License: MPL-2.0
Issues: 5 open

stars

Forks: 3
Commits: 81 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

webis-de/wasp

No description.

Stack

Java Maven

Updated: 2026-06-30
Lists: 1 list mention
First commit: 2018-03-25
History: 5 history points
License: MIT
Issues: 4 open

stars

Forks: 4
Commits: 57 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

archivesunleashed/notebooks

Various examples of notebooks for working with web archives with the Archives Unleashed Toolkit, and derivatives generated by the Archives Unleashed Toolkit.

Stack

Jupyter Notebook

GitHub topics

#juypter-notebook #notebooks #pyspark-notebook #python3 #spark #web-archives

Updated: 2022-12-05
Lists: 1 list mention
First commit: 2019-11-06
History: 5 history points
License: Apache-2.0
Issues: 0 open

stars

Forks: 5
Commits: 185 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

helgeho/Web2Warc

An easy-to-use and highly customizable crawler that enables you to create your own little Web archives (WARC/CDX)

Stack

Scala

Updated: 2017-10-09
Lists: 1 list mention
First commit: 2016-01-29
History: 5 history points
License: MIT
Issues: 1 open

stars

Forks: 3
Commits: 11 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

alard/wget-lua

Wget with Lua extension

Stack

Updated: 2015-12-17
Lists: 1 list mention
First commit: 1999-12-02
History: 5 history points
License: GPL-3.0
Issues: 0 open

stars

Forks: 9
Commits: 3,507 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

steffenfritz/html2warc

simple script to convert web resources to a single warc file

Stack

Python

Updated: 2023-05-11
Lists: 1 list mention
First commit: 2015-01-19
History: 5 history points
License: MIT
Issues: 0 open

stars

Forks: 3
Commits: 16 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

vphill/web-archiving-course

Web Archiving Course

Updated: 2024-03-04
Lists: 1 list mention
First commit: 2022-02-22
History: 5 history points
License: NOASSERTION
Issues: 10 open

stars

Forks: 6
Commits: 103 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

harvard-lil/warcbench

A tool for exploring, analyzing, transforming, recombining, and extracting data from WARC (Web ARChive) files.

Stack

Python pytest uv

Updated: 2025-07-30
Lists: 1 list mention
First commit: 2025-02-24
History: 5 history points
License: MIT
Issues: 3 open

stars

Forks: 0
Commits: 348 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

lanl/Zotero-Robust-Links-Extension

Create Robust Links from within Zotero

Stack

JavaScript

GitHub topics

#link-rot #memento #reference-rot #references #web-archives

Updated: 2022-05-10
Lists: 1 list mention
First commit: 2019-08-29
History: 5 history points
License: NOASSERTION
Issues: 3 open

stars

Forks: 2
Commits: 94 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

midwork-finds-jobs/duckdb-web-archive

DuckDB extension to fetch pages from Wayback Machine & Common Crawl

AI dev

Stack

C++ CMake vcpkg

GitHub topics

#archived #cdx-api #common-crawl #duckdb #duckdb-extension #sql

Updated: 2026-06-27
Lists: 1 list mention
First commit: 2025-11-21
History: 5 history points
License: MIT
Issues: 2 open

stars

Forks: 1
Commits: 75 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

internetarchive/arch

Web application for distributed compute analysis of Archive-It web archive collections.

Stack

Scala PEP 517 uv

Updated: 2026-03-24
Lists: 1 list mention
First commit: 2020-11-30
History: 5 history points
License: AGPL-3.0
Issues: Disabled

stars

Forks: 3
Commits: 569 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

richardlehane/webarchive

golang readers for ARC and WARC webarchive formats

Stack

Go Go modules

Updated: 2023-04-03
Lists: 1 list mention
First commit: 2015-09-21
History: 5 history points
License: Apache-2.0
Issues: 0 open

stars

Forks: 2
Commits: 57 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

natliblux/warc-safe

A tool for detecting viruses and NSFW material in WARC files

Stack

Python Flask pip

GitHub topics

#antivirus #nsfw-classifier #warc #warc-safe #webarchiving

Updated: 2026-06-09
Lists: 1 list mention
First commit: 2024-05-03
History: 5 history points
License: GPL-3.0
Issues: 1 open

stars

Forks: 1
Commits: 33 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

internetarchive/Sparkling

Internet Archive's Sparkling Data Processing Library

Stack

Scala

Updated: 2026-05-04
Lists: 1 list mention
First commit: 2022-04-28
History: 5 history points
License: MIT
Issues: 1 open

stars

Forks: 2
Commits: 47 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

nlnwa/gowarcserver

No description.

Stack

Go Cobra gRPC Go Go modules

Updated: 2025-03-31
Lists: 1 list mention
First commit: 2021-02-02
History: 5 history points
License: Apache-2.0
Issues: 10 open

stars

Forks: 2
Commits: 372 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

Guillaume-Levrier/PANDORAE

A data retrieval & exploration protocol designed to investigate science and policy processes

Stack

JavaScript Electron npm

Updated: 2026-06-09
Lists: 1 list mention
First commit: 2018-12-10
History: 5 history points
License: MIT
Issues: 1 open

stars

Forks: 1
Commits: 1,514 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

unt-libraries/py-wasapi-client

A client for the Archive-It And Webrecorder WASAPI Data Transfer API

Stack

Python pytest pip

Updated: 2019-10-18
Lists: 1 list mention
First commit: 2017-08-10
History: 5 history points
License: BSD-3-Clause
Issues: 0 open

stars

Forks: 5
Commits: 118 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

oduwsdl/ORS

Object Resource Stream and CDXJ Drafts

Updated: 2018-11-28
Lists: 1 list mention
First commit: 2015-10-06
History: 5 history points
License: Unknown
Issues: 6 open

stars

Forks: 2
Commits: 20 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

emmadickson/unwarcit

No description.

Stack

Python pip

Updated: 2022-01-07
Lists: 1 list mention
First commit: 2021-12-13
History: 5 history points
License: Apache-2.0
Issues: 1 open

stars

Forks: 1
Commits: 8 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

wabarc/playback

Playback webpages from Wayback Machine

Stack

Go Go modules

GitHub topics

#go #playback #wayback-machine #webpage

Updated: 2026-07-10
Lists: 1 list mention
First commit: 2021-04-17
History: 5 history points
License: GPL-3.0
Issues: 7 open

stars

Forks: 2
Commits: 21 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

oduwsdl/MementoMap

A Tool to Summarize Web Archive Holdings

Stack

Python pip

GitHub topics

#memento #mementomap #profiling #python #ukvs #web-archive

Updated: 2021-06-15
Lists: 1 list mention
First commit: 2019-01-20
History: 5 history points
License: MIT
Issues: 0 open

stars

Forks: 1
Commits: 63 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

httpreserve/tikalinkextract

Tika based link (URL) extractor for httpreserve

Stack

HTML Go modules

GitHub topics

#archives #code4lib #digitalpreservation #httpreserve #iipc #tika

Updated: 2025-04-26
Lists: 1 list mention
First commit: 2017-04-03
History: 5 history points
License: Unknown
Issues: 6 open

stars

Forks: 0
Commits: 51 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

ukwa/ukwa-heritrix

The UKWA Heritrix3 custom modules and Docker builder.

Archived

Stack

Java Maven pip

Updated: 2024-12-02
Lists: 1 list mention
First commit: 2013-06-06
History: 5 history points
License: Unknown
Issues: 46 open

stars

Forks: 7
Commits: 575 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

arcalex/warcrefs

Web archive deduplication tools

Stack

Java Maven

Updated: 2018-10-18
Lists: 1 list mention
First commit: 2014-04-22
History: 5 history points
License: Unknown
Issues: 11 open

stars

Forks: 1
Commits: 39 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

Search awesome repositories

Find repositories

Put your repository first

How it works

Pricing

How it works

Pricing