Highlighted
Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome List
Awesome Web Archiving
An Awesome List for getting started with web archiving
iipc/awesome-web-archiving
#awesome
#awesome-list
#webarchiving
- List stars
- 2,605
- README repos
- 105
- Indexed repos
- 103
- List commits
- 162
- Forks
- 196
- Open issues
- 11
Tracked list growth
GitHub stars and default-branch commits for iipc/awesome-web-archiving.
Latest scan 2026-07-17 10:49
Likes history
GitHub starsCommits history
Default branch commits
Latest repo push 2026-07-15
ArchiveBox/ArchiveBox
🗃 Open source self-hosted web archiving. Takes URLs/browser history/bookmarks/Pocket/Pinboard/etc., saves HTML, JS, PDFs, media, and more...
AI dev
GitHub topics
- Updated
- 2026-07-15
- Lists
- 4 list mentions
- First commit
- 2017-05-05
- License
- MIT
- Issues
- 174 open
27,940
stars
- Forks
- 1,553
- Commits
- 5,730 commits
- Star growth, last 7 days
- +49 +0.2%
- Commit velocity, last 7 days
- +2 0.0%
jjjake/internetarchive
A Python and Command-Line Interface to Archive.org
AI dev
- Updated
- 2026-07-06
- Lists
- 1 list mention
- First commit
- 2012-08-15
- License
- AGPL-3.0
- Issues
- 101 open
1,882
stars
- Forks
- 249
- Commits
- 2,331 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
ArchiveTeam/grab-site
The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns
- Updated
- 2025-05-23
- Lists
- 1 list mention
- First commit
- 2015-02-05
- License
- NOASSERTION
- Issues
- 103 open
1,598
stars
- Forks
- 157
- Commits
- 1,174 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
DocNow/twarc
A command line tool (and Python library) for archiving Twitter JSON
- Updated
- 2025-10-31
- Lists
- 1 list mention
- First commit
- 2013-01-14
- License
- MIT
- Issues
- 48 open
1,393
stars
- Forks
- 254
- Commits
- 1,804 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
bellingcat/auto-archiver
Automatically archive links to videos, images, and social media content from Google Sheets (and more).
- Updated
- 2026-07-10
- Lists
- 1 list mention
- First commit
- 2021-01-15
- License
- MIT
- Issues
- 16 open
1,095
stars
- Forks
- 107
- Commits
- 1,546 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
WikiTeam/wikiteam
Tools for downloading and preserving wikis. We archive wikis, from Wikipedia to tiniest wikis. As of 2026, WikiTeam has preserved more than 600,000 wikis.
GitHub topics
- Updated
- 2026-01-10
- Lists
- 1 list mention
- First commit
- 2011-04-05
- License
- GPL-3.0
- Issues
- 172 open
853
stars
- Forks
- 175
- Commits
- 1,143 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
internetarchive/brozzler
brozzler - distributed browser-based web crawler
- Updated
- 2026-07-07
- Lists
- 1 list mention
- First commit
- 2014-01-21
- License
- Apache-2.0
- Issues
- 60 open
807
stars
- Forks
- 115
- Commits
- 1,789 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
oduwsdl/ipwb
InterPlanetary Wayback: A distributed and persistent archive replay system using IPFS
GitHub topics
- Updated
- 2026-06-15
- Lists
- 1 list mention
- First commit
- 2016-03-04
- License
- MIT
- Issues
- 160 open
654
stars
- Forks
- 41
- Commits
- 1,622 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
akamhy/waybackpy
Wayback Machine API interface & a command-line tool
- Updated
- 2024-02-26
- Lists
- 2 list mentions
- First commit
- 2020-05-02
- License
- MIT
- Issues
- 21 open
594
stars
- Forks
- 40
- Commits
- 497 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
webrecorder/warcio
Streaming WARC/ARC library for fast web archive IO
GitHub topics
- Updated
- 2026-06-10
- Lists
- 1 list mention
- First commit
- 2017-03-02
- License
- Apache-2.0
- Issues
- 62 open
459
stars
- Forks
- 70
- Commits
- 161 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
internetarchive/warcprox
WARC writing MITM HTTP/S proxy
- Updated
- 2026-06-17
- Lists
- 1 list mention
- First commit
- 2012-07-18
- License
- Unknown
- Issues
- 28 open
456
stars
- Forks
- 66
- Commits
- 1,116 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
oduwsdl/archivenow
A Tool To Push Web Resources Into Web Archives
GitHub topics
- Updated
- 2024-01-23
- Lists
- 1 list mention
- First commit
- 2017-02-09
- License
- MIT
- Issues
- 17 open
434
stars
- Forks
- 40
- Commits
- 186 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
Florents-Tselai/WarcDB
WarcDB: Web crawl data as SQLite databases.
- Updated
- 2024-07-13
- Lists
- 1 list mention
- First commit
- 2022-05-29
- License
- Apache-2.0
- Issues
- 9 open
406
stars
- Forks
- 10
- Commits
- 73 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
internetarchive/warctools
Command line tools and libraries for handling and manipulating WARC files (and HTTP contents)
- Updated
- 2025-08-18
- Lists
- 1 list mention
- First commit
- 2010-12-04
- License
- MIT
- Issues
- 17 open
- Forks
- 33
- Commits
- 266 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
chfoo/warcat
Tool and library for handling Web ARChive (WARC) files.
- Updated
- 2024-10-11
- Lists
- 1 list mention
- First commit
- 2013-04-09
- License
- GPL-3.0
- Issues
- 15 open
165
stars
- Forks
- 20
- Commits
- 78 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
recrm/ArchiveTools
A collection of tools for archiving and analysing the internet.
- Updated
- 2022-07-06
- Lists
- 1 list mention
- First commit
- 2015-01-14
- License
- GPL-3.0
- Issues
- 2 open
79
stars
- Forks
- 15
- Commits
- 30 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
justinlittman/fbarc
A commandline tool and Python library for archiving data from Facebook using the Graph API.
Archived
GitHub topics
- Updated
- 2018-01-29
- Lists
- 1 list mention
- First commit
- 2017-02-22
- License
- CC0-1.0
- Issues
- 3 open
78
stars
- Forks
- 11
- Commits
- 68 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
turicas/crau
Easy-to-use Web archiver
- Updated
- 2026-04-13
- Lists
- 1 list mention
- First commit
- 2019-10-26
- License
- LGPL-3.0
- Issues
- 11 open
64
stars
- Forks
- 10
- Commits
- 76 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
peterk/warcworker
A dockerized, queued high fidelity web archiver based on Squidwarc
- Updated
- 2024-07-09
- Lists
- 1 list mention
- First commit
- 2018-07-21
- License
- GPL-3.0
- Issues
- 6 open
62
stars
- Forks
- 9
- Commits
- 34 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
webrecorder/har2warc
Convert HTTP Archive (HAR) -> Web Archive (WARC) format
- Updated
- 2018-10-21
- Lists
- 1 list mention
- First commit
- 2017-03-16
- License
- Apache-2.0
- Issues
- 3 open
55
stars
- Forks
- 4
- Commits
- 18 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
ikreymer/webarchive-indexing
Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.
- Updated
- 2017-12-04
- Lists
- 1 list mention
- First commit
- 2015-02-26
- License
- MIT
- Issues
- 5 open
- Forks
- 12
- Commits
- 47 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
commoncrawl/whirlwind-python
A whirlwind tour of Common Crawl's data using Python
- Updated
- 2026-06-15
- Lists
- 1 list mention
- First commit
- 2024-06-23
- License
- Apache-2.0
- Issues
- 0 open
- Forks
- 9
- Commits
- 41 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
PromyLOPh/crocoite
Web archiving using Google Chrome
Archived
GitHub topics
- Updated
- 2019-12-30
- Lists
- 1 list mention
- First commit
- 2017-11-17
- License
- MIT
- Issues
- 12 open
45
stars
- Forks
- 7
- Commits
- 282 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
harvard-lil/bag-nabit
Download and attach provenance to public datasets
- Updated
- 2025-03-31
- Lists
- 1 list mention
- First commit
- 2024-11-28
- License
- Unknown
- Issues
- 1 open
- Forks
- 1
- Commits
- 39 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
steffenfritz/html2warc
simple script to convert web resources to a single warc file
Stack
- Updated
- 2023-05-11
- Lists
- 1 list mention
- First commit
- 2015-01-19
- License
- MIT
- Issues
- 0 open
- Forks
- 3
- Commits
- 16 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
harvard-lil/warcbench
A tool for exploring, analyzing, transforming, recombining, and extracting data from WARC (Web ARChive) files.
- Updated
- 2025-07-30
- Lists
- 1 list mention
- First commit
- 2025-02-24
- License
- MIT
- Issues
- 3 open
- Forks
- 0
- Commits
- 348 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
natliblux/warc-safe
A tool for detecting viruses and NSFW material in WARC files
GitHub topics
- Updated
- 2026-06-09
- Lists
- 1 list mention
- First commit
- 2024-05-03
- License
- GPL-3.0
- Issues
- 1 open
18
stars
- Forks
- 1
- Commits
- 33 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
unt-libraries/py-wasapi-client
A client for the Archive-It And Webrecorder WASAPI Data Transfer API
- Updated
- 2019-10-18
- Lists
- 1 list mention
- First commit
- 2017-08-10
- License
- BSD-3-Clause
- Issues
- 0 open
- Forks
- 5
- Commits
- 118 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
emmadickson/unwarcit
No description.
- Updated
- 2022-01-07
- Lists
- 1 list mention
- First commit
- 2021-12-13
- License
- Apache-2.0
- Issues
- 1 open
13
stars
- Forks
- 1
- Commits
- 8 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
oduwsdl/MementoMap
A Tool to Summarize Web Archive Holdings
GitHub topics
- Updated
- 2021-06-15
- Lists
- 1 list mention
- First commit
- 2019-01-20
- License
- MIT
- Issues
- 0 open
12
stars
- Forks
- 1
- Commits
- 63 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%
internetarchive/bagnabit2warc
Convert a bag-nabit dataset stored in a ZIP into a full-content WARC.
Stack
- Updated
- 2026-03-09
- Lists
- 1 list mention
- First commit
- 2026-03-09
- License
- AGPL-3.0
- Issues
- 0 open
- Forks
- 0
- Commits
- 2 commits
- Star growth, last 7 days
- 0 0.0%
- Commit velocity, last 7 days
- 0 0.0%