Highlighted
Open highlighted repo slot
Put your repository first
Promote a GitHub repo at the top of Awesome repository list views for 7 days.
Awesome
GitHub projects from awesome lists
Search awesome repositories
Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.
- Repos indexed
- 9,945
- Awesome lists tracked
- 76
- Current results
- 103
103 repos shown
archivesunleashed/twut
An open-source toolkit for analyzing line-oriented JSON Twitter archives with Apache Spark.
Scala
#apache-spark#spark#spark-packages#tweets#twitter-data
1 awesome list
41 commits
first commit 2019-11-29
2 history points
updated 2026-03-17
httpreserve/linkstat
CLI implementation of httpreserve that can test links and retrieve internet archive replacements
Go
#archives#cli#code4lib#digipres#digital-preservation
1 awesome list
18 commits
first commit 2019-03-19
2 history points
updated 2024-11-21
web-archive-group/heritrix-walkthrough
No description.
Shell
1 awesome list
6 commits
first commit 2016-06-01
2 history points
updated 2016-06-10
helgeho/HadoopConcatGz
A Splitable Hadoop InputFormat for Concatenated GZIP Files and *.(w)arc.gz
Java
#hadoop#spark#warc#web-archiving#webarchive
1 awesome list
55 commits
first commit 2016-08-08
2 history points
updated 2018-02-07
sul-dlss-deprecated/wasapi-downloader
Java application to download WARCs from WASAPI
Java
#application#infrastructure#java
1 awesome list
348 commits
first commit 2017-04-28
2 history points
updated 2025-11-17
netarchivesuite/jwat-tools
JWAT Tools
Java
1 awesome list
141 commits
first commit 2012-02-02
2 history points
updated 2023-12-13
commoncrawl/whirlwind-python-notebook
A jupyter notebook illistrating the basics of Common Crawl's datasets.
Jupyter Notebook
#aws#open-datasets#s3#sagemaker
1 awesome list
7 commits
first commit 2025-10-28
2 history points
updated 2025-12-31
midwork-finds-jobs/duckdb_warc
DuckDB extension for parsing WARC files
Rust
1 awesome list
139 commits
first commit 2024-09-16
2 history points
updated 2026-02-05
netarchivesuite/jwat
Java Web Archive Toolkit
Java
1 awesome list
331 commits
first commit 2011-10-04
2 history points
updated 2023-12-13
commoncrawl/whirlwind-java
A whirlwind tour of Common Crawl's data using Java
Java
1 awesome list
17 commits
first commit 2025-10-21
2 history points
updated 2026-04-20
N0taN3rd/node-cdxj
Parse CDXJ(https://github.com/oduwsdl/ORS/wiki/CDXJ) files with node.js
JavaScript
#cdxj#web-archives#webarchive#webarchiving
1 awesome list
13 commits
first commit 2017-05-18
2 history points
updated 2017-07-20
helgeho/WarcPartitioner
Partition (W)ARC Files by MIME Type and Year
Java
#hadoop#warc#web-archiving#webarchive
1 awesome list
2 commits
first commit 2017-02-13
2 history points
updated 2017-02-13
internetarchive/bagnabit2warc
Convert a bag-nabit dataset stored in a ZIP into a full-content WARC.
Python
1 awesome list
2 commits
first commit 2026-03-09
2 history points
updated 2026-03-09