Awesome

GitHub projects from awesome lists

Search awesome repositories

Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.

Continue with GitHub Browse awesome lists Request a list

Repos indexed: 17,373
Awesome lists tracked: 125
Current results: 31

Find repositories

Start broad, then narrow by ecosystem, freshness, health, and growth.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Awesome list

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

31 repos shown

Topic: big-data

Browse

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

apache/spark

Apache Spark - A unified analytics engine for large-scale data processing

AI dev

Stack

Scala Bundler Maven npm

GitHub topics

#big-data #java #jdbc #python #r #scala

Updated: 2026-07-15
Lists: 4 list mentions
First commit: 2010-03-29
History: 10 history points
License: Apache-2.0
Issues: 450 open

43,614

stars

Forks: 29,275
Commits: 49,068 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apache/flink

Apache Flink

AI dev

Stack

Java pytest Maven pip uv

GitHub topics

#big-data #flink #java #python #scala #sql

Updated: 2026-07-14
Lists: 1 list mention
First commit: 2010-12-15
History: 5 history points
License: Apache-2.0
Issues: Disabled

26,176

stars

Forks: 13,989
Commits: 38,168 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

thingsboard/thingsboard

Open-source IoT Platform - Device management, data collection, processing and visualization.

Stack

Java Angular Express Spring Boot Tailwind CSS Gradle Maven npm

GitHub topics

#big-data #cloud #coap-server #dashboards #http #iiot

Updated: 2026-06-30
Lists: 2 list mentions
First commit: 2016-12-01
History: 5 history points
License: Apache-2.0
Issues: 275 open

21,976

stars

Forks: 6,356
Commits: 28,122 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

andkret/Cookbook

The Data Engineering Cookbook

Stack

Python

GitHub topics

#best-practices #big-data #cookbook #data-engineer #data-engineering

Updated: 2026-06-12
Lists: 0 list mentions
First commit: 2019-03-10
History: 46 history points
License: Apache-2.0
Issues: 127 open

15,178

stars

Forks: 2,721
Commits: 403 commits
Star growth, last 7 days: +5 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

vesoft-inc/nebula

A distributed, fast open-source graph database featuring horizontal scalability and high availability

Stack

C++ pytest CMake pip

GitHub topics

#big-data #cpp #database #distributed #distributed-systems #graph

Updated: 2026-05-18
Lists: 1 list mention
First commit: 2018-08-14
History: 4 history points
License: Apache-2.0
Issues: 678 open

12,290

stars

Forks: 1,323
Commits: 3,552 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

provectus/kafka-ui

Open-Source Web UI for Apache Kafka Management

Stack

Java NestJS React React Native Spring Boot Maven npm pnpm

GitHub topics

#apache-kafka #big-data #cluster-management #event-streaming #hacktoberfest #kafka

Updated: 2024-07-26
Lists: 1 list mention
First commit: 2019-11-26
History: 4 history points
License: Apache-2.0
Issues: 351 open

12,228

stars

Forks: 1,410
Commits: 1,918 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

quickwit-oss/quickwit

Cloud-native OSS search engine for observability

AI dev

Stack

Rust Axum React Vite Warp Cargo pip Pipenv

GitHub topics

#big-data #cloud-native #cloud-storage #distributed-tracing #log-management #logs

Updated: 2026-07-06
Lists: 4 list mentions
First commit: 2021-04-13
History: 5 history points
License: Apache-2.0
Issues: 778 open

11,390

stars

Forks: 562
Commits: 3,733 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

cython/cython

The most widely used Python to C compiler

Stack

Cython PEP 517 pip

GitHub topics

#big-data #c #cpp #cpython #cpython-extensions #cython

Updated: 2026-07-09
Lists: 2 list mentions
First commit: 2006-10-20
History: 8 history points
License: Apache-2.0
Issues: 1,513 open

10,788

stars

Forks: 1,613
Commits: 20,452 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

catboost/catboost

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Stack

C++ Cargo CMake Conan

GitHub topics

#big-data #catboost #categorical-features #coreml #cuda #data-mining

Updated: 2026-07-15
Lists: 4 list mentions
First commit: 2017-07-18
History: 10 history points
License: Apache-2.0
Issues: 710 open

9,025

stars

Forks: 1,312
Commits: 50,592 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apache/datafusion

Apache DataFusion SQL Query Engine

AI dev

Stack

Rust Cargo npm PEP 517

GitHub topics

#arrow #big-data #dataframe #datafusion #olap #python

Updated: 2026-07-17
Lists: 1 list mention
First commit: 2016-02-05
History: 25 history points
License: Apache-2.0
Issues: 2,060 open

8,986

stars

Forks: 2,230
Commits: 14,046 commits
Star growth, last 7 days: +22 +0.2%
Commit velocity, last 7 days: +82 +0.6%

Website GitHub

delta-io/delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

Stack

Scala Astro Vite Maven PEP 517 pip

GitHub topics

#acid #analytics #big-data #delta-lake #spark

Updated: 2026-07-14
Lists: 1 list mention
First commit: 2019-04-12
History: 5 history points
License: Apache-2.0
Issues: 1,547 open

8,905

stars

Forks: 2,137
Commits: 5,361 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apache/beam

Apache Beam is a unified programming model for Batch and Streaming data processing.

AI dev

Stack

Java Cobra Flutter Gin gRPC Go Dart Pub Go modules Gradle

GitHub topics

#batch #beam #big-data #golang #java #python

Updated: 2026-07-15
Lists: 1 list mention
First commit: 2014-12-13
History: 5 history points
License: Apache-2.0
Issues: 4,009 open

8,628

stars

Forks: 4,604
Commits: 47,584 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

h2oai/h2o-3

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

AI dev

Stack

Jupyter Notebook Bundler Gradle npm

GitHub topics

#automl #big-data #data-science #deep-learning #distributed #ensemble-learning

Updated: 2026-07-12
Lists: 3 list mentions
First commit: 2014-03-03
History: 8 history points
License: Apache-2.0
Issues: 2,879 open

7,498

stars

Forks: 2,030
Commits: 32,791 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

arkime/arkime

Arkime is an open source, large scale, full packet capturing, indexing, and database system.

AI dev

Stack

C Express Svelte Vite Vue npm

GitHub topics

#big-data #c #javascript #network-monitoring #nsm #packet-capture

Updated: 2026-07-06
Lists: 1 list mention
First commit: 2012-07-06
History: 2 history points
License: Apache-2.0
Issues: 40 open

7,408

stars

Forks: 1,158
Commits: 6,605 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

feast-dev/feast

The Open Source Feature Store for AI/ML

AI dev

Stack

Python Astro Cobra FastAPI gRPC Go Go modules Maven npm

GitHub topics

#big-data #data-engineering #data-quality #data-science #feature-store #features

Updated: 2026-07-17
Lists: 2 list mentions
First commit: 2018-12-10
History: 57 history points
License: Apache-2.0
Issues: 391 open

7,135

stars

Forks: 1,368
Commits: 4,622 commits
Star growth, last 7 days: +8 +0.1%
Commit velocity, last 7 days: +6 +0.1%

Website GitHub

Eventual-Inc/Daft

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

AI dev

Stack

Rust Cargo CMake PEP 517

GitHub topics

#ai-engineering #ai-pipeline #arrow #artificial-intelligence #big-data #data-engineering

Updated: 2026-07-17
Lists: 0 list mentions
First commit: 2022-05-10
History: 24 history points
License: Apache-2.0
Issues: 353 open

5,634

stars

Forks: 522
Commits: 4,492 commits
Star growth, last 7 days: +14 +0.2%
Commit velocity, last 7 days: +8 +0.2%

Website GitHub

microsoft/SynapseML

Simple and Distributed Machine Learning

AI dev

Stack

Scala React npm PEP 517 pip

GitHub topics

#ai #apache-spark #azure #big-data #cognitive-services #data-science

Updated: 2026-07-06
Lists: 1 list mention
First commit: 2017-06-02
History: 5 history points
License: MIT
Issues: 392 open

5,231

stars

Forks: 863
Commits: 1,792 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

crate/crate

CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is PostgreSQL-compatible, and based on Lucene.

Stack

Java Maven pip

GitHub topics

#analytics #big-data #cratedb #database #dbms #distributed

Updated: 2026-07-06
Lists: 1 list mention
First commit: 2013-04-10
History: 2 history points
License: Apache-2.0
Issues: 310 open

4,414

stars

Forks: 602
Commits: 17,606 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

lakehq/sail

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

AI dev

Stack

Rust Axum pytest React Tailwind CSS Cargo PEP 517 pnpm

GitHub topics

#apache-iceberg #apache-spark #arrow #artificial-intelligence #big-data #data-engineering

Updated: 2026-07-17
Lists: 1 list mention
First commit: 2023-12-21
History: 20 history points
License: Apache-2.0
Issues: 215 open

3,188

stars

Forks: 194
Commits: 1,458 commits
Star growth, last 7 days: +50 +1.6%
Commit velocity, last 7 days: +14 +1.0%

Website GitHub

FeatureBaseDB/featurebase

A crazy fast analytical database, built on bitmaps. Perfect for ML applications. Learn more at: http://docs.featurebase.com/. Start a Docker instance: https://hub.docker.com/r/featurebasedb/featurebase

Archived

Stack

Go Cobra Echo Gin gRPC Go Go modules npm Yarn

GitHub topics

#big-data #bitmap #database #go #index #pilosa

Updated: 2024-02-21
Lists: 1 list mention
First commit: 2013-10-16
History: 54 history points
License: Apache-2.0
Issues: 7 open

2,525

stars

Forks: 238
Commits: 5,576 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

man-group/ArcticDB

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.

AI dev

Stack

C++ Flask pytest CMake PEP 517 pip

GitHub topics

#big-data #data #data-analysis #data-science #database #dataframe

Updated: 2026-07-02
Lists: 1 list mention
First commit: 2023-03-15
History: 2 history points
License: NOASSERTION
Issues: 298 open

2,418

stars

Forks: 200
Commits: 2,020 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

gchq/Gaffer

A large-scale entity and relation database supporting aggregation of properties

Archived

Stack

Java Spring Boot Maven

GitHub topics

#accumulo #aggregation #big-data #graph #graph-database #hadoop

Updated: 2025-06-06
Lists: 1 list mention
First commit: 2015-12-14
History: 5 history points
License: Apache-2.0
Issues: 138 open

1,788

stars

Forks: 363
Commits: 7,332 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

matanolabs/matano

Open source security data lake for threat hunting, detection & response, and cybersecurity analytics at petabyte scale on AWS

Stack

Rust Cargo Gradle npm

GitHub topics

#alerting #apache-iceberg #aws #aws-security #big-data #cloud

Updated: 2025-01-08
Lists: 2 list mentions
First commit: 2022-07-14
History: 3 history points
License: Apache-2.0
Issues: 55 open

1,681

stars

Forks: 120
Commits: 576 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

commsor/titanoboa

Titanoboa makes complex workflows easy. It is a low-code workflow orchestration platform for JVM - distributed, highly scalable and fault tolerant.

Stack

Clojure

GitHub topics

#big-data #distributed #distributed-systems #esb #integrations #ipaas

Updated: 2022-09-16
Lists: 0 list mentions
First commit: 2018-05-22
History: 22 history points
License: AGPL-3.0
Issues: 13 open

922

stars

Forks: 50
Commits: 295 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

rakam-io/rakam-api

📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)

Stack

Java Maven

GitHub topics

#analytics #analytics-platform #bi-server #big-data #java

Updated: 2021-11-13
Lists: 1 list mention
First commit: 2014-01-16
History: 54 history points
License: AGPL-3.0
Issues: 13 open

793

stars

Forks: 100
Commits: 1,762 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

synmetrix/synmetrix

Synmetrix – production-ready open source semantic layer on Cube

Stack

JavaScript Express npm Yarn

GitHub topics

#big-data #bigquery #business-intelligence #clickhouse #cube #cubejs

Updated: 2025-02-07
Lists: 1 list mention
First commit: 2021-06-18
History: 2 history points
License: NOASSERTION
Issues: 9 open

620

stars

Forks: 48
Commits: 251 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

metriql/metriql

The metrics layer for your data. Join us at https://metriql.com/slack

Stack

Kotlin Vite Vue Maven npm

GitHub topics

#analytics #big-data #dbt #olap

Updated: 2023-03-29
Lists: 1 list mention
First commit: 2021-05-23
History: 2 history points
License: Apache-2.0
Issues: 33 open

328

stars

Forks: 29
Commits: 364 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

Hydrospheredata/mist

Serverless proxy for Spark cluster

Stack

Scala pytest pip

GitHub topics

#apache-spark #api #big-data #serverless

Updated: 2026-04-13
Lists: 2 list mentions
First commit: 2016-02-01
History: 5 history points
License: Apache-2.0
Issues: 32 open

325

stars

Forks: 69
Commits: 2,020 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apache/predictionio-sdk-ruby

PredictionIO Ruby SDK

Archived

Stack

Ruby Bundler

GitHub topics

#big-data #predictionio #scala

Updated: 2018-09-20
Lists: 1 list mention
First commit: 2013-02-01
History: 3 history points
License: Apache-2.0
Issues: Disabled

191

stars

Forks: 37
Commits: 119 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

archivesunleashed/aut

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

Stack

Scala Maven

GitHub topics

#analysis #apache-spark #big-data #big-data-analytics #dataframe #digital-humanities

Updated: 2025-12-05
Lists: 1 list mention
First commit: 2013-07-13
History: 5 history points
License: Apache-2.0
Issues: 5 open

158

stars

Forks: 33
Commits: 1,032 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

Search awesome repositories

Find repositories

Put your repository first

How it works

Pricing

How it works

Pricing