Awesome List

Awesome Bigdata

A curated list of awesome big data frameworks, ressources and other awesomeness.

oxnr/awesome-bigdata #awesome #awesome-list #bigdata #data #data-analytics #data-science #data-stream #data-visualization #data-warehouse #database #distributed-database #series-database #stream-processing #streaming-data #visualize-data 404 Not Found | https://api.github.com/repos/streamsets/datacollector | message=Not Found | rate_limit_remaining=3838 | rate_limit_reset=1788436169

Open GitHub

List stars: 14,609
README repos: 214
Indexed repos: 186
List commits: 592
Forks: 2,588
Open issues: 5

Tracked list growth

GitHub stars and default-branch commits for oxnr/awesome-bigdata.

Latest scan 2026-09-03 10:49

Likes history

GitHub stars

Commits history

Default branch commits

Indexed repositories

25 repos matching these filters.

Framework: pytest

Latest repo push 2026-09-04

Browse

Filter this list

Search within Awesome Bigdata or narrow by ecosystem and project health.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

apache/superset

Apache Superset is a Data Visualization and Data Exploration Platform

AI dev

Stack

Python Celery Express Flask pytest npm PEP 517 pip

GitHub topics

#analytics #apache #apache-superset #asf #bi #business-analytics

Updated: 2026-09-03
Lists: 5 list mentions
First commit: 2015-07-02
License: Apache-2.0
Issues: 611 open

74,609

stars

Forks: 18,226
Commits: 22,589 commits
Star growth, last 7 days: +102 +0.1%
Commit velocity, last 7 days: +111 +0.5%

Website GitHub

keras-team/keras

Deep Learning for humans

AI dev

Stack

Python pytest PEP 517 pip

GitHub topics

#data-science #deep-learning #jax #machine-learning #neural-networks #python

Updated: 2026-09-03
Lists: 4 list mentions
First commit: 2015-03-28
License: Apache-2.0
Issues: 223 open

64,275

stars

Forks: 19,773
Commits: 12,673 commits
Star growth, last 7 days: +15 0.0%
Commit velocity, last 7 days: +29 +0.2%

Website GitHub

apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

AI dev

Stack

Python Celery Cobra FastAPI Flask Go modules Gradle npm

GitHub topics

#airflow #apache #apache-airflow #automation #dag #data-engineering

Updated: 2026-09-03
Lists: 5 list mentions
First commit: 2014-10-06
License: Apache-2.0
Issues: 2,097 open

46,706

stars

Forks: 17,744
Commits: 40,787 commits
Star growth, last 7 days: +81 +0.2%
Commit velocity, last 7 days: +75 +0.2%

Website GitHub

milvus-io/milvus

Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search

AI dev

Stack

Go Axum Cobra Gin gRPC Go Cargo CMake Conan

GitHub topics

#anns #cloud-native #diskann #distributed #embedding-database #embedding-similarity

Updated: 2026-09-04
Lists: 8 list mentions
First commit: 2019-03-18
License: Apache-2.0
Issues: 1,320 open

45,963

stars

Forks: 4,223
Commits: 25,502 commits
Star growth, last 7 days: +106 +0.2%
Commit velocity, last 7 days: +49 +0.2%

Website GitHub

apache/spark

Apache Spark - A unified analytics engine for large-scale data processing

AI dev

Stack

Scala pytest Bundler Maven npm

GitHub topics

#big-data #java #jdbc #python #r #scala

Updated: 2026-08-01
Lists: 4 list mentions
First commit: 2010-03-29
License: Apache-2.0
Issues: 465 open

43,763

stars

Forks: 29,298
Commits: 49,379 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

ray-project/ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

AI dev

Stack

Python Celery FastAPI pytest React Maven npm PEP 517

GitHub topics

#data-science #deep-learning #deployment #distributed #hyperparameter-optimization #hyperparameter-search

Updated: 2026-08-03
Lists: 5 list mentions
First commit: 2016-02-07
License: Apache-2.0
Issues: 3,492 open

43,414

stars

Forks: 7,873
Commits: 31,230 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

getredash/redash

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

Stack

Python Flask pytest React npm pip pnpm

GitHub topics

#analytics #athena #bi #bigquery #business-intelligence #dashboard

Updated: 2026-08-18
Lists: 5 list mentions
First commit: 2013-10-25
License: BSD-2-Clause
Issues: 802 open

28,766

stars

Forks: 4,624
Commits: 7,979 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

taosdata/TDengine

High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios

AI dev

Stack

C Gin gRPC Go pytest Cargo CMake Conan

GitHub topics

#bigdata #cloud-native #cluster #connected-vehicles #database #distributed

Updated: 2026-08-21
Lists: 2 list mentions
First commit: 2019-07-11
License: AGPL-3.0
Issues: 448 open

25,079

stars

Forks: 5,018
Commits: 86,668 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

plotly/dash

Data Apps & Dashboards for Python. No JavaScript Required.

AI dev

Stack

Python Celery FastAPI Flask pytest npm pip Yarn

GitHub topics

#ai #bioinformatics #charting #dash #dashboards #data-science

Updated: 2026-08-01
Lists: 2 list mentions
First commit: 2015-04-10
License: MIT
Issues: 540 open

24,360

stars

Forks: 2,311
Commits: 9,948 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

matplotlib/matplotlib

matplotlib: plotting with Python

Stack

Python pytest Tornado Meson npm uv

GitHub topics

#data-science #data-visualization #gtk #matplotlib #plotting #python

Updated: 2026-08-25
Lists: 3 list mentions
First commit: 2003-05-12
License: Unknown
Issues: 1,473 open

23,105

stars

Forks: 8,452
Commits: 55,125 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

spotify/luigi

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

Stack

Python pytest Tornado PEP 517 uv

GitHub topics

#hadoop #luigi #orchestration-framework #python #scheduling

Updated: 2026-07-18
Lists: 3 list mentions
First commit: 2011-11-17
License: Apache-2.0
Issues: 165 open

18,756

stars

Forks: 2,462
Commits: 4,320 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

AI dev

Stack

Python FastAPI Flask LangChain Next.js npm PEP 517 pip

GitHub topics

#analytics #dagster #data-engineering #data-integration #data-orchestrator #data-pipelines

Updated: 2026-08-22
Lists: 4 list mentions
First commit: 2018-04-30
License: Apache-2.0
Issues: 2,595 open

16,050

stars

Forks: 2,255
Commits: 27,882 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

alibaba/zvec

A lightweight, lightning-fast, in-process vector database

Stack

C++ pytest CMake PEP 517

GitHub topics

#agent-skills #db #embedded #faiss #hnsw #llm-memory

Updated: 2026-07-30
Lists: 3 list mentions
First commit: 2025-12-30
License: Apache-2.0
Issues: 72 open

15,355

stars

Forks: 973
Commits: 333 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

yugabyte/yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.

AI dev

Stack

C Cobra Express FastAPI gRPC Go CMake Go modules Gradle

GitHub topics

#cloud-native #cpp #database #distributed-database #distributed-sql #distributed-sql-database

Updated: 2026-09-03
Lists: 2 list mentions
First commit: 2005-06-28
License: NOASSERTION
Issues: 8,347 open

10,510

stars

Forks: 1,312
Commits: 39,737 commits
Star growth, last 7 days: +9 +0.1%
Commit velocity, last 7 days: +110 +0.3%

Website GitHub

YelpArchive/elastalert

Easy & Flexible Alerting With ElasticSearch

Stack

Python pytest pip

Updated: 2024-08-07
Lists: 1 list mention
First commit: 2015-02-19
License: Apache-2.0
Issues: 1,401 open

7,988

stars

Forks: 1,686
Commits: 2,082 commits
Star growth, last 7 days: +1 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

feast-dev/feast

The Open Source Feature Store for AI/ML

AI dev

Stack

Python Astro Cobra FastAPI gRPC Go Go modules Maven npm

GitHub topics

#big-data #data-engineering #data-quality #data-science #feature-store #features

Updated: 2026-09-02
Lists: 2 list mentions
First commit: 2018-12-10
License: Apache-2.0
Issues: 426 open

7,248

stars

Forks: 1,423
Commits: 4,719 commits
Star growth, last 7 days: +14 +0.2%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

MaterializeInc/materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

AI dev

Stack

Rust Axum FastAPI pytest React Cargo npm PEP 517

GitHub topics

#cdc #data-mesh #data-store #database #distributed-systems #kafka

Updated: 2026-09-03
Lists: 2 list mentions
First commit: 2017-12-05
License: NOASSERTION
Issues: Disabled

6,365

stars

Forks: 512
Commits: 50,906 commits
Star growth, last 7 days: +2 0.0%
Commit velocity, last 7 days: +50 +0.1%

Website GitHub

numenta/nupic-legacy

Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex.

Stack

Python pytest Tornado pip

GitHub topics

#artificial-intelligence #hierarchical-temporal-memory #machine-intelligence #neocortex

Updated: 2024-12-03
Lists: 1 list mention
First commit: 2013-04-05
License: MIT
Issues: 465 open

6,350

stars

Forks: 1,529
Commits: 6,627 commits
Star growth, last 7 days: +1 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

aimhubio/aim

Aim 💫 — An easy-to-use & supercharged open-source experiment tracker.

Stack

Python FastAPI pytest React npm PEP 517 pip

GitHub topics

#ai #data-science #data-visualization #experiment-tracking #machine-learning #metadata

Updated: 2026-08-26
Lists: 4 list mentions
First commit: 2019-05-31
License: Apache-2.0
Issues: 469 open

6,245

stars

Forks: 407
Commits: 2,265 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

infiniflow/infinity

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

AI dev

Stack

C++ Next.js pytest React Svelte CMake Go modules npm

GitHub topics

#ai-native #approximate-nearest-neighbor-search #bm25 #cpp20 #cpp20-modules #embedding

Updated: 2026-08-01
Lists: 4 list mentions
First commit: 2022-07-18
License: Apache-2.0
Issues: 64 open

4,658

stars

Forks: 435
Commits: 3,159 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

d2iq-archive/marathon

Deploy and manage containers (including Docker) on top of Apache Mesos at scale.

Archived

Stack

Scala pytest Bundler npm pip

GitHub topics

#dcos #dcos-orchestration-guild

Updated: 2022-09-08
Lists: 1 list mention
First commit: 2013-06-25
License: Apache-2.0
Issues: 29 open

4,026

stars

Forks: 830
Commits: 6,969 commits
Star growth, last 7 days: -2 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

chdb-io/chdb

chDB is an in-process OLAP SQL Engine 🚀 powered by ClickHouse

AI dev

Stack

Python pytest PEP 517 pip

GitHub topics

#chdb #clickhouse #clickhouse-database #clickhouse-server #data-science #database

Updated: 2026-07-29
Lists: 2 list mentions
First commit: 2008-12-01
License: Apache-2.0
Issues: 38 open

2,852

stars

Forks: 127
Commits: 3,368 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

mara/mara-pipelines

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

Stack

Python pytest PEP 517 pip

GitHub topics

#data #data-integration #etl #pipeline #postgresql #python

Updated: 2023-12-15
Lists: 1 list mention
First commit: 2018-04-08
License: MIT
Issues: 26 open

2,089

stars

Forks: 100
Commits: 172 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

Hydrospheredata/mist

Serverless proxy for Spark cluster

Stack

Scala pytest pip

GitHub topics

#apache-spark #api #big-data #serverless

Updated: 2026-04-13
Lists: 2 list mentions
First commit: 2016-02-01
License: Apache-2.0
Issues: 32 open

325

stars

Forks: 69
Commits: 2,020 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

linkedin/isolation-forest

A distributed Spark/Scala implementation of the isolation forest and extended isolation forest algorithms for unsupervised outlier detection, featuring support for scalable training and ONNX export for easy cross-platform inference.

Stack

Scala pytest Gradle PEP 517 pip

GitHub topics

#anomaly-detection #isolation-forest #linkedin #machine-learning #onnx #outlier-detection

Updated: 2026-08-04
Lists: 2 list mentions
First commit: 2019-08-12
License: NOASSERTION
Issues: 1 open

264

stars

Forks: 54
Commits: 103 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

Activity

Default branch: master
Last pushed: 2026-07-31
GitHub updated: 2026-09-03
Created: 2014-07-04
First commit: -
Last scanned: 2026-09-03 10:49
Watchers: 817

Indexed repo mix

Repo stars: 1,520,593
Repo forks: 355,072
Active: 153
Archived: 33

Languages

Java (44) Go (25) Python (25) C++ (19) JavaScript (15) Scala (15) C (8) Erlang (5) Rust (5) TypeScript (4) Clojure (3) C# (2)

Awesome Bigdata

Tracked list growth

Likes history

Commits history

Indexed repositories

Filter this list

Put your repository first

How it works

Pricing