Awesome List

Awesome Bigdata

A curated list of awesome big data frameworks, ressources and other awesomeness.

oxnr/awesome-bigdata #awesome #awesome-list #bigdata #data #data-analytics #data-science #data-stream #data-visualization #data-warehouse #database #distributed-database #series-database #stream-processing #streaming-data #visualize-data 404 Not Found | https://api.github.com/repos/streamsets/datacollector | message=Not Found | rate_limit_remaining=4577 | rate_limit_reset=1784288973

Open GitHub

List stars: 14,494
README repos: 211
Indexed repos: 183
List commits: 592
Forks: 2,586
Open issues: 9

Tracked list growth

GitHub stars and default-branch commits for oxnr/awesome-bigdata.

Latest scan 2026-07-17 10:49

Likes history

GitHub stars

Commits history

Default branch commits

Indexed repositories

22 repos matching these filters.

Framework: pytest

Latest repo push 2026-07-17

Browse

Filter this list

Search within Awesome Bigdata or narrow by ecosystem and project health.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

apache/superset

Apache Superset is a Data Visualization and Data Exploration Platform

AI dev

Stack

TypeScript Celery Express Flask pytest npm PEP 517 pip

GitHub topics

#analytics #apache #apache-superset #asf #bi #business-analytics

Updated: 2026-07-17
Lists: 5 list mentions
First commit: 2015-07-02
License: Apache-2.0
Issues: 658 open

73,865

stars

Forks: 17,897
Commits: 21,437 commits
Star growth, last 7 days: +88 +0.1%
Commit velocity, last 7 days: +160 +0.8%

Website GitHub

scikit-learn/scikit-learn

scikit-learn: machine learning in Python

AI dev

Stack

Python pytest Meson PEP 517 pip

GitHub topics

#data-analysis #data-science #machine-learning #python #statistics

Updated: 2026-07-15
Lists: 3 list mentions
First commit: 2010-01-05
License: BSD-3-Clause
Issues: 2,111 open

66,688

stars

Forks: 27,177
Commits: 33,829 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

keras-team/keras

Deep Learning for humans

AI dev

Stack

Python pytest PEP 517 pip

GitHub topics

#data-science #deep-learning #jax #machine-learning #neural-networks #python

Updated: 2026-07-16
Lists: 4 list mentions
First commit: 2015-03-28
License: Apache-2.0
Issues: 212 open

64,169

stars

Forks: 19,739
Commits: 12,542 commits
Star growth, last 7 days: -18 0.0%
Commit velocity, last 7 days: +23 +0.2%

Website GitHub

apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

AI dev

Stack

Python Celery Cobra FastAPI Flask Go modules Gradle npm

GitHub topics

#airflow #apache #apache-airflow #automation #dag #data-engineering

Updated: 2026-07-17
Lists: 5 list mentions
First commit: 2014-10-06
License: Apache-2.0
Issues: 1,739 open

46,152

stars

Forks: 17,393
Commits: 39,742 commits
Star growth, last 7 days: +61 +0.1%
Commit velocity, last 7 days: +101 +0.3%

Website GitHub

milvus-io/milvus

Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search

AI dev

Stack

Go Axum Cobra Gin gRPC Go Cargo CMake Conan

GitHub topics

#anns #cloud-native #diskann #distributed #embedding-database #embedding-similarity

Updated: 2026-07-17
Lists: 7 list mentions
First commit: 2019-03-18
License: Apache-2.0
Issues: 1,101 open

45,249

stars

Forks: 4,123
Commits: 25,109 commits
Star growth, last 7 days: +71 +0.2%
Commit velocity, last 7 days: +48 +0.2%

Website GitHub

ray-project/ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

AI dev

Stack

Python Celery FastAPI pytest React Maven npm PEP 517

GitHub topics

#data-science #deep-learning #deployment #distributed #hyperparameter-optimization #hyperparameter-search

Updated: 2026-07-16
Lists: 4 list mentions
First commit: 2016-02-07
License: Apache-2.0
Issues: 3,449 open

43,253

stars

Forks: 7,796
Commits: 31,066 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

getredash/redash

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

Stack

Python Flask pytest React npm pip pnpm

GitHub topics

#analytics #athena #bi #bigquery #business-intelligence #dashboard

Updated: 2026-07-09
Lists: 5 list mentions
First commit: 2013-10-25
License: BSD-2-Clause
Issues: 795 open

28,691

stars

Forks: 4,614
Commits: 7,975 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

taosdata/TDengine

High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios

Stack

C Gin gRPC Go pytest Cargo CMake Conan

GitHub topics

#bigdata #cloud-native #cluster #connected-vehicles #database #distributed

Updated: 2026-07-06
Lists: 2 list mentions
First commit: 2019-07-11
License: AGPL-3.0
Issues: 428 open

24,953

stars

Forks: 5,011
Commits: 85,319 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

plotly/dash

Data Apps & Dashboards for Python. No JavaScript Required.

AI dev

Stack

Python Celery FastAPI Flask pytest npm pip Yarn

GitHub topics

#ai #bioinformatics #charting #dash #dashboards #data-science

Updated: 2026-07-10
Lists: 2 list mentions
First commit: 2015-04-10
License: MIT
Issues: 543 open

24,307

stars

Forks: 2,307
Commits: 9,902 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

matplotlib/matplotlib

matplotlib: plotting with Python

Stack

Python pytest Tornado Meson npm uv

GitHub topics

#data-science #data-visualization #gtk #matplotlib #plotting #python

Updated: 2026-07-07
Lists: 3 list mentions
First commit: 2003-05-12
License: Unknown
Issues: 1,477 open

22,966

stars

Forks: 8,385
Commits: 54,882 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

spotify/luigi

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

Stack

Python pytest Tornado PEP 517 uv

GitHub topics

#hadoop #luigi #orchestration-framework #python #scheduling

Updated: 2026-07-01
Lists: 3 list mentions
First commit: 2011-11-17
License: Apache-2.0
Issues: 163 open

18,745

stars

Forks: 2,457
Commits: 4,318 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

AI dev

Stack

Python FastAPI Flask LangChain Next.js npm PEP 517 pip

GitHub topics

#analytics #dagster #data-engineering #data-integration #data-orchestrator #data-pipelines

Updated: 2026-07-06
Lists: 4 list mentions
First commit: 2018-04-30
License: Apache-2.0
Issues: 2,568 open

15,795

stars

Forks: 2,184
Commits: 27,770 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

yugabyte/yugabyte-db

YugabyteDB - the cloud native distributed SQL database for mission-critical applications.

AI dev

Stack

C Cobra Express FastAPI gRPC Go CMake Go modules Gradle

GitHub topics

#cloud-native #cpp #database #distributed-database #distributed-sql #distributed-sql-database

Updated: 2026-07-17
Lists: 2 list mentions
First commit: 2005-06-28
License: NOASSERTION
Issues: 8,056 open

10,412

stars

Forks: 1,287
Commits: 38,913 commits
Star growth, last 7 days: +11 +0.1%
Commit velocity, last 7 days: +99 +0.3%

Website GitHub

YelpArchive/elastalert

Easy & Flexible Alerting With ElasticSearch

Stack

Python pytest pip

Updated: 2024-08-07
Lists: 1 list mention
First commit: 2015-02-19
License: Apache-2.0
Issues: 1,401 open

7,988

stars

Forks: 1,690
Commits: 2,082 commits
Star growth, last 7 days: -2 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

feast-dev/feast

The Open Source Feature Store for AI/ML

AI dev

Stack

Python Astro Cobra FastAPI gRPC Go Go modules Maven npm

GitHub topics

#big-data #data-engineering #data-quality #data-science #feature-store #features

Updated: 2026-07-17
Lists: 2 list mentions
First commit: 2018-12-10
License: Apache-2.0
Issues: 391 open

7,135

stars

Forks: 1,368
Commits: 4,622 commits
Star growth, last 7 days: +8 +0.1%
Commit velocity, last 7 days: +6 +0.1%

Website GitHub

numenta/nupic-legacy

Numenta Platform for Intelligent Computing is an implementation of Hierarchical Temporal Memory (HTM), a theory of intelligence based strictly on the neuroscience of the neocortex.

Stack

Python pytest Tornado pip

GitHub topics

#artificial-intelligence #hierarchical-temporal-memory #machine-intelligence #neocortex

Updated: 2024-12-03
Lists: 1 list mention
First commit: 2013-04-05
License: MIT
Issues: 465 open

6,352

stars

Forks: 1,533
Commits: 6,627 commits
Star growth, last 7 days: +1 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

MaterializeInc/materialize

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

AI dev

Stack

Rust Axum FastAPI pytest React Cargo npm PEP 517

GitHub topics

#cdc #data-mesh #data-store #database #distributed-systems #kafka

Updated: 2026-07-17
Lists: 2 list mentions
First commit: 2017-12-05
License: NOASSERTION
Issues: Disabled

6,334

stars

Forks: 512
Commits: 50,232 commits
Star growth, last 7 days: +6 +0.1%
Commit velocity, last 7 days: +90 +0.2%

Website GitHub

aimhubio/aim

Aim 💫 — An easy-to-use & supercharged open-source experiment tracker.

Stack

Python FastAPI pytest React npm PEP 517 pip

GitHub topics

#ai #data-science #data-visualization #experiment-tracking #machine-learning #metadata

Updated: 2026-07-09
Lists: 4 list mentions
First commit: 2019-05-31
License: Apache-2.0
Issues: 466 open

6,182

stars

Forks: 400
Commits: 2,265 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

d2iq-archive/marathon

Deploy and manage containers (including Docker) on top of Apache Mesos at scale.

Archived

Stack

Scala pytest Bundler npm pip

GitHub topics

#dcos #dcos-orchestration-guild

Updated: 2022-09-08
Lists: 1 list mention
First commit: 2013-06-25
License: Apache-2.0
Issues: 29 open

4,031

stars

Forks: 831
Commits: 6,969 commits
Star growth, last 7 days: -1 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

mara/mara-pipelines

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

Stack

Python pytest PEP 517 pip

GitHub topics

#data #data-integration #etl #pipeline #postgresql #python

Updated: 2023-12-15
Lists: 1 list mention
First commit: 2018-04-08
License: MIT
Issues: 26 open

2,089

stars

Forks: 100
Commits: 172 commits
Star growth, last 7 days: +1 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

Hydrospheredata/mist

Serverless proxy for Spark cluster

Stack

Scala pytest pip

GitHub topics

#apache-spark #api #big-data #serverless

Updated: 2026-04-13
Lists: 2 list mentions
First commit: 2016-02-01
License: Apache-2.0
Issues: 32 open

325

stars

Forks: 69
Commits: 2,020 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

linkedin/isolation-forest

A distributed Spark/Scala implementation of the isolation forest and extended isolation forest algorithms for unsupervised outlier detection, featuring support for scalable training and ONNX export for easy cross-platform inference.

Stack

Scala pytest Gradle PEP 517 pip

GitHub topics

#anomaly-detection #isolation-forest #linkedin #machine-learning #onnx #outlier-detection

Updated: 2026-06-12
Lists: 2 list mentions
First commit: 2019-08-12
License: NOASSERTION
Issues: 1 open

260

stars

Forks: 53
Commits: 101 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

Activity

Default branch: master
Last pushed: 2026-05-19
GitHub updated: 2026-07-17
Created: 2014-07-04
First commit: -
Last scanned: 2026-07-17 10:49
Watchers: 821

Indexed repo mix

Repo stars: 1,493,476
Repo forks: 351,988
Active: 150
Archived: 33

Languages

Java (44) Go (25) Python (23) C++ (18) JavaScript (15) Scala (15) C (8) Erlang (5) TypeScript (5) Rust (4) Clojure (3) C# (2)

Awesome Bigdata

Tracked list growth

Likes history

Commits history

Indexed repositories

Filter this list

Put your repository first

How it works

Pricing