Awesome Bigdata

Apache Superset is a Data Visualization and Data Exploration Platform

scikit-learn: machine learning in Python

Deep Learning for humans

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

A library for efficient similarity search and clustering of dense vectors.

High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

An orchestration platform for the development, production, and observation of data assets.

A lightweight, lightning-fast, in-process vector database

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.

The Open Source Feature Store for AI/ML

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

Aim 💫 — An easy-to-use & supercharged open-source experiment tracker.

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

chDB is an in-process OLAP SQL Engine 🚀 powered by ClickHouse

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

Open-source ETL/ELT on DuckDB. Write, wire, or draw one pipeline: 350+ components, 160+ connectors, dbt, CDC, data quality, a Python API, and MCP for AI agents. Runs anywhere, no lock-in.

A distributed Spark/Scala implementation of the isolation forest and extended isolation forest algorithms for unsupervised outlier detection, featuring support for scalable training and ONNX export for easy cross-platform inference.