Search awesome repositories

Apache Spark - A unified analytics engine for large-scale data processing

Apache Flink

Open-source IoT Platform - Device management, data collection, processing and visualization.

The Data Engineering Cookbook

A distributed, fast open-source graph database featuring horizontal scalability and high availability

Open-Source Web UI for Apache Kafka Management

Cloud-native OSS search engine for observability

The most widely used Python to C compiler

Apache DataFusion SQL Query Engine

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

Apache Beam is a unified programming model for Batch and Streaming data processing.

H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

Arkime is an open source, large scale, full packet capturing, indexing, and database system.

The Open Source Feature Store for AI/ML

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is PostgreSQL-compatible, and based on Lucene.

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

A crazy fast analytical database, built on bitmaps. Perfect for ML applications. Learn more at: http://docs.featurebase.com/. Start a Docker instance: https://hub.docker.com/r/featurebasedb/featurebase

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.

A large-scale entity and relation database supporting aggregation of properties

Open source security data lake for threat hunting, detection & response, and cybersecurity analytics at petabyte scale on AWS

Titanoboa makes complex workflows easy. It is a low-code workflow orchestration platform for JVM - distributed, highly scalable and fault tolerant.

📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)

Synmetrix – production-ready open source semantic layer on Cube

DuckDB-powered data lake analytics from Postgres

The metrics layer for your data. Join us at https://metriql.com/slack

Serverless proxy for Spark cluster

PredictionIO Ruby SDK