Awesome Bigdata

Apache Pulsar - distributed pub-sub messaging system

Apache Druid: a high performance real-time analytics database.

Gephi - The Open Graph Viz Platform

Apache Heron (Incubating) is a realtime, distributed, fault-tolerant stream processing engine from Twitter

Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

Web UI for PrestoDB.

A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

A Graph Traversal Language (no longer active - see Apache TinkerPop)

Secor is a service implementing Kafka log persistence

A large-scale entity and relation database supporting aggregation of properties

Distributed object store

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Fast scalable time series database

Elassandra = Elasticsearch + Apache Cassandra

HiBench is a big data benchmark suite.

An open source event analytics platform

SQL-based streaming analytics platform at scale

Twitter's collection of LZO and Protocol Buffer-related Hadoop, Pig, Hive, and HBase code.

Fast and reliable message broker built on top of Kafka.

Netflix's distributed Data Pipeline

📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)

🦎 A multi-protocol edge & service proxy. Seamlessly interface web apps, IoT clients, & microservices to Apache Kafka® via declaratively defined, stateless APIs.

A distributed system designed to ingest and process time series data

A flexible, partial, out-of-order and real-time typeahead search library

Distributed database specialized in exporting key/value data from Hadoop

Spring XD makes it easy to solve common big data problems such as data ingestion and export, real-time analytics, and batch workflow orchestration

An open-source columnar data format designed for fast & realtime analytic with big data.

No description.

An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration

Accumulo backed time series database

A Relational Database Backed by Apache Kafka

realtime search/indexing system

Scalable Machine Learning in Scalding

Hadoop log aggregator and dashboard

Haeinsa is linearly scalable multi-row, multi-table transaction library for HBase

Apache Tephra: Transactions for HBase.

RDF-Centric Map/Reduce Framework and Freebase data conversion tool

Statistical Workload Injector for MapReduce - Project at UC Berkeley AMP Lab

Hadoop Data Integration with various databases, ftp servers, salesforce. Incremental update, dedup, append, merge your data on Hadoop.

Mirror of Apache Slider

Tuple MapReduce for Hadoop: Hadoop API made easy

Develop streaming applications for IBM Streams in Python, Java & Scala.

DocId set compression and set operation library