Awesome

GitHub projects from awesome lists

Search awesome repositories

Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.

Continue with GitHub Browse awesome lists Request a list

Repos indexed: 18,423
Awesome lists tracked: 132
Current results: 18

Find repositories

Start broad, then narrow by ecosystem, freshness, health, and growth.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Awesome list

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

18 repos shown

Topic: data-pipelines

Browse

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

pathwaycom/pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Stack

Python Axum FastAPI LangChain pytest Cargo npm PEP 517

GitHub topics

#batch-processing #data-analytics #data-pipelines #data-processing #dataflow #etl

Updated: 2026-09-08
Lists: 4 list mentions
First commit: 2023-07-21
History: 10 history points
License: NOASSERTION
Issues: 35 open

62,319

stars

Forks: 1,685
Commits: 2,238 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

apache/airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

AI dev

Stack

Python Celery Cobra FastAPI Flask Go modules Gradle npm

GitHub topics

#airflow #apache #apache-airflow #automation #dag #data-engineering

Updated: 2026-09-16
Lists: 5 list mentions
First commit: 2014-10-06
History: 122 history points
License: Apache-2.0
Issues: 2,130 open

46,868

stars

Forks: 17,850
Commits: 41,131 commits
Star growth, last 7 days: +62 +0.1%
Commit velocity, last 7 days: +147 +0.4%

Website GitHub

dagster-io/dagster

An orchestration platform for the development, production, and observation of data assets.

AI dev

Stack

Python FastAPI Flask LangChain Next.js npm PEP 517 pip

GitHub topics

#analytics #dagster #data-engineering #data-integration #data-orchestrator #data-pipelines

Updated: 2026-08-22
Lists: 4 list mentions
First commit: 2018-04-30
History: 11 history points
License: Apache-2.0
Issues: 2,595 open

16,050

stars

Forks: 2,255
Commits: 27,882 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

Unstructured-IO/unstructured

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

Stack

HTML pytest pip uv

GitHub topics

#data-pipelines #deep-learning #document-image-analysis #document-image-processing #document-parser #document-parsing

Updated: 2026-09-10
Lists: 2 list mentions
First commit: 2022-06-29
History: 1 history point
License: Apache-2.0
Issues: 306 open

15,412

stars

Forks: 1,323
Commits: 1,928 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

apache/dolphinscheduler

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

AI dev

Stack

Java Spring Boot Vite Vue Maven npm pnpm

GitHub topics

#airflow #azkaban #cloud-native #data-pipelines #job-scheduler #orchestration

Updated: 2026-09-15
Lists: 1 list mention
First commit: 2019-03-01
History: 6 history points
License: Apache-2.0
Issues: 132 open

14,475

stars

Forks: 5,092
Commits: 9,072 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

kedro-org/kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

AI dev

Stack

Python FastAPI Jupyter pytest PEP 517 pip

GitHub topics

#agentic-ai #agentic-workflow #data-pipelines #hacktoberfest #kedro #machine-learning

Updated: 2026-09-09
Lists: 1 list mention
First commit: 2019-05-16
History: 7 history points
License: NOASSERTION
Issues: 138 open

10,989

stars

Forks: 1,074
Commits: 4,399 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

mage-ai/mage-ai

🧙 Build, run, and manage data pipelines for integrating and transforming data.

Stack

Python LangChain Next.js pytest React npm PEP 517 pip

GitHub topics

#artificial-intelligence #data #data-engineering #data-integration #data-pipelines #data-science

Updated: 2026-09-10
Lists: 1 list mention
First commit: 2022-05-16
History: 7 history points
License: Apache-2.0
Issues: 624 open

8,821

stars

Forks: 990
Commits: 5,790 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

OpenDCAI/DataFlow

Easy Data Preparation with latest LLMs-based Operators and Pipelines.

Stack

Python FastAPI Flask Gradio pytest PEP 517 pip

GitHub topics

#data #data-agent #data-cleaning #data-pipelines #data-processing #data-science

Updated: 2026-08-18
Lists: 2 list mentions
First commit: 2024-10-13
History: 1 history point
License: Apache-2.0
Issues: 9 open

8,003

stars

Forks: 1,098
Commits: 1,194 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

fluvio-community/fluvio

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

Stack

Rust Cargo

GitHub topics

#cloud-native #data-analytics #data-flow #data-integration #data-pipelines #distributed-systems

Updated: 2026-08-11
Lists: 2 list mentions
First commit: 2019-09-04
History: 4 history points
License: Apache-2.0
Issues: 140 open

5,244

stars

Forks: 531
Commits: 2,574 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

StructuredLabs/preswald

Preswald is a WASM packager for Python-based interactive data apps: bundle full complex data workflows, particularly visualizations, into single files, runnable completely in-browser, using Pyodide, DuckDB, Pandas, and Plotly, Matplotlib, etc. Build dashboards, reports, and notebooks that run offline, load fast, and share like a document.

Stack

Python FastAPI pytest React Tailwind CSS npm PEP 517

GitHub topics

#ai #analytics #analytics-engineering #copilot #data #data-applications

Updated: 2026-06-11
Lists: 2 list mentions
First commit: 2024-12-14
History: 2 history points
License: Apache-2.0
Issues: 330 open

4,274

stars

Forks: 647
Commits: 546 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

orchest/orchest

Build data pipelines, the easy way 🛠️

Archived

Stack

TypeScript Celery Cobra Flask gRPC Go Go modules npm PEP 517

GitHub topics

#airflow #cloud #dag #data-pipelines #data-science #deployment

Updated: 2023-06-06
Lists: 1 list mention
First commit: 2019-11-28
History: 4 history points
License: Apache-2.0
Issues: 130 open

4,131

stars

Forks: 260
Commits: 9,688 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

ucbepic/docetl

A system for agentic LLM-powered data processing and ETL

AI dev

Stack

Python FastAPI Next.js pytest React npm pip uv

GitHub topics

#agents #data #data-pipelines #document-analysis #document-processing #elt

Updated: 2026-09-05
Lists: 2 list mentions
First commit: 2024-07-09
History: 108 history points
License: MIT
Issues: 45 open

4,095

stars

Forks: 443
Commits: 918 commits
Star growth, last 7 days: +14 +0.3%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

Netflix/maestro

Maestro: Netflix’s Workflow Orchestrator

Stack

Java Spring Boot Gradle

GitHub topics

#agentic-workflow #analytics #automation #batch-processing #dag #data-engineering

Updated: 2026-09-03
Lists: 1 list mention
First commit: 2024-04-17
History: 7 history points
License: Apache-2.0
Issues: 58 open

3,834

stars

Forks: 311
Commits: 173 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

meltano/meltano

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

AI dev

Stack

Python pytest React Tailwind CSS npm PEP 517 pip

GitHub topics

#connectors #data #data-engineering #data-pipelines #dataops #dataops-platform

Updated: 2026-09-15
Lists: 1 list mention
First commit: 2018-04-27
History: 107 history points
License: MIT
Issues: 147 open

2,626

stars

Forks: 269
Commits: 12,976 commits
Star growth, last 7 days: +6 +0.2%
Commit velocity, last 7 days: +5 0.0%

Website GitHub

bruin-data/bruin

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

AI dev

Stack

Go Cobra Flask gRPC Go Vite Cargo Go modules npm

GitHub topics

#analytics #bigquery #data-analysis #data-ingestion #data-modeling #data-pipelines

Updated: 2026-09-10
Lists: 2 list mentions
First commit: 2023-07-25
History: 44 history points
License: Apache-2.0
Issues: 59 open

1,702

stars

Forks: 90
Commits: 8,417 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

pyper-dev/pyper

Concurrent Python made simple

Stack

Python pytest Bundler PEP 517 pip

GitHub topics

#asyncio #concurrency #data #data-collection #data-engineering #data-pipelines

Updated: 2025-02-04
Lists: 0 list mentions
First commit: 2024-05-22
History: 42 history points
License: MIT
Issues: 5 open

1,517

stars

Forks: 32
Commits: 115 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

amphi-ai/amphi-etl

visual data prep powered by python

AI dev

Stack

TypeScript pytest React Tornado npm PEP 517 pip

GitHub topics

#analytics-automation #data #data-analysis #data-pipelines #data-preparation #data-science

Updated: 2026-08-24
Lists: 1 list mention
First commit: 2024-03-21
History: 106 history points
License: NOASSERTION
Issues: 127 open

1,402

stars

Forks: 107
Commits: 549 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

Burla-Cloud/burla

The most efficient distributed computing framework.

AI dev

Stack

Python FastAPI pytest React Starlette Bun npm uv

GitHub topics

#batch-processing #data-pipelines #etl-pipelines #python

Updated: 2026-09-14
Lists: 0 list mentions
First commit: 2024-03-19
History: 105 history points
License: NOASSERTION
Issues: 2 open

258

stars

Forks: 11
Commits: 906 commits
Star growth, last 7 days: -1 -0.4%
Commit velocity, last 7 days: +3 +0.3%

Website GitHub

Search awesome repositories

Find repositories

Put your repository first

How it works

Pricing

How it works

Pricing