Awesome

GitHub projects from awesome lists

Search awesome repositories

Search names, descriptions, topics, tags, and stacks, then tune results by ecosystem, freshness, health, and cross-list signal.

Continue with GitHub Browse awesome lists Request a list

Repos indexed: 18,423
Awesome lists tracked: 132
Current results: 15

Find repositories

Start broad, then narrow by ecosystem, freshness, health, and growth.

Clear 1 refinement

Search repositories

Search mode

Keyword Semantic

Tune results

The controls most people need first.

Awesome list

Language

Freshness

Sort

Direction

More filters Topics, generated tags, stack, files, age, archive status, and growth.

Ecosystem

GitHub topic

Generated tag

Framework or stack

Package manager

Files

Has file

Choose a suggestion or use commas to require multiple files.

Health

Minimum stars

Repository age

Uses known first-commit dates.

Archive status

AI development signals

Momentum

Unmaintained for

Commit velocity

Star growth

Reset filters

15 repos shown

Topic: data-processing

Browse

Highlighted

Open highlighted repo slot

Put your repository first

Promote a GitHub repo at the top of Awesome repository list views for 7 days.

pathwaycom/pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

Stack

Python Axum FastAPI LangChain pytest Cargo npm PEP 517

GitHub topics

#batch-processing #data-analytics #data-pipelines #data-processing #dataflow #etl

Updated: 2026-09-08
Lists: 4 list mentions
First commit: 2023-07-21
History: 10 history points
License: NOASSERTION
Issues: 35 open

62,319

stars

Forks: 1,685
Commits: 2,238 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

cocoindex-io/cocoindex

Incremental engine for long horizon agents 🌟 Star if you like it!

AI dev

Stack

Rust Astro Axum FastAPI pytest Cargo npm PEP 517

GitHub topics

#agentic-data-framework #ai #ai-agents #change-data-capture #codebase-intelligence #context-engineering

Updated: 2026-08-31
Lists: 2 list mentions
First commit: 2025-03-03
History: 1 history point
License: Apache-2.0
Issues: 76 open

11,461

stars

Forks: 894
Commits: 2,045 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

johnkerl/miller

Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

AI dev

Stack

Go Go modules pip

GitHub topics

#command-line #command-line-tools #csv #csv-format #data-cleaning #data-processing

Updated: 2026-09-14
Lists: 1 list mention
First commit: 2015-05-03
History: 6 history points
License: NOASSERTION
Issues: 72 open

10,018

stars

Forks: 243
Commits: 9,427 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

rocketride-org/rocketride-server

High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE. Includes VS Code extension, TypeScript/Python SDKs, and Docker deployment.

AI dev

Stack

Python FastAPI LangChain pytest React CMake npm PEP 517

GitHub topics

#ai #cpp #data-pipeline #data-processing #machine-learning #mcp

Updated: 2026-09-09
Lists: 2 list mentions
First commit: 2026-02-20
History: 1 history point
License: MIT
Issues: 355 open

8,437

stars

Forks: 2,737
Commits: 923 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

TomWright/dasel

Unified querying, transformation, and modification of JSON, TOML, YAML, XML, INI, HCL, KDL and CSV.

Stack

Go Go modules

GitHub topics

#cli #config #configuration #data-processing #data-structures #data-wrangling

Updated: 2026-08-01
Lists: 1 list mention
First commit: 2020-09-22
History: 8 history points
License: MIT
Issues: 23 open

8,010

stars

Forks: 173
Commits: 877 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

OpenDCAI/DataFlow

Easy Data Preparation with latest LLMs-based Operators and Pipelines.

Stack

Python FastAPI Flask Gradio pytest PEP 517 pip

GitHub topics

#data #data-agent #data-cleaning #data-pipelines #data-processing #data-science

Updated: 2026-08-18
Lists: 2 list mentions
First commit: 2024-10-13
History: 1 history point
License: Apache-2.0
Issues: 9 open

8,003

stars

Forks: 1,098
Commits: 1,194 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

datajuicer/data-juicer

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

Stack

Python FastAPI pytest Streamlit uv

GitHub topics

#data #data-analysis #data-pipeline #data-processing #data-science #data-visualization

Updated: 2026-09-09
Lists: 2 list mentions
First commit: 2023-08-01
History: 7 history points
License: Apache-2.0
Issues: 59 open

7,036

stars

Forks: 425
Commits: 628 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

NVIDIA/DALI

A GPU-accelerated library containing highly optimized building blocks and an execution engine for data processing to accelerate deep learning training and inference applications.

AI dev

Stack

C++ CMake PEP 517 pip

GitHub topics

#audio-processing #data-augmentation #data-processing #deep-learning #fast-data-pipeline #gpu

Updated: 2026-09-09
Lists: 1 list mention
First commit: 2018-06-01
History: 7 history points
License: Apache-2.0
Issues: 218 open

5,757

stars

Forks: 677
Commits: 4,363 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

deepseek-ai/smallpond

A lightweight data processing framework built on DuckDB and 3FS.

Stack

Python pytest PEP 517

GitHub topics

#data-processing #duckdb

Updated: 2025-03-05
Lists: 1 list mention
First commit: 2025-02-25
History: 2 history points
License: MIT
Issues: 34 open

5,004

stars

Forks: 452
Commits: 3 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

GitHub

unionai-oss/pandera

A light-weight, flexible, and expressive statistical data testing library

AI dev

Stack

Python FastAPI pytest pip uv

GitHub topics

#assertions #data-assertions #data-check #data-cleaning #data-processing #data-validation

Updated: 2026-09-09
Lists: 3 list mentions
First commit: 2018-11-01
History: 11 history points
License: MIT
Issues: 423 open

4,451

stars

Forks: 440
Commits: 1,240 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

numaproj/numaflow

Kubernetes-native platform to run massively parallel data/streaming jobs

Stack

Rust Axum Cobra Gin gRPC Go Cargo Go modules npm

GitHub topics

#data-processing #hacktoberfest #k8s #kubernetes #map-reduce #pipeline

Updated: 2026-09-03
Lists: 1 list mention
First commit: 2022-05-20
History: 7 history points
License: Apache-2.0
Issues: 287 open

2,828

stars

Forks: 172
Commits: 1,886 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

datachain-ai/datachain

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

AI dev

Stack

Python pytest PEP 517

GitHub topics

#ai-agents #claude-code #codex #data-context-layer #data-processing #harness-engineering

Updated: 2026-09-11
Lists: 1 list mention
First commit: 2024-06-25
History: 43 history points
License: Apache-2.0
Issues: 103 open

2,819

stars

Forks: 158
Commits: 1,168 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

pyper-dev/pyper

Concurrent Python made simple

Stack

Python pytest Bundler PEP 517 pip

GitHub topics

#asyncio #concurrency #data #data-collection #data-engineering #data-pipelines

Updated: 2025-02-04
Lists: 0 list mentions
First commit: 2024-05-22
History: 42 history points
License: MIT
Issues: 5 open

1,517

stars

Forks: 32
Commits: 115 commits
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

GitHub

svenkreiss/pysparkling

A pure Python implementation of Apache Spark's RDD and DStream interfaces.

Stack

Python pytest Tornado pip

GitHub topics

#apache-spark #data-processing #data-science #python

Updated: 2024-09-03
Lists: 1 list mention
First commit: 2015-05-09
History: 8 history points
License: NOASSERTION
Issues: 11 open

270

stars

Forks: 45
Commits: 1,534 commits
Star growth, last 7 days: No 7-day history
Commit velocity, last 7 days: No 7-day history

Website GitHub

bordanattila/OpenF1_tutorial

Interactive F1 strategy dashboard tutorial using the OpenF1 API, Streamlit, Plotly, and pandas.

Stack

Python Streamlit pip

GitHub topics

#data-processing #data-visualization #f1 #formula1 #openf1 #plotly

Updated: 2025-05-29
Lists: 0 list mentions
First commit: 2025-05-29
History: 105 history points
License: Unknown
Issues: 1 open

stars

Forks: 19
Commits: 1 commit
Star growth, last 7 days: 0 0.0%
Commit velocity, last 7 days: 0 0.0%

Website GitHub

Search awesome repositories

Find repositories

Put your repository first

How it works

Pricing

How it works

Pricing