Search awesome repositories

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

LlamaIndex is the leading document agent and OCR platform

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

AKShare is an elegant and simple financial data interface library for Python, built for human beings! 开源财经数据接口库

🧙 Build, run, and manage data pipelines for integrating and transforming data.

A roadmap connecting many of the most important concepts in machine learning, how to learn them and what tools to use to perform them.

The leader in Customer Data Infrastructure

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers catalog.data.gov, open.canada.ca/data, data.humdata.org among many other sites.

Mimesis is a fast Python library for generating fake data in multiple languages.

DeepAnalyze is the first agentic LLM for autonomous data science. 🎈你的AI数据分析师，自动分析大量数据，一键生成专业分析报告！

A system for agentic LLM-powered data processing and ETL

Extract data from a wide range of Internet sources into a pandas DataFrame.

Deepnote is a drop-in replacement for Jupyter with an AI-first design, sleek UI, new blocks, and native data integrations. Use Python, R, and SQL locally in your favorite IDE, then scale to Deepnote cloud for real-time collaboration, Deepnote agent, and deployable data apps. https://deepnote.com/

Jupyter extensions that help you write code faster: Context aware AI Chat, Autocomplete, and Spreadsheet

Colour Science for Python

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

Malloy is a modern open source language for describing data relationships and transformations.

ArcticDB is a high performance, serverless DataFrame database built for the Python Data Science ecosystem.

AI code-writing assistant that understands data content

A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

The modern replacement for Jupyter Notebooks

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

An Internet-Scale Database.

In-memory tabular data in Julia