Trending
Data
Pipelines, warehouses, orchestration and analytics engineering.
Trending
New
Most starred
551 repositories ·
topics: data-engineering, etl, data-pipeline, database, duckdb
-
201The Open Source Feature Store for AI/ML+5.0/wkmeasured★ 7.4k31
-
202Open-source search database for full-text, vector, and hybrid search with real-time indexing and SQL.+4.0/wkmeasured★ 12.1k31
-
203High-throughput synthetic and pretraining dataset sifter for TypeSafe Jev. Rust streaming core, Parquet and JSONL I/O, typed Choice/Score/Noul judgments, speculative fan-out, 24.0 rows/sec measured.+2.2/daylifetime avg★ 10731
-
204MacOS app written in Swift that bulk exports Apple Notes (including iCloud Notes) to a multitude of formats preserving note folder structure.+1.4/daymeasured★ 76430
-
205Scalable and efficient data transformation framework - backwards compatible with dbt.+6.0/wkmeasured★ 3.3k30
-
206A comprehensive guide to building a modern data warehouse with SQL Server, including ETL processes, data modeling, and analytics.+1.7/daymeasured★ 1.1k30
-
207Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.+3.0/wkmeasured★ 92030
-
208The First Real-Time Search Analytics Database+1.1/daymeasured★ 96130
-
209Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.+4.0/wkmeasured★ 8k30
-
210Open-source, self-hosted workspace for pages, databases and AI. One Node.js process with SQLite.+2.3/daymeasured★ 10230
-
211Native lakehouse experience in Postgres powered by DuckDB & DuckLake+2.0/wkmeasured★ 14930
-
212A new SOTA for RAG — an original retrieval architecture and an open-source knowledge base for humans and agents.+6.0/wkmeasured★ 2.7k30
-
213Learn how to design, develop, deploy and iterate on production-grade ML applications.+5.0/wkmeasured★ 3.4k30
-
214Free Open-source AI SQL client that helps developers and data teams explore databases using natural language.+4.0/wkmeasured★ 11130
-
215OpenFGA-to-PostgreSQL authorization compiler. Generates specialized SQL functions from .fga schemas that query your existing tables. Zero sync, always consistent.+4.0/wkmeasured★ 11130
-
216lakeFS - Data version control for your data lake | Git for data+4.0/wkmeasured★ 5.6k30
-
217Database migrations. CLI and Golang library.+1.3/daymeasured★ 19k29
-
218Hop Orchestration Platform+6.0/wkmeasured★ 1.5k29
-
219Data Contracts engine for the modern data stack. https://www.soda.io+5.0/wkmeasured★ 2.5k29
-
220Complete structured archive of every CIA World Factbook edition from 1990-2025. 281 countries, 36 years, 1M+ parsed data fields in SQL Server. Python ETL scripts included. Website created to allow users to explore the data without downloading. www.worldfactbookarchive.org+3.0/wkmeasured★ 11929
-
221A simple terminal UI for PostgreSQL, written in Go with Bubble Tea+1.0/daymeasured★ 32229
-
222ktx is an executable context layer for data and analytics agents 🐙 Allow Claude Code, Codex, or other AI agents to query analytical databases accurately and with full context of your company+4.0/wkmeasured★ 1.6k29
-
223A curated list of awesome dbt resources+3.0/wkmeasured★ 1.7k29
-
224Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.−4.0/daymeasured★ 62k29
-
225A distributed, fast open-source graph database featuring horizontal scalability and high availability+1.3/daymeasured★ 12.5k29
-
226Your entire engineering context, deeply understood+4.0/wkmeasured★ 1.5k29
-
227The local-first database that runs on every JS runtime and replicates with your existing backend - no vendor, no lock-in - https://rxdb.info/+1.0/wkmeasured★ 23.4k29
-
228ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data replication between MySQL, PostgreSQL, Redis, MongoDB, Kafka and ClickHouse, ideal for disaster recovery (DR) and migration scenarios.+2.0/wkmeasured★ 61229
-
229The database IDE for VS Code, Cursor, and Windsurf. 80+ databases: Postgres, MySQL, SQL Server, MongoDB, Snowflake, and more. AI queries, ER diagrams, SQL notebooks.+3.0/wkmeasured★ 36329
-
230The open document intelligence platform for builders and hackers - DMS for the agentic world+5.0/wkmeasured★ 1.5k29
-
231Flowfile is a visual ETL tool and Python library combining drag-and-drop workflows with Polars dataframes. Build data pipelines visually, define flows programmatically with a Polars-like API, and export to standalone Python code. Perfect for fast, intuitive data processing from development to production.+6.0/wkmeasured★ 41029
-
232A no-code RAG server that keeps answers in sync with your documents. Works with 8 vector databases. Built on Pathway.−4.3/daymeasured★ 58.7k29
-
233Fancy stream processing made operationally mundane+2.0/wkmeasured★ 8.8k29
-
234The Prisma ORM: write your application in TypeScript with human-readable queries instead of SQL. Host your database on Prisma Postgres and your app on Prisma Compute.−1.6/daymeasured★ 47.7k28
-
235Maestro: Netflix’s Workflow Orchestrator+2.0/wkmeasured★ 3.9k28
-
236First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.+2.0/wkmeasured★ 1.4k28
-
237🧙 Build, run, and manage data pipelines for integrating and transforming data.+3.0/wkmeasured★ 8.8k28
-
238Beginner data engineering project - batch edition+6.0/wkmeasured★ 61528
-
239Lightweight and extensible compatibility layer between dataframe libraries!+4.0/wkmeasured★ 1.7k28
-
240OLake - Fastest Databases, Kafka & S3 Replication to Apache Iceberg with Table optimization (Called OLake Fusion). ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Supported sources : Postgres, MongoDB, MySQL, Oracle, MSSql, DB2, Kafka, S3.+4.0/wkmeasured★ 1.6k28
-
241🔥 A well-tested feature-rich modular Firebase implementation for React Native. Supports both iOS & Android platforms for all Firebase services.+1.0/wkmeasured★ 12.3k28
-
242Curso para aprender los fundamentos del lenguaje SQL y bases de datos relacionales desde cero y para principiantes.+5.4/daymeasured★ 10.5k28
-
243Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.+3.0/wkmeasured★ 2.6k28
-
244An open source cybersecurity protocol for syncing decentralized graph data.+2.0/wkmeasured★ 19.2k28
-
245select * from logs; Tailpipe is an open source SIEM for instant log insights, powered by DuckDB. Analyze millions of events in seconds, right from your terminal.+1.0/wkmeasured★ 58528
-
246Free, open-source SQL Server and Postgres performance monitoring. Collectors, real-time alerts, graphical plan viewer, MCP server for AI analysis. Supports SQL 2016-2025, Azure SQL, AWS RDS, Postgres, Aurora Postgres.+3.0/wkmeasured★ 52828
-
247Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.flatmeasured★ 20.8k28
-
248A Scala 3 web framework. Direct-style. The case class is the source of truth. Deploy with one command.+1.2/daylifetime avg★ 11927
-
249Nornicdb is a distributed low-latency, Graph+Vector, Temporal MVCC with all sub-ms HNSW search, graph traversal, and writes. Using Neo4j Bolt/Cypher and qdrant's gRPC means you can switch with no changes while adding intelligent features like schemas, managed embeddings, reranking+llm, GPU accel, Auto-TLP, Policy-based Memory Decay, and MCP server.+4.0/wkmeasured★ 92427
-
250Dex lets your agent build your analytics.+1.1/daymeasured★ 11127