Data

Visualize and share your data. All in SQL. Powered by DuckDB.

A Modern Embedded SQL Database written in Rust

Logical Replication extension for PostgreSQL 17, 16, 15, 14, 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.

🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with DuckDB or PostgreSQL

Home of the Open Data Contract Standard (ODCS).

React components to build CSV files on the fly basing on Array/literal object of data

A curated, but incomplete, list of data-centric AI resources.

ClickBench: a Benchmark For Analytical Databases

A Data Engineering & Machine Learning Knowledge Hub

Scratch is a swiss army knife for big data.

Open-source data contract testing for Snowflake, Databricks, BigQuery, Kafka & 15 more. Built on the Open Data Contract Standard (ODCS). MIT.

qStudio - Free SQL Analysis Tool

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

📙 Awesome Data Catalogs and Observability Platforms.

A comprehensive guide to building a modern data warehouse with SQL Server, including ETL processes, data modeling, and analytics.

Personal Data Engineering Projects

Open Source Business Intelligence Tool

zerocode-tdd is a community-developed, free, open-source, outcome-driven automated testing framework for Data Pipelines, ETL, REST API, Kafka(Data Streams), Databases and Load scenarios. all defined in simple JSON or YAML — with zero coding.

🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊

Ergonomic bindings to duckdb for Rust

FlyFish is a data visualization coding platform. We can create a data model quickly in a simple way, and quickly generate a set of data visualization solutions by dragging.

The First Real-Time Search Analytics Database

开源中国股市（A股）本地数据湖：沪深股票行情、A股股票数据与金融数据，面向量化研究、量化交易与回测，含财务数据、期货数据、资金流向、公告、指数行业与宏观。55 个日更数据集，复权与历史成分内置，行级溯源、PIT，MCP 原生。免注册、免 Token，自托管。Open-source China stock market data lake for quant research, backtesting and AI agents.

A fast, multi-threaded Redis® and Valkey alternative written in Rust: drop-in compatible, with full-text and vector search built in

A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!

SawitDB: The First Agricultural-Based Database with Anti-Corruption Protocols. Built for Data Sovereignty to oppose expensive, unreliable state infrastructure.

Egeria core

Nornicdb is a distributed low-latency, Graph+Vector, Temporal MVCC with all sub-ms HNSW search, graph traversal, and writes. Using Neo4j Bolt/Cypher and qdrant's gRPC means you can switch with no changes while adding intelligent features like schemas, managed embeddings, reranking+llm, GPU accel, Auto-TLP, Policy-based Memory Decay, and MCP server.

Sling is a CLI tool that extracts data from a source storage/database and loads it in a target storage/database.

Un repositorio más con conceptos básicos, desafíos técnicos y recursos sobre ingeniería de datos en español 🧙✨

🔥 🔥 🔥Open Source & AI driven Data Onboarding Platform:Free flatfile.com alternative

SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time).

Smarter data pipelines for audio.

Community-driven, simple, yet powerful framework for fast, cost-effective distributed Compute over Data.

Accumulated knowledge and experience in the field of Data Engineering

Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.

Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.

Polyglot workflows without leaving the comfort of your technology stack.

ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)

Workflow Automation Engine

Supplementary Materials for the The Complete dbt (Data Build Tool) Bootcamp Udemy course

Practical Data Engineering: A Hands-On Real-Estate Project Guide

Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.

Open-source agentic data engineering harness for dbt, SQL, and cloud warehouses. 100+ tools, 10 warehouses, AI-powered.

A carefully crafted, thoroughly tested, optimized companion library for SQLAlchemy

Mass processing data with a complete ETL for .net developers

qData is an open-source data governance and data development platform that integrates ETL, data development, metadata management, data quality, data assets, API services, and AI-powered data Q&A.

Durable Streams API

Data API Framework for AI Agents and Data Apps

Oxyde ORM is a type-safe, Pydantic-centric asynchronous ORM with a high-performance Rust core designed for clarity, speed, and reliability.