Data

企业数据分析、统计分析、数据可视化与管理报告工作台｜Analysis Lab、可追溯证据、FastAPI、Next.js、Windows。

Practical Data Engineering: A Hands-On Real-Estate Project Guide

📗 SheetJS Spreadsheet Data Toolkit -- New home https://git.sheetjs.com/SheetJS/sheetjs

Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.

Roadmap to becoming a data engineer in 2021

Gel supercharges Postgres with a modern data model, graph queries, Auth & AI solutions, and much more.

Realm is a mobile database: a replacement for SQLite & ORMs

Augmentation pipeline for rendering synthetic paper printing, faxing, scanning and copy machine processes

Simple and fast JSON database

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

A curated collection of 300+ engineering blog articles from top tech companies. Learn how the best engineering teams solve real-world problems at scale.

The agent-first database

Formation PostgreSQL 18 | De débutant à avancé | 21 chapitres + 7 annexes | Architecture, SQL avancé, Performance, HA, Production | Français

A lightweight data processing framework built on DuckDB and 3FS.

Code for "Efficient Data Processing in Spark" Course

Open-source, low-latency key/value engine built on Valkey with query subscriptions and hierarchical storage tiers.

Data Science Roadmap from A to Z

More than 2000+ Data engineer interview questions.

Application-level quota management for Laravel with calendar periods, Eloquent integration, middleware, and multiple storage backends.

Streaming reactive and dataflow graphs in Python

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

Next-generation SQL static analyzer written in Rust. 282+ rules. Zero false positives. Security, performance, reliability, cost, compliance, quality. 14 SQL dialects. Offline. CI/CD ready.

End-to-end Data Lakehouse project built on Databricks, following the Medallion Architecture (Bronze, Silver, Gold). Covers real-world data engineering and analytics workflows using Spark, PySpark, SQL, Delta Lake, and Unity Catalog. Designed for learning, portfolio building, and job interviews.

A lightweight stream processing library for Go

Un repositorio más con conceptos básicos, desafíos técnicos y recursos sobre ingeniería de datos en español 🧙✨

SQL Translator is a tool for converting natural language queries into SQL code using artificial intelligence. This project is 100% free and open source.

Concurrent Python made simple

大数据采集,抽取平台,zdh_web是zdh系列服务的可视化管理平台，包含数据采集,调度,权限,审批流,私域营销等模块

Memphis.dev is a highly scalable and effortless data streaming platform

A curated list of open source tools used in analytics platforms and data engineering ecosystem

Extract a slice of your production database to reproduce bugs locally. Point dbslice at a record, it follows foreign keys and gives you a complete, importable subset

A graph database provides Cypher query, fluent builder and management UI implemented in Golang.

Datart is a next generation Data Visualization Open Platform

The modern replacement for Jupyter Notebooks

SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time).

An experimental embedded SQL engine in C++20. Query Parquet, CSV, JSON, Arrow, Avro, SQLite, and Excel files directly with SQL, in-process. Early-stage.

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

Use LLMs to robustly extract web data

Compilation of high-profile real-world examples of failed machine learning projects

Data processing & ETL framework for Ruby

Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks.

Example end to end data engineering project.

📖 Data Engineering from Raw Corpora

A hackable data integration & analysis tool to enable non technical users to edit data processing jobs and visualise data on demand.

Data ETL & Analysis on the dataset 'Baby Names from Social Security Card Applications - National Data'.

This repository is a getting started guide to Singer.

a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.

React components to build CSV files on the fly basing on Array/literal object of data

A curated, but incomplete, list of data-centric AI resources.

A Data Engineering & Machine Learning Knowledge Hub