Data

Postgres rewritten in Rust, now faster than Postgres and Clickhouse

lakeFS - Data version control for your data lake | Git for data

Where data access meets operational intelligence

A lightweight data processing framework built on DuckDB and 3FS.

A user friendly TUI for SQL databases. Written in python. Supports SQL server, Mysql, PostreSQL, SQLite, Turso and more.

DeepAnalyze is the first agentic LLM for autonomous data science. 🎈你的AI数据分析师，自动分析大量数据，一键生成专业分析报告！

Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.

Privacy and Security focused Segment-alternative, in Golang and React

.NET aot orm, VB.NET/C# orm, Mysql/PostgreSQL/SqlServer/Oracle orm, Sqlite/Firebird/Clickhouse/DuckDB orm, 达梦/金仓/虚谷/翰高/高斯 orm, 神通 orm, 南大通用 orm, 国产 orm, TDengine orm, QuestDB orm, MsAccess orm.

Data Science Roadmap from A to Z

SQL Translator is a tool for converting natural language queries into SQL code using artificial intelligence. This project is 100% free and open source.

A system for agentic LLM-powered data processing and ETL

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

A list of useful resources to learn Data Engineering from scratch

ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

Maestro: Netflix’s Workflow Orchestrator

Blazing-fast, Neuro-symbolic Data-Wrangling toolkit

A curated list with resources about node-based UIs

SQL Database Explorer [SQLite, libSQL, PostgreSQL, MySQL/MariaDB, ClickHouse, DuckDB, Microsoft SQL Server]

A curated list of awesome ETL frameworks, libraries, and software.

A desktop application for viewing and analyzing tabular data

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

Open-source inference server and production cluster for all the models your agent needs.

Memphis.dev is a highly scalable and effortless data streaming platform

Learn how to design, develop, deploy and iterate on production-grade ML applications.

Scalable and efficient data transformation framework - backwards compatible with dbt.

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

An Awesome List of Open-Source Data Engineering Projects

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

The fastest business intelligence tool for humans and agents.

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

A new SOTA for RAG — an original retrieval architecture and an open-source knowledge base for humans and agents.

Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

Self-hosted database backup manager with a web UI. Schedule, backup, and restore MySQL, PostgreSQL, MariaDB, Microsoft SQL Server, MongoDB, SQLite & Redis to S3, SFTP, Samba or local storage. SSH Tunnel support.

Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends

Compile Time Async Dynamic SQL ORM

Data Contracts engine for the modern data stack. https://www.soda.io

A low code Machine Learning personalized ranking service for articles, listings, search results, recommendations that boosts user engagement. A friendly Learn-to-Rank engine

The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

Actively curated list of awesome BI tools. PRs welcome!

🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications

Datart is a next generation Data Visualization Open Platform

The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.

The modern replacement for Jupyter Notebooks

A lightweight stream processing library for Go

CLI task management & automation tool

A native, all-in-one workspace for databases, SSH, SFTP, terminals, remote desktop, monitoring, and AI.