Data

Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Know your data better！Datavines is Next-gen Data Observability Platform, support metadata manage and data quality.

MacOS app written in Swift that bulk exports Apple Notes (including iCloud Notes) to a multitude of formats preserving note folder structure.

Supercharge BigQuery with BigFunctions

Compilation of high-profile real-world examples of failed machine learning projects

Utility that converts an .etl file containing a Windows network packet capture into .pcapng format.

The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.

Bindings and ADO.NET Provider for DuckDB

ER diagram generator. Paste CREATE TABLE statements and get a clean, interactive ERD — runs 100% in your browser, nothing uploaded

Transform, query, and download geospatial data on the web.

Open-source database & workspace for AI agents — structured data, durable knowledge, reusable skills, runnable apps, and human review on the writes that matter. Local-first and self-hostable.

VectorFlow is a high volume vector embedding pipeline that ingests raw data, transforms it into vectors and writes it to a vector DB of your choice.

📄 ⚙️ ETL processes for medical and scientific papers

数据流引擎是一款面向数据集成、数据同步、数据交换、数据共享、任务配置、任务调度的底层数据驱动引擎。数据流引擎采用管执分离、多流层、插件库等体系应对大规模数据任务、数据高频上报、数据高频采集、异构数据兼容的实际数据问题。

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch

PGX — Lightweight Postgres for AI agents. Written in Rust. Thousands of isolated databases in one runtime🐘

Fast and lightweight DB Client made in Rust and GPUI.

The open source agentic content system to make your contents alive. Self-hosted on any platform. ◑

SQL Notebook — Casual data exploration in SQL

The fully open-source DuckDB workbench that runs in your browser. SQL editor, notebooks, charts, AI assistant. No install, no signup, no backend — your data never leaves the tab.

The premier open source Data Quality solution

Self-hostable, privacy-focused website analytics.

A type-safe Haskell SQL library

Browser-based data analytics & BI with DuckDB-WASM. Privacy-first: your data stays local; no server required.

WhatTheDuck is an open-source web application built on DuckDB. It allows users to upload CSV and Parquet files, store them in tables, and perform SQL queries on the data.

Synmetrix – production-ready open source semantic layer on Cube

Secure memory management for AI Agents • Ensures data integrity • Reduces hallucinations • Maintains consistent long-term context

A curated list of open source tools used in analytics platforms and data engineering ecosystem

Simple, easy-to-use database manager

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

A MCP (Model Context Protocol) server for interacting with dbt.

Beginner data engineering project - batch edition

The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.

ApeCloud's Data Transfer Suite, written in Rust. Provides ultra-fast data replication between MySQL, PostgreSQL, Redis, MongoDB, Kafka and ClickHouse, ideal for disaster recovery (DR) and migration scenarios.

Yet another redundant workflow engine

🍁 Sycamore is an LLM-powered search and analytics platform for unstructured data.

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)

Declarative YAML around SQL for dbt-native dashboards: easy for agents to write, easy for humans to audit.

A list about Apache Kafka

Unified MySQL, Postgres & FlightSQL Server, Powered by DuckDB.

A simplified, lightweight ETL Framework based on Apache Spark

Augmentation pipeline for rendering synthetic paper printing, faxing, scanning and copy machine processes

select * from logs; Tailpipe is an open source SIEM for instant log insights, powered by DuckDB. Analyze millions of events in seconds, right from your terminal.

A fast viewer for CSV/Parquet files and databases such as DuckDB, SQLite, PostgreSQL, MySQL, Clickhouse, etc., base on Tauri

A hackable data integration & analysis tool to enable non technical users to edit data processing jobs and visualise data on demand.

SmartCode = IDataSource -> IBuildTask -> IOutput => Build Everything!!!

Local-first Apple Health MCP server. Lets AI assistants answer questions about your sleep, workouts, activity and heart data from a local export.

基于web版kettle开发的一套分布式综合调度,管理,ETL开发的用户专业版B/S架构工具

A curated list of awesome system integration software and resources.

Data ETL & Analysis on the dataset 'Baby Names from Social Security Card Applications - National Data'.