Data

A low code Machine Learning personalized ranking service for articles, listings, search results, recommendations that boosts user engagement. A friendly Learn-to-Rank engine

Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.

A dbt adapter for Databricks.

MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.

A carefully crafted, thoroughly tested, optimized companion library for SQLAlchemy

Community-driven, simple, yet powerful framework for fast, cost-effective distributed Compute over Data.

A Python stream processing engine modeled after Yahoo! Pipes

SDK for robotics teams to verify the quality of their data used for AI model training.

Postgres, MySQL, or MariaDB to Elasticsearch/OpenSearch sync

Clean APIs for data cleaning. Python implementation of R package Janitor

The best place to learn data engineering. Built and maintained by the data engineering community.

Actively maintained successor to Alibaba DataX — a fast, versatile, open-source ETL tool for 30+ RDBMS and NoSQL data sources.

A Modern Embedded SQL Database written in Rust

Realm is a mobile database: a replacement for Core Data & SQLite

A local-first relational database for the browser, React Native and your backend.

Server application to serve U.S. federal spending data via a RESTful API

100% open-source, AI-native web analytics with cookieless tracking, open exports, custom domains, and SSO—self-hosted as one Go binary or managed in EU/US cloud.

A curated list of awesome system integration software and resources.

🍉 Reactive & asynchronous database for powerful React and React Native apps ⚡️

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

SQLAlchemy driver for DuckDB

🕵️Track prices of the Hetzner dedicated server auction

Egeria core

Infrastructure for agent-built apps and workflows

Workflow Automation Engine

ORM, but better

zerocode-tdd is a community-developed, free, open-source, outcome-driven automated testing framework for Data Pipelines, ETL, REST API, Kafka(Data Streams), Databases and Load scenarios. all defined in simple JSON or YAML — with zero coding.

A SQL transformation engine that type-checks your whole pipeline and catches breaking changes before they run — branches, replay, column-level lineage, compile-time contracts, per-model cost. Adapters: Databricks, Snowflake, BigQuery, DuckDB. Single static Rust binary. Apache 2.0.

Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.

Fast DuckDB-powered TypeScript library for tabular analysis, geospatial analysis, network graph analysis, vector similarity search, AI, Google Sheets, and data visualization.

Production-grade SQL gateway in front of DuckDB Quack + DuckLake + Iceberg. Multi-tenant pools, pluggable auth (DB/JWT/OIDC), table-level ACLs, role-aware routing, and a live admin console

Agentic SQL database manager for 40+ databases — PostgreSQL, MySQL, SQL Server, SQLite, DuckDB, ClickHouse, Oracle and more. Lightweight, privacy-first, AI-powered, cross-platform. Built with Tauri (Rust).

Self-hostable, privacy-focused website analytics.

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

Interactive Emacs database client for MySQL, PostgreSQL, SQLite, MongoDB, Redis, and JDBC.

Independent ground-up Rust reimplementation of SQLite with page-level MVCC concurrent writers (serializable snapshot isolation). Runs today in compatibility mode over standard SQLite files; RaptorQ erasure-coded self-healing durability is in-progress research, not yet a live guarantee.

The open-source pulse on software across Greece

Faucet is an open-source, single-binary server that turns any SQL database into a secure, governed REST API with role-based access control and native MCP support for applications and AI agents.

Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.

Genblaze is an open source Python SDK for orchestrating generative AI media pipelines across video, audio, and image providers with built in provenance for every output.

An Awesome List of Open-Source Data Engineering Projects

Practice Databricks coding skills with hands-on exercises. Import into Databricks Free Edition, write code, run assertions, check pass/fail. Covers Delta Lake, Spark SQL, PySpark, Auto Loader, medallion architecture, window functions, and more.

Open-source agentic ops platform for cloud, SRE, DevOps, support and FinOps teams. AI agents investigate incidents, cut cloud and Kubernetes waste and run approval-gated runbooks across Kubernetes, AWS, Azure, GCP and your databases. Self-hosted, bring your own models.

omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

ReplicaDB is open source tool for database replication, designed for efficiently transferring bulk data between relational and non-relational databases

A fast viewer for CSV/Parquet files and databases such as DuckDB, SQLite, PostgreSQL, MySQL, Clickhouse, etc., base on Tauri

A fast, native database client for PostgreSQL, MySQL and SQLite. Written in Rust with egui; runs on Linux, macOS and Windows.

Simple, easy-to-use database manager

A database client for PostgreSQL, SQLite, Redis, and ClickHouse, built with Tauri, React, and TypeScript.

A Spark plugin for reading and writing Excel files