9.3k repos tracked
updated 2d ago
Trending

Data

Pipelines, warehouses, orchestration and analytics engineering.

Trending New Most starred 551 repositories · topics: data-engineering, etl, data-pipeline, database, duckdb
  1. 351
    AI agent tooling for data engineering workflows.
    Python created 9mo ago pushed 5d ago agentic-workflowagentsai
    flat
    measured
    ★ 458
    22
  2. 352
    AgentSeek is an application development lifecycle toolkit for AI ecosystem apps, built by OceanBase OSS Team.
    Python created 8mo ago pushed 30d ago agentic-aiagentsai
    +2.0/wk
    measured
    ★ 204
    22
  3. 353
    DuckDB extension for querying, modifying, and storing data through Apache Arrow Flight servers.
    C++ created 2y ago pushed 16d ago apache-arrowarrowarrow-flight
    +1.0/wk
    measured
    ★ 351
    22
  4. 354
    loads CSV, TSV, LTSV, JSON, JSONL, Parquet, XLSX, ACH, and Fedwire files into SQLite; includes prep and frame for cleanup and in-memory transforms
    Go created 14mo ago pushed 4d ago achcsvdata-preprocessing
    flat
    measured
    ★ 383
    22
  5. 355
    DuckDB Foreign Data Wrapper for PostgreSQL
    C created 6y ago pushed 5d ago duckdbduckdb-fdwfdw
    −1.0/wk
    measured
    ★ 415
    22
  6. 356
    Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.
    Go created 5y ago pushed 2y ago airflowanalyticsanalytics-engineering
    +1.0/wk
    measured
    ★ 769
    22
  7. 357
    Discover, mask, and verify sensitive data in SQL databases — an auditable scan → mask → validate workflow for safe database copies.
    Python created 5mo ago pushed 4d ago anonymizationdata-engineeringdata-governance
    +1.0/wk
    measured
    ★ 150
    22
  8. 358
    DataMate is an enterprise-level data processing platform designed for model fine-tuning and RAG retrieval.
    Python created 15mo ago pushed 4d ago data-evaluationdata-pipelinedata-synthesis
    flat
    measured
    ★ 374
    22
  9. 359
    Established strong JEV based natural language to SQL performance and well-covered semantic operators for PostgreSQL with optimized cost and latency. Reusable evidence, with native execution, parallel plans and explainable embeddings in development preview.
    Python created 45d ago pushed 13d ago databaseembeddingsjev
    +1.0/wk
    measured
    ★ 260
    22
  10. 360
    A minimal DuckDB Wasm build for browsers and serverless environments like Cloudflare Workers
    TypeScript created 10mo ago pushed 13d ago cloudflare-workersduckdbemscripten
    +1.0/wk
    measured
    ★ 270
    22
  11. 361
    🚀 GizmoSQL — High-Performance Database Server
    C++ created 2y ago pushed 5d ago adbcapache-arrowapache-arrow-flight
    flat
    measured
    ★ 376
    22
  12. 362
    A live database structure viewer for Laravel that renders your schema as a scrollable, zoomable ER diagram. Structure only, never data.
    PHP created 3mo ago pushed 3d ago databasedatabase-schemadeveloper-tools
    flat
    measured
    ★ 292
    22
  13. 363
    Datos públicos de Chile curados, normalizados y validados: geografía, demografía, economía, salud, educación y distritos electorales. Listos para consumir en una línea de código con Polars, DuckDB, SQLite y Excel.
    Python created 5mo ago pushed 5d ago chiledata-engineeringdatasets
    +1.0/wk
    measured
    ★ 137
    22
  14. 364
    DuckDB extension for serving SQL queries over HTTP and returning JSON results.
    C++ created 2y ago pushed 16d ago apiapi-servercommunity-extension
    +1.0/wk
    measured
    ★ 285
    22
  15. 365
    DuckDB extension for Microsoft SQL Server (TDS + TLS), with catalog integration and pushdown.
    C++ created 9mo ago pushed 5d ago duckdbduckdb-extensionhugr
    +1.0/wk
    measured
    ★ 136
    22
  16. 366
    ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
    C# created 10y ago pushed 20d ago avrocinchoo-etlcsharp
    flat
    measured
    ★ 861
    22
  17. 367
    Linter for dangerous Postgres migration patterns in Diesel and SQLx. Prevents downtime caused by unsafe schema changes.
    Rust created 11mo ago pushed 4d ago clicode-reviewdatabase
    +1.0/wk
    measured
    ★ 122
    22
  18. 368
    Open-source text-to-SQL engine you tune and measure on your own database
    Python created 3mo ago pushed 4d ago data-agentsdatabasedatabricks
    +1.0/wk
    measured
    ★ 112
    22
  19. 369
    CLI task management & automation tool
    Python created 12y ago pushed 48d ago build-automationbuild-systembuild-tool
    +1.0/wk
    measured
    ★ 2.1k
    22
  20. 370
    The duckdb R package
    R created 3y ago pushed 15d ago databaseduckdbolap
    +1.0/wk
    measured
    ★ 231
    22
  21. 371
    A modern, open-source database client for desktop and web.
    Rust created 10mo ago pushed 32d ago databasedatabase-clientdbeaver-alternative
    +3.0/wk
    measured
    ★ 147
    22
  22. 372
    Genomic sequence storage, indexing and alignment inside PostgreSQL: https://bioseqdb.com/
    VHDL created 41d ago pushed 2d ago alignmentbioinformaticsbiology
    −1.0/wk
    measured
    ★ 207
    22
  23. 373
    Turn any folder into a Notion-style database — 7 views, 17 column types, formulas, relations, rollups, and subtasks. All stored in plain Markdown frontmatter.
    TypeScript created 8mo ago pushed 2mo ago calendarchartsdatabase
    +2.0/wk
    measured
    ★ 155
    22
  24. 374
    A collection of tools for extracting FHIR resources and analytics services on top of that data.
    Jupyter Notebook created 6y ago pushed 4d ago analyticsapache-beamdata-pipeline
    flat
    measured
    ★ 228
    22
  25. 375
    ERD Builder Pro is a database design and documentation tool for developers. Build ERDs, flowcharts, notes, and drawings — all in one workspace
    TypeScript created 7mo ago pushed 3d ago codingdata-engineeringdata-visualization
    flat
    measured
    ★ 207
    22
  26. 376
    A lightweight local first SQL analytics tool. Get your data 🦆 in a row
    TypeScript created 20mo ago pushed 7d ago analyticsdataduckdb
    flat
    measured
    ★ 256
    22
  27. 377
    All-in-one intelligent assistant powered by LlamaIndex — RAG, GraphRAG, NL2SQL, Skills & Memory with multimodal support.
    Python created 7mo ago pushed 15d ago datadata-preparationdatabase
    −1.0/wk
    measured
    ★ 455
    22
  28. 378
    RaccoonX(DBCheck): Database Inspection Tool
    Python created 7mo ago pushed 3d ago aidatabaseinspection
    flat
    measured
    ★ 181
    22
  29. 379
    SQL Database Explorer [SQLite, libSQL, PostgreSQL, MySQL/MariaDB, ClickHouse, DuckDB, Microsoft SQL Server]
    Rust created 2y ago pushed 2mo ago clickhouseduckdblibsql
    +2.0/wk
    measured
    ★ 3.7k
    22
  30. 380
    A drop-in replacement for dplyr, powered by DuckDB for speed.
    R created 3y ago pushed 15d ago analyticsdataframedplyr
    flat
    measured
    ★ 399
    22
  31. 381
    Text-to-SQL access control: schema selection that gives the LLM only the tables the caller is allowed to read. Row-level-security-aware table pruning for LangChain, MCP and any SQL agent. Postgres, Oracle, MySQL, SQL Server. 75% fewer prompt tokens.
    Python created 2mo ago pushed 3d ago access-controlai-agentsdata-engineering
    −5.0/wk
    measured
    ★ 174
    21
  32. 382
    An open source, standard data file format for graph data storage and retrieval.
    Java created 4y ago pushed 28d ago big-datadata-orchestrationetl
    +1.0/wk
    measured
    ★ 380
    21
  33. 383
    Drop-in replacement for Apache Spark UI
    TypeScript created 3y ago pushed 20d ago apache-sparkbig-datadata-pipeline
    flat
    measured
    ★ 498
    21
  34. 384
    Kafka data generator and load testing tool - generate fake messages, simulate producers and consumers, test broker failures, and run chaos engineering scenarios
    Go created 10mo ago pushed 3d ago apache-kafkaapache-sparkchaos-engineering
    flat
    measured
    ★ 164
    21
  35. 385
    Mass processing data with a complete ETL for .net developers
    C# created 8y ago pushed 29d ago business-intelligencecsvcsv-parser
    flat
    measured
    ★ 813
    21
  36. 386
    A portable Datamart and Business Intelligence suite built with Docker, Dagster, dbt, DuckDB and Superset
    Python created 3y ago pushed 13d ago business-intelligencedagsterdata-engineering
    −1.0/wk
    measured
    ★ 266
    21
  37. 387
    Stream Postgres to Apache Iceberg on S3 via logical replication, queryable over the Postgres wire protocol.
    Go created 7mo ago pushed 16d ago duckdbicebergparquet
    flat
    measured
    ★ 310
    21
  38. 388
    A command-line tool for querying databases
    Rust created 9mo ago pushed 15d ago adbcapache-arrowcli
    flat
    measured
    ★ 300
    21
  39. 389
    Terminal database client with AI chat and an MCP server
    Go created 2mo ago pushed 14d ago auroraclicockroachdb
    +1.0/wk
    measured
    ★ 108
    21
  40. 390
    AI-native cross-platform database workbench for developers and data teams.
    TypeScript created 2mo ago pushed 3d ago aiai-assistantclickhouse
    −2.0/wk
    measured
    ★ 127
    21
  41. 391
    A 股、期货、期权数据本地化管道:Tushare Pro 拉取 → Parquet 分区存储 → DuckDB 本地查询,支持增量同步与定时调度
    Python created 6mo ago pushed 24d ago a-shareapschedulerchina-stock
    +1.0/wk
    measured
    ★ 197
    21
  42. 392
    Microsoft Fabric Unified Data Foundation with Databricks & Purview 2026
    HTML created 4mo ago pushed 3d ago ai-azd-templatesazd-templateazd-templates
    flat
    measured
    ★ 114
    21
  43. 393
    Focusing on building industry-leading ETL engines.
    Rust created 10mo ago pushed 11d ago data-collectordataprocessingetl
    flat
    measured
    ★ 187
    21
  44. 394
    Making DAG construction easier
    Python created 6y ago pushed 17d ago airflowdata-etldata-pipeline
    flat
    measured
    ★ 286
    21
  45. 395
    A type-safe Haskell SQL library
    Haskell created 11y ago pushed 31d ago duckdbhaskellorm
    flat
    measured
    ★ 637
    21
  46. 396
    Open Lakehouse Compute Fabric: an open-source lakehouse platform with Microsoft Fabric Compute Extension.
    Python created 27d ago pushed 19d ago apache-icebergapache-sparkdata-engineering
    −1.0/wk
    measured
    ★ 274
    21
  47. 397
    计算机基础(计算机网络/操作系统/数据库/Git...)面试问题全面总结,包含详细的follow-up question以及答案;全部采用【问题+追问+答案】的形式,即拿即用,直击互联网大厂面试;可用于模拟面试、面试前复习、短期内快速备战面试...
    created 6y ago pushed 2y ago computer-networkcomputer-sciencedatabase
    +4.0/wk
    measured
    ★ 10.3k
    21
  48. 398
    👶🏻 신입 개발자 전공 지식 & 기술 면접 백과사전 📖
    Java created 7y ago pushed 17mo ago algorithmcomputer-sciencecs
    +1.7/day
    measured
    ★ 17.7k
    21
  49. 399
    Git For database Systems
    Rust created 8mo ago pushed 12d ago agentic-aiagentsclickhouse
    flat
    measured
    ★ 163
    21
  50. 400
    Roblox DirectX 11 Graphics Boost 2026: Fix Lag and Rendering Issues on Windows
    HTML created 46d ago pushed 2d ago adonis-loadercompilercomposable
    flat
    measured
    ★ 70
    20
← previous page 8 of 12 next →