9.3k repos tracked
updated 2d ago
Trending

Data

Pipelines, warehouses, orchestration and analytics engineering.

Trending New Most starred 551 repositories · topics: data-engineering, etl, data-pipeline, database, duckdb
  1. 1
    High-performance AI pipeline engine with a C++ core and 50+ Python-extensible nodes. Build, debug, and scale LLM workflows with 13+ model providers, 8+ vector databases, and agent orchestration, all from your IDE. Includes VS Code extension, TypeScript/Python SDKs, and Docker deployment.
    Python created 8mo ago pushed 2d ago aicppdata-pipeline
    +180/day
    measured
    ★ 19.9k
    86
  2. 2
    TSP自托管、零运维的 A 股「自建策略 + 监控 + 回测」量化工作台 | LLM能力驱使策略定制+个股分析+复盘 | 自由接入第三方数据源与个性化扩展数据 | 个人开源
    Python created 4mo ago pushed 3d ago a-stockai-agentaigc
    +99/day
    measured
    ★ 7.1k
    79
  3. 3
    AI search engine and MCP server for bots and agents. One Rust binary indexes code, docs, logs, JSONL/CSV and PDFs for BM25 full-text, vector and hybrid search, code search without grep, agent memory and RAG. Elasticsearch-compatible API; Corpus Hub serves hundreds of reference memory corpora.
    Rust created 4mo ago pushed 2d ago agent-memoryai-agentsbm25
    +242/day
    measured
    ★ 19.2k
    73
  4. 4
    25 MB lightweight cross-platform database client for 100+ databases, including MySQL, PostgreSQL, SQLite, Redis, MongoDB, DuckDB, SQL Server, and Dameng. Built-in AI, MCP Server, CLI, desktop and Docker. | 轻量级跨平台数据库管理工具,支持 MySQL、PostgreSQL、SQLite、Redis、MongoDB、达梦等 100+ 数据库,提供桌面端、Docker、CLI、内置 AI 助手和 MCP。
    Rust created 6mo ago pushed 2d ago aicliclickhouse
    +148/day
    measured
    ★ 29.2k
    71
  5. 5
    The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
    Clojure created 11y ago pushed 2d ago analyticsbibusiness-intelligence
    +33/day
    measured
    ★ 50.2k
    66
  6. 6
    The 30 Days of Python programming challenge is a step-by-step guide to learn the Python programming language in 30 days. This challenge may take more than 100 days. Follow your own pace. These videos may help too: https://www.youtube.com/channel/UC7PNRuno1rzYPb1xLa4yktw
    Python created 6y ago pushed 39d ago 30-days-of-pythondatadata-science
    +89/day
    measured
    ★ 78.2k
    64
  7. 7
    Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
    Java created 7y ago pushed 2d ago analyticsbig-datadata-science
    +11/day
    measured
    ★ 13.5k
    63
  8. 8
    Development at the speed of light
    Rust created 3y ago pushed 2d ago databasedataorientedgame-development
    +16/day
    measured
    ★ 25.5k
    61
  9. 9
    The Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.
    TypeScript created 7y ago pushed 2d ago aialternativeauth
    +39/day
    measured
    ★ 114.6k
    60
  10. 10
    Database diagrams editor that allows you to visualize and design your DB with a single query.
    TypeScript created 2y ago pushed 6d ago databasedbdb-migration
    +7.3/day
    measured
    ★ 23.1k
    60
  11. 11
    LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
    Python created 2y ago pushed 2d ago ai-agentsdata-engineeringdocument-ai
    +9.7/day
    measured
    ★ 7.4k
    58
  12. 12
    A system for agentic LLM-powered data processing and ETL
    Python created 2y ago pushed 8d ago agentsdatadata-pipelines
    +7.7/day
    measured
    ★ 4.2k
    58
  13. 13
    VictoriaMetrics: fast, cost-effective monitoring solution and time series database
    Go created 8y ago pushed 2d ago databasegrafanagraphite
    +11/day
    measured
    ★ 18k
    57
  14. 14
    Take your agent-built product live: hosting, database, domain, email, payments — on your own accounts. Open-source Agent Skill + zero-dependency Node CLI: detect → plan → approve → apply → verify. No GoLive account, backend or telemetry.
    TypeScript created 45d ago pushed 27d ago agent-skillagent-skillsai-agents
    +10/day
    measured
    ★ 1.4k
    57
  15. 15
    A lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.
    Rust created 8y ago pushed 2d ago aiapiapp-search
    +12/day
    measured
    ★ 59.8k
    54
  16. 16
    DuckDB is a high-performance analytical database system
    C++ created 8y ago pushed 2d ago analyticsdatabaseembedded-database
    +22/day
    measured
    ★ 42.8k
    54
  17. 17
    Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼
    Jupyter Notebook created 5y ago pushed 23d ago coursedata-engineeringdbt
    +22/day
    measured
    ★ 46.7k
    53
  18. 18
    qData is an open-source data governance and data development platform that integrates ETL, data development, metadata management, data quality, data assets, API services, and AI-powered data Q&A.
    Java created 18mo ago pushed 2d ago chatbidata-governancedata-middle-platform
    +11/day
    measured
    ★ 811
    53
  19. 19
    Graph-Native Infrastructure for Context and Accountable AI Systems
    Python created 16mo ago pushed 2d ago agent-memoryaiai-governance
    +19/day
    measured
    ★ 14.4k
    52
  20. 20
    A lightweight, cloud-native GIS platform for visualizing, exploring, and analyzing geospatial data. It runs in the web browser, on the desktop, on mobile, and inside Jupyter notebooks.
    TypeScript created 5mo ago pushed 3d ago cesiumdata-scienceduckdb
    +20/day
    measured
    ★ 8.4k
    52
  21. 21
    The flexible backend for all your projects 🐰 Turn your DB into a headless CMS, admin panels, or apps with a custom UI, instant APIs, auth & more.
    TypeScript created 13y ago pushed 2d ago apiappcms
    +15/day
    measured
    ★ 38.8k
    51
  22. 22
    GenBI (Generative BI) for AI agents, an open-source, governed text-to-SQL through an open context layer that turns natural-language questions into trusted dashboards, charts, and SQL across 20+ data sources, such as BigQuery, Snowflake, PostgreSQL, ClickHouse, Amazon Redshift, Databricks and more.
    Python created 2y ago pushed 3d ago ai-agentsbigquerybusiness-intelligence
    +11/day
    measured
    ★ 18.1k
    51
  23. 23
    Servidor MCP e CLI para localizar empresas brasileiras por CNAE e região, utilizando dados abertos de CNPJ da Receita Federal.
    TypeScript created 6d ago pushed 5d ago cnaecnpjduckdb
    +36/day
    lifetime avg
    ★ 127
    51
  24. 24
    📚 C/C++ 技术面试基础知识总结,包括语言、程序库、数据结构、算法、系统、网络、链接装载库等知识及面试经验、招聘、内推等信息。This repository is a summary of the basic knowledge of recruiting job seekers and beginners in the direction of C/C++ technology, including language, program library, data structure, algorithm, system, network, link loading library, interview experience, recruitment, recommendation, etc.
    C++ created 8y ago pushed 52d ago algorithmccpp
    +3.3/day
    measured
    ★ 38.3k
    50
  25. 25
    ClickHouse® is a real-time analytics database management system
    C++ created 10y ago pushed 2d ago aianalyticsbig-data
    +13/day
    measured
    ★ 50.8k
    50
  26. 26
    TypeScript & JavaScript ORM for Node.js — supports PostgreSQL, MySQL, MariaDB, SQLite, SQL Server, Oracle, and more.
    TypeScript created 10y ago pushed 3d ago active-recordcockroachdbdata-mapper
    +1.3/day
    measured
    ★ 36.7k
    50
  27. 27
    Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
    Java created 7y ago pushed 2d ago ai-agentsautomationcontrol-plane
    +13/day
    measured
    ★ 30.1k
    50
  28. 28
    MCP Toolbox for Databases is an open source MCP server for databases.
    Go created 2y ago pushed 3d ago agentagentsai
    +8.1/day
    measured
    ★ 16.8k
    50
  29. 29
    The fastest path to AI-powered full stack observability, even for lean teams.
    Go created 13y ago pushed 2d ago aialertingcncf
    +11/day
    measured
    ★ 81.3k
    50
  30. 30
    A native, all-in-one workspace for databases, SSH, SFTP, terminals, remote desktop, monitoring, and AI.
    Rust created 4mo ago pushed 2d ago ai-agentsclickhousedameng
    +12/day
    measured
    ★ 2.1k
    50
  31. 31
    A SQL database in Rust: SQLite-compatible, now also speaking Postgres (experimental). The LLVM of databases.
    Rust created 3y ago pushed 2d ago databaseembedded-databasesql
    +12/day
    measured
    ★ 25.2k
    49
  32. 32
    Free, simple, and intuitive online database diagram editor and SQL generator.
    JavaScript created 3y ago pushed 6d ago databasedatabase-designdatabase-diagram
    +11/day
    measured
    ★ 40.2k
    49
  33. 33
    Open-source database & workspace for AI agents — structured data, durable knowledge, reusable skills, runnable apps, and human review on the writes that matter. Local-first and self-hostable.
    TypeScript created 4mo ago pushed 3d ago agent-workspaceagentic-aiai-agents
    +13/day
    measured
    ★ 708
    49
  34. 34
    Free universal database tool and SQL client
    Java created 11y ago pushed 2d ago aidatabasedatabricks
    +9.9/day
    measured
    ★ 52.3k
    48
  35. 35
    A flexible distributed key-value database that is optimized for caching and other realtime workloads.
    C created 2y ago pushed 2d ago cachedatabasekey-value
    +9.3/day
    measured
    ★ 27.6k
    48
  36. 36
    Apache Superset is a Data Visualization and Data Exploration Platform
    Python created 11y ago pushed 2d ago analyticsapacheapache-superset
    +9.1/day
    measured
    ★ 75.4k
    48
  37. 37
    PGX — Lightweight Postgres for AI agents. Written in Rust. Thousands of isolated databases in one runtime🐘
    Rust created 30d ago pushed 4d ago agentsai-infrastructureai-infrastructure-development
    +20/day
    measured
    ★ 677
    48
  38. 38
    Postgres rewritten in Rust, now faster than Postgres and Clickhouse
    Rust created 6mo ago pushed 12d ago ai-assisted-developmentdatabasepostgres
    +15/day
    measured
    ★ 5.8k
    48
  39. 39
    Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package
    Go created 13y ago pushed 8d ago databasegomariadb
    +1.1/day
    measured
    ★ 15.3k
    48
  40. 40
    Drone Log analyzer: A high-performance universal dashboard application for organizing and analyzing DJI/Litchi flight logs privately in one place. Supports plugin for custom flight log formats. Built with Tauri v2, DuckDB, and React.
    TypeScript created 9mo ago pushed 8d ago dashboarddata-analysisdata-visualization
    +2.0/day
    measured
    ★ 1.6k
    48
  41. 41
    Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
    Python created 11y ago pushed 2d ago airflowapacheapache-airflow
    +8.7/day
    measured
    ★ 47.4k
    47
  42. 42
    open-source agentic AI data assistant for the next generation of AI + Data products.
    Python created 3y ago pushed 3d ago agentsbgidatabase
    +6.7/day
    measured
    ★ 20.3k
    47
  43. 43
    Durable Streams API
    Rust created 12mo ago pushed 8d ago databasedurabledurable-streams
    +3.7/day
    measured
    ★ 811
    47
  44. 44
    the portable Python dataframe library
    Python created 11y ago pushed 3d ago bigqueryclickhousedatabase
    +1.6/day
    measured
    ★ 6.7k
    47
  45. 45
    Database manager for MySQL, PostgreSQL, SQL Server, MongoDB, SQLite and others. Runs under Windows, Linux, Mac or as web application
    JavaScript created 6y ago pushed 3d ago cassandraclickhousecockroachdb
    +2.7/day
    measured
    ★ 7.4k
    47
  46. 46
    Dolt – Git for Data
    Go created 7y ago pushed 3d ago agent-memoryagent-memory-serverai-agents
    +8.3/day
    measured
    ★ 24.8k
    47
  47. 47
    Browser-based data analytics & BI with DuckDB-WASM. Privacy-first: your data stays local; no server required.
    JavaScript created 2y ago pushed 19d ago business-analyticsbusiness-intelligencecsv
    +2.0/day
    measured
    ★ 633
    46
  48. 48
    A fast, multi-threaded Redis® and Valkey alternative written in Rust: drop-in compatible, with full-text and vector search built in
    Rust created 23d ago pushed 2d ago cachedatabasedragonfly-alternative
    +13/day
    measured
    ★ 934
    46
  49. 49
    Automatic SQL injection and database takeover tool
    Python created 14y ago pushed 3d ago api-securityappsecdatabase
    +7.3/day
    measured
    ★ 38.9k
    46
  50. 50
    Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
    Java created 10y ago pushed 2d ago apache-kafkacdcchange-data-capture
    +5.3/day
    measured
    ★ 13.3k
    46
page 1 of 12 next →