9.3k repos tracked
updated 2d ago
Trending

Data

Pipelines, warehouses, orchestration and analytics engineering.

Trending New Most starred 551 repositories · topics: data-engineering, etl, data-pipeline, database, duckdb
  1. 101
    Powerpipe: Dashboards for DevOps. Visualize cloud configurations. Assess security posture against a massive library of benchmarks. Build custom dashboards with code.
    TypeScript created 3y ago pushed 2d ago awsazurecis
    +2.0/wk
    measured
    ★ 526
    40
  2. 102
    high-performance graph database for real-time use cases
    Go created 11y ago pushed 4d ago databasedistributedgo
    +1.0/day
    measured
    ★ 21.8k
    40
  3. 103
    Harness for building data engineering agents designed around human control, correctness, and safety.
    Java created 5y ago pushed 3d ago apidata-engineeringdata-pipeline
    +3.0/wk
    measured
    ★ 235
    40
  4. 104
    TypeScript ETL toolkit for indexing Ethereum, Solana, and Substrate data, sourced from SQD Network.
    TypeScript created 4y ago pushed 4d ago blockchainblockchain-indexerethereum
    +3.0/wk
    measured
    ★ 1.3k
    40
  5. 105
    🔥🔥超过1000本的计算机经典书籍、个人笔记资料以及本人在各平台发表文章中所涉及的资源等。书籍资源包括C/C++、Java、Python、Go语言、数据结构与算法、操作系统、后端架构、计算机系统知识、数据库、计算机网络、设计模式、前端、汇编以及校招社招各种面经~
    created 5y ago pushed 12mo ago algorithmsccpp
    +21/day
    measured
    ★ 27.9k
    40
  6. 106
    Fast key-value DB in Go.
    Go created 9y ago pushed 18d ago databasedocument-databasego
    +1.0/day
    measured
    ★ 15.8k
    40
  7. 107
    Apache ZooKeeper
    Java created 17y ago pushed 3d ago apacheconfiguration-managementconsensus
    +6.0/wk
    measured
    ★ 12.8k
    39
  8. 108
    Database Reporting Tool and Tasks (.Net)
    C# created 11y ago pushed 3d ago ai-agentsai-toolsbusiness-intelligence
    +2.0/wk
    measured
    ★ 1.6k
    39
  9. 109
    ✨ AI Spreadsheet for Business
    TypeScript created 4y ago pushed 2d ago ai-agentairtableairtable-alternative
    +2.9/day
    measured
    ★ 22k
    39
  10. 110
    🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
    Rust created 6y ago pushed 3d ago change-data-capturedatadata-collection
    +2.0/wk
    measured
    ★ 987
    39
  11. 111
    High-assurance AI data visualization IDE & analytics workbench. Natural language to charts with local ground-truth table verification.
    Python created 40d ago pushed 5d ago ai-analyticsanalyticsbi
    +6.6/day
    lifetime avg
    ★ 249
    39
  12. 112
    Distributed reliable key-value store for the most critical data of a distributed system
    Go created 13y ago pushed 2d ago cncfconsensusdatabase
    +2.3/day
    measured
    ★ 52.4k
    39
  13. 113
    Workflow Engine for Kubernetes
    Go created 9y ago pushed 2d ago airflowargoargo-workflows
    +2.1/day
    measured
    ★ 17.1k
    38
  14. 114
    A database migration tool. Supports SQL migrations and Go functions.
    Go created 10y ago pushed 5d ago databasedatabase-migrationsgo
    +3.1/day
    measured
    ★ 11.7k
    38
  15. 115
    Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
    Rust created 5mo ago pushed 3d ago cdcconnectorsdata-engineering
    +2.4/day
    measured
    ★ 1.6k
    38
  16. 116
    Skills related to Data Engineering Work for Claude Code
    Shell created 9mo ago pushed 12d ago claudeclaude-codeclaude-skills
    +3.0/wk
    measured
    ★ 133
    38
  17. 117
    Privacy and Security focused Segment-alternative, in Golang and React
    Go created 7y ago pushed 2d ago bigquerycdpcustomer-data
    +1.0/day
    measured
    ★ 4.5k
    38
  18. 118
    Quilt is a Scientific Data Management Platform on AWS that helps teams and AI find, trust, and reuse data through deeply versioned, context-rich data packages.
    TypeScript created 9y ago pushed 3d ago datadata-engineeringdata-version-control
    +1.0/wk
    measured
    ★ 1.4k
    38
  19. 119
    Cognitive memory database for AI agents — consolidates duplicates, detects contradictions, fades stale memories via temporal decay. Rust, Apache-2.0, ships as library / MCP server / HTTP cluster.
    Rust created 7mo ago pushed 10d ago agent-memoryai-agentsanthropic
    +2.0/wk
    measured
    ★ 179
    38
  20. 120
    Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
    Python created 8y ago pushed 2d ago automationdatadata-engineering
    +2.4/day
    measured
    ★ 24.1k
    38
  21. 121
    Source-available semantic and context layer, rule engine, and semantic sidecar for agentic AI and governed analytics. Compiles declarative YAML models into optimized SQL, KPIs, and semantic context across 8 SQL dialects.
    Python created 9mo ago pushed 3d ago agentic-aianalytics-as-codebigquery
    +2.0/wk
    measured
    ★ 103
    38
  22. 122
    Self-hosted database backup manager with a web UI. Schedule, backup, and restore MySQL, PostgreSQL, MariaDB, Microsoft SQL Server, MongoDB, SQLite & Redis to S3, SFTP, Samba or local storage. SSH Tunnel support.
    PHP created 11mo ago pushed 3d ago backupbackup-restoredatabase
    +2.7/day
    measured
    ★ 2.6k
    38
  23. 123
    🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.
    Rust created 7y ago pushed 3d ago backenddatabasegraph
    +1.0/day
    measured
    ★ 21.4k
    38
  24. 124
    One database for user-user (follows), user-item (likes), and item-item (related items) interactions — precomputed at write time, served as simple lookups
    Kotlin created 12mo ago pushed 17d ago databasegraphdbhigh-throughput
    +2.0/wk
    measured
    ★ 230
    38
  25. 125
    Open Source Feature Flags, Experimentation, and Product Analytics
    TypeScript created 5y ago pushed 2d ago ab-testingabtestabtesting
    +2.9/day
    measured
    ★ 8.6k
    38
  26. 126
    A modern replacement for Redis and Memcached
    C++ created 4y ago pushed 2d ago cachecppdatabase
    +2.1/day
    measured
    ★ 31.8k
    38
  27. 127
    A library that provides an embeddable, persistent key-value store for fast storage.
    C++ created 13y ago pushed 3d ago databasestorage-engine
    +2.1/day
    measured
    ★ 32.2k
    38
  28. 128
    The data transformation tool the open-source community deserves.
    Python created 14d ago pushed 2d ago data-engineeringduckdb
    +5.3/day
    measured
    ★ 305
    37
  29. 129
    Graphs for Everyone
    Java created 13y ago pushed 16d ago cypherdatabasegraph
    +2.9/day
    measured
    ★ 17.4k
    37
  30. 130
    Every database. One studio. A free, open-source database GUI for SQL, document, and key-value databases.
    TypeScript created 11mo ago pushed 3d ago clickhousecloudflare-d1database-client
    +3.3/day
    measured
    ★ 310
    37
  31. 131
    DeepAnalyze is the first agentic LLM for autonomous data science. 🎈你的AI数据分析师,自动分析大量数据,一键生成专业分析报告!
    Python created 13mo ago pushed 5d ago agentagenticagentic-ai
    +3.1/day
    measured
    ★ 4.8k
    37
  32. 132
    The SQL IDE for Your Terminal. Now with hsql, your agent's favorite sql client.
    Python created 3y ago pushed 3d ago agentcliclient
    +2.3/day
    measured
    ★ 6.5k
    37
  33. 133
    Secure memory management for AI Agents • Ensures data integrity • Reduces hallucinations • Maintains consistent long-term context
    Rust created 8mo ago pushed 3d ago agentaidatabase
    +6.0/wk
    measured
    ★ 624
    37
  34. 134
    Open-source agentic data engineering harness for dbt, SQL, and cloud warehouses. 100+ tools, 10 warehouses, AI-powered.
    TypeScript created 8mo ago pushed 2d ago agentagentic-data-engineeringai
    +4.0/wk
    measured
    ★ 818
    37
  35. 135
    View SQLite databases
    Rust created 10mo ago pushed 15d ago databasegnomelinux
    +2.0/wk
    measured
    ★ 107
    37
  36. 136
    CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
    Go created 12y ago pushed 3d ago cockroachdbdatabasedistributed-database
    +1.9/day
    measured
    ★ 32.6k
    37
  37. 137
    Unified local observability for AI coding assistants
    Go created 10mo ago pushed 53d ago aiclaude-codecodex-cli
    +4.0/wk
    measured
    ★ 283
    37
  38. 138
    YugabyteDB - the cloud native distributed SQL database for mission-critical applications.
    C created 9y ago pushed 2d ago cloud-nativecppdatabase
    +2.3/day
    measured
    ★ 10.7k
    37
  39. 139
    Open-source inference server and production cluster for all the models your agent needs.
    Python created 3y ago pushed 2d ago bgecolbertdata-pipeline
    +2.9/day
    measured
    ★ 3.5k
    36
  40. 140
    👾 nao is the #1 open source agentic BI. (1) Create context with nao-core cli, (2) deploy nao chat interface for everyone
    TypeScript created 10mo ago pushed 2d ago agentic-analyticsanalyticsanalytics-engineering
    +3.1/day
    measured
    ★ 2k
    36
  41. 141
    Open Source Business Intelligence Tool
    Python created 4y ago pushed 3d ago analyticsbibusiness-intelligence
    +5.0/wk
    measured
    ★ 1k
    36
  42. 142
    Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
    Go created 5y ago pushed 3d ago dashboard-friendlydatadata-analysis
    +1.3/day
    measured
    ★ 3.2k
    36
  43. 143
    Rich MySQL Terminal Client with AutoCompletion, Syntax Highlighting, and Dataframes
    Python created 11y ago pushed 2d ago auto-completiondata-sciencedatabase
    +5.0/wk
    measured
    ★ 12k
    36
  44. 144
    Modern documentation site generator for dbt Core — lineage explorer, health scoring, full-text search. Live demo: https://demo.docglow.com
    Python created 8mo ago pushed 8d ago analytics-engineeringdata-catalogdata-engineering
    +2.0/wk
    measured
    ★ 153
    36
  45. 145
    Free Course : Data, Machine Learning, Deep Learning and AI Engineering, the long way round: the math, the code, the systems and the security. 173 lessons, 166 runnable labs, no shortcuts. Free under CC BY-NC 4.0.
    Python created 42d ago pushed 10d ago ai-engineeringcoursedata-engineering
    +5.0/day
    measured
    ★ 172
    36
  46. 146
    The MongoDB Database
    C++ created 17y ago pushed 3d ago c-plus-plusdatabasemongodb
    +1.6/day
    measured
    ★ 28.7k
    36
  47. 147
    Where data access meets operational intelligence
    Go created 2y ago pushed 3d ago ai-agentdata-catalogdata-governance
    +6.0/wk
    measured
    ★ 5.1k
    36
  48. 148
    A light TablePlus alternative for PostgreSQL that your AI agents can use too, over MCP. macOS and Windows.
    Rust created 10d ago pushed 7d ago ai-agentsclaude-codedatabase
    +5.9/day
    lifetime avg
    ★ 44
    35
  49. 149
    Scalable datastore for metrics, events, and real-time analytics
    Rust created 13y ago pushed 3d ago databasegoinfluxdb
    +1.4/day
    measured
    ★ 31.8k
    35
  50. 150
    Flink CDC is a streaming data integration tool
    Java created 6y ago pushed 3d ago batchcdcchange-data-capture
    +4.0/wk
    measured
    ★ 6.5k
    35
← previous page 3 of 12 next →