9.3k repos tracked
updated 2d ago
Most starred

Data

Pipelines, warehouses, orchestration and analytics engineering.

Trending New Most starred 580 repositories · topics: data-engineering, etl, data-pipeline, database, duckdb
  1. 1
    The Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.
    TypeScript created 7y ago pushed 2d ago aialternativeauth
    +39/day
    measured
    ★ 114.6k
    60
  2. 2
    The fastest path to AI-powered full stack observability, even for lean teams.
    Go created 13y ago pushed 2d ago aialertingcncf
    +11/day
    measured
    ★ 81.3k
    50
  3. 3
    The 30 Days of Python programming challenge is a step-by-step guide to learn the Python programming language in 30 days. This challenge may take more than 100 days. Follow your own pace. These videos may help too: https://www.youtube.com/channel/UC7PNRuno1rzYPb1xLa4yktw
    Python created 6y ago pushed 39d ago 30-days-of-pythondatadata-science
    +89/day
    measured
    ★ 78.2k
    64
  4. 4
    For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.
    C created 17y ago pushed 2d ago cachecachingdatabase
    +5.7/day
    measured
    ★ 77.2k
    45
  5. 5
    Apache Superset is a Data Visualization and Data Exploration Platform
    Python created 11y ago pushed 2d ago analyticsapacheapache-superset
    +9.1/day
    measured
    ★ 75.4k
    48
  6. 6
    Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
    Python created 3y ago pushed 2d ago batch-processingdata-analyticsdata-pipelines
    −4.0/day
    measured
    ★ 62k
    29
  7. 7
    A lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.
    Rust created 8y ago pushed 2d ago aiapiapp-search
    +12/day
    measured
    ★ 59.8k
    54
  8. 8
    A no-code RAG server that keeps answers in sync with your documents. Works with 8 vector databases. Built on Pathway.
    Python created 3y ago pushed 5d ago chromadbdocument-qadocument-search
    −4.3/day
    measured
    ★ 58.7k
    29
  9. 9
    Distributed reliable key-value store for the most critical data of a distributed system
    Go created 13y ago pushed 2d ago cncfconsensusdatabase
    +2.3/day
    measured
    ★ 52.4k
    39
  10. 10
    Free universal database tool and SQL client
    Java created 11y ago pushed 2d ago aidatabasedatabricks
    +9.9/day
    measured
    ★ 52.3k
    48
  11. 11
    ClickHouse® is a real-time analytics database management system
    C++ created 10y ago pushed 2d ago aianalyticsbig-data
    +13/day
    measured
    ★ 50.8k
    50
  12. 12
    The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
    Clojure created 11y ago pushed 2d ago analyticsbibusiness-intelligence
    +33/day
    measured
    ★ 50.2k
    66
  13. 13
    Learn how to develop, deploy and iterate on production-grade ML applications.
    Jupyter Notebook created 8y ago pushed 8mo ago data-engineeringdata-qualitydata-science
    +14/day
    measured
    ★ 49.9k
    46
  14. 14
    The Prisma ORM: write your application in TypeScript with human-readable queries instead of SQL. Host your database on Prisma Postgres and your app on Prisma Compute.
    TypeScript created 7y ago pushed 4d ago cockroachdbdatabasejavascript
    −1.6/day
    measured
    ★ 47.7k
    28
  15. 15
    Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
    Python created 11y ago pushed 2d ago airflowapacheapache-airflow
    +8.7/day
    measured
    ★ 47.4k
    47
  16. 16
    Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. Join the course here 👇🏼
    Jupyter Notebook created 5y ago pushed 23d ago coursedata-engineeringdbt
    +22/day
    measured
    ★ 46.7k
    53
  17. 17
    👩‍💻👨‍💻 Awesome cheatsheets for popular programming languages, frameworks and development tools. They include everything you should know in one single file.
    JavaScript created 9y ago pushed 21d ago backendbashcheatsheet
    +3.3/day
    measured
    ★ 46.6k
    41
  18. 18
    DuckDB is a high-performance analytical database system
    C++ created 8y ago pushed 2d ago analyticsdatabaseembedded-database
    +22/day
    measured
    ★ 42.8k
    54
  19. 19
    TiDB is built for agentic workloads that grow unpredictably, with ACID guarantees and native support for transactions, analytics, and vector search. No data silos. No noisy neighbors. No infrastructure ceiling.
    Go created 11y ago pushed 2d ago agentagent-contextagent-memory
    +3.5/day
    measured
    ★ 40.7k
    43
  20. 20
    Free, simple, and intuitive online database diagram editor and SQL generator.
    JavaScript created 3y ago pushed 6d ago databasedatabase-designdatabase-diagram
    +11/day
    measured
    ★ 40.2k
    49
  21. 21
    An open-source engine that turns continuous content streams into search and monitoring. Durable ingestion, hybrid search, alerts, and plugins for formats, models and business rules.
    Go created 3y ago pushed 5d ago aiapichatbot
    +2.4/day
    measured
    ★ 39.6k
    43
  22. 22
    Automatic SQL injection and database takeover tool
    Python created 14y ago pushed 3d ago api-securityappsecdatabase
    +7.3/day
    measured
    ★ 38.9k
    46
  23. 23
    The flexible backend for all your projects 🐰 Turn your DB into a headless CMS, admin panels, or apps with a custom UI, instant APIs, auth & more.
    TypeScript created 13y ago pushed 2d ago apiappcms
    +15/day
    measured
    ★ 38.8k
    51
  24. 24
    📚 C/C++ 技术面试基础知识总结,包括语言、程序库、数据结构、算法、系统、网络、链接装载库等知识及面试经验、招聘、内推等信息。This repository is a summary of the basic knowledge of recruiting job seekers and beginners in the direction of C/C++ technology, including language, program library, data structure, algorithm, system, network, link loading library, interview experience, recruitment, recommendation, etc.
    C++ created 8y ago pushed 52d ago algorithmccpp
    +3.3/day
    measured
    ★ 38.3k
    50
  25. 25
    TypeScript & JavaScript ORM for Node.js — supports PostgreSQL, MySQL, MariaDB, SQLite, SQL Server, Oracle, and more.
    TypeScript created 10y ago pushed 3d ago active-recordcockroachdbdata-mapper
    +1.3/day
    measured
    ★ 36.7k
    50
  26. 26
    📗 SheetJS Spreadsheet Data Toolkit -- New home https://git.sheetjs.com/SheetJS/sheetjs
    created 13y ago pushed 2y ago angularbuncsv
    −3.0/wk
    measured
    ★ 36.3k
    13
  27. 27
    A scalable, distributed, collaborative, document-graph database, for the realtime web
    Rust created 4y ago pushed 4d ago backend-as-a-servicecloud-databasedatabase
    +5.1/day
    measured
    ★ 33.3k
    43
  28. 28
    CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
    Go created 12y ago pushed 3d ago cockroachdbdatabasedistributed-database
    +1.9/day
    measured
    ★ 32.6k
    37
  29. 29
    A library that provides an embeddable, persistent key-value store for fast storage.
    C++ created 13y ago pushed 3d ago databasestorage-engine
    +2.1/day
    measured
    ★ 32.2k
    38
  30. 30
    A modern replacement for Redis and Memcached
    C++ created 4y ago pushed 2d ago cachecppdatabase
    +2.1/day
    measured
    ★ 31.8k
    38
  31. 31
    Scalable datastore for metrics, events, and real-time analytics
    Rust created 13y ago pushed 3d ago databasegoinfluxdb
    +1.4/day
    measured
    ★ 31.8k
    35
  32. 32
    📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
    created 6y ago pushed 2y ago applied-data-scienceapplied-machine-learningcomputer-vision
    +7.9/day
    measured
    ★ 30.8k
    31
  33. 33
    Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
    Java created 7y ago pushed 2d ago ai-agentsautomationcontrol-plane
    +13/day
    measured
    ★ 30.1k
    50
  34. 34
    Search infrastructure for AI
    Rust created 4y ago pushed 8d ago agentsaiai-agents
    +6.0/day
    measured
    ★ 29.6k
    44
  35. 35
    25 MB lightweight cross-platform database client for 100+ databases, including MySQL, PostgreSQL, SQLite, Redis, MongoDB, DuckDB, SQL Server, and Dameng. Built-in AI, MCP Server, CLI, desktop and Docker. | 轻量级跨平台数据库管理工具,支持 MySQL、PostgreSQL、SQLite、Redis、MongoDB、达梦等 100+ 数据库,提供桌面端、Docker、CLI、内置 AI 助手和 MCP。
    Rust created 6mo ago pushed 2d ago aicliclickhouse
    +148/day
    measured
    ★ 29.2k
    71
  36. 36
    The MongoDB Database
    C++ created 17y ago pushed 3d ago c-plus-plusdatabasemongodb
    +1.6/day
    measured
    ★ 28.7k
    36
  37. 37
    Chat2DB is a free, cross-platform, local-first database client and SQL workspace for developers, DBAs, analysts, and data teams. Connect to 40+ databases, manage data, edit and run SQL, and use your own AI model to generate, explain, and optimize queries. Available on desktop, web, Docker, and CLI, with MCP support.
    Java created 3y ago pushed 6d ago aiclickhousedatabase
    +1.0/day
    measured
    ★ 28.4k
    33
  38. 38
    🔥🔥超过1000本的计算机经典书籍、个人笔记资料以及本人在各平台发表文章中所涉及的资源等。书籍资源包括C/C++、Java、Python、Go语言、数据结构与算法、操作系统、后端架构、计算机系统知识、数据库、计算机网络、设计模式、前端、汇编以及校招社招各种面经~
    created 5y ago pushed 12mo ago algorithmsccpp
    +21/day
    measured
    ★ 27.9k
    40
  39. 39
    REST API for any Postgres database
    Haskell created 12y ago pushed 3d ago apiautomatic-apidatabase
    +1.0/day
    measured
    ★ 27.7k
    34
  40. 40
    A flexible distributed key-value database that is optimized for caching and other realtime workloads.
    C created 2y ago pushed 2d ago cachedatabasekey-value
    +9.3/day
    measured
    ★ 27.6k
    48
  41. 41
    Development at the speed of light
    Rust created 3y ago pushed 2d ago databasedataorientedgame-development
    +16/day
    measured
    ★ 25.5k
    61
  42. 42
    High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios
    C created 7y ago pushed 3d ago bigdatacloud-nativecluster
    +1.1/day
    measured
    ★ 25.2k
    34
  43. 43
    A SQL database in Rust: SQLite-compatible, now also speaking Postgres (experimental). The LLVM of databases.
    Rust created 3y ago pushed 2d ago databaseembedded-databasesql
    +12/day
    measured
    ★ 25.2k
    49
  44. 44
    Dolt – Git for Data
    Go created 7y ago pushed 3d ago agent-memoryagent-memory-serverai-agents
    +8.3/day
    measured
    ★ 24.8k
    47
  45. 45
    Official home of the DB Browser for SQLite (DB4S) project. Previously known as "SQLite Database Browser" and "Database Browser for SQLite". Website at:
    C++ created 12y ago pushed 6d ago c-plus-pluscross-platformdatabase
    +1.4/day
    measured
    ★ 24.8k
    35
  46. 46
    Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
    Python created 8y ago pushed 2d ago automationdatadata-engineering
    +2.4/day
    measured
    ★ 24.1k
    38
  47. 47
    Chat with your database or your datalake (SQL, CSV, parquet). PandasAI makes data analysis conversational using LLMs and RAG.
    Python created 3y ago pushed 12mo ago aicsvdata
    +2.6/day
    measured
    ★ 23.9k
    24
  48. 48
    《Designing Data-Intensive Application》DDIA 第一版 / 第二版 中文翻译
    Python created 8y ago pushed 3d ago bookdatabaseddia
    +4.6/day
    measured
    ★ 23.9k
    44
  49. 49
    Modern and easy to use SQL client for MySQL, Postgres, SQLite, SQL Server, and more. Linux, MacOS, and Windows.
    TypeScript created 7y ago pushed 10d ago cockroachdbdatabasedatagrip-alternative
    —
    stale
    ★ 23.8k
    27
  50. 50
    A time-series database for high-performance real-time analytics packaged as a Postgres extension
    C created 9y ago pushed 2d ago analyticsdatabasefinancial-analysis
    +4.4/day
    measured
    ★ 23.8k
    42
page 1 of 12 next →