9.3k repos tracked
updated 2d ago
Trending

Data

Pipelines, warehouses, orchestration and analytics engineering.

Trending New Most starred 551 repositories · topics: data-engineering, etl, data-pipeline, database, duckdb
  1. 251
    A javascript library to run SQLite on the web.
    JavaScript created 14y ago pushed 12d ago databaseemscriptenjavascript
    +1.0/wk
    measured
    ★ 13.7k
    27
  2. 252
    :kangaroo: - PouchDB is a pocket-sized database.
    JavaScript created 16y ago pushed 3d ago couchdbdatabasehacktoberfest
    −1.0/wk
    measured
    ★ 17.6k
    27
  3. 253
    Query robotics, drone, and IoT data in plain English through an MCP server, with an intelligent edge data reduction pipeline that keeps only the data that matters.
    Python created 16mo ago pushed 3d ago ardupilotbetaflightclaude
    +1.0/wk
    measured
    ★ 401
    27
  4. 254
    QuackOSM: an open-source Python and CLI tool for reading OpenStreetMap PBF files using DuckDB
    Python created 2y ago pushed 3d ago duckdbgeogeospatial
    +1.0/wk
    measured
    ★ 380
    27
  5. 255
    Actively curated list of awesome BI tools. PRs welcome!
    created 9y ago pushed 2y ago awesome-listbusiness-intelligencedata-analysis
    +5.0/wk
    measured
    ★ 2.3k
    27
  6. 256
    Illuminate your data. Agent framework turning natural language into SQL, charts, dashboards and reports.
    Python created 6y ago pushed 2d ago ai-agentsconversational-aidata-exploration
    +3.0/wk
    measured
    ★ 326
    27
  7. 257
    从数据到智能:企业级数据平台的构建、演进与 Agentic BI 实践。一个数据开发的八年数据工程手记:从 0 到 1 构建企业级医药数据平台,再到 Data + AI 转型下的 Agentic BI。
    HTML created 4mo ago pushed 3mo ago agentic-bidata-agentdata-ai
    +2.0/wk
    measured
    ★ 140
    27
  8. 258
    A cognition aware database engine for AI agent memory. Purpose built in Rust with WAL, HNSW, knowledge graphs, and speculative context pre assembly. Not a wrapper, a ground up storage engine that thinks.
    Rust created 7mo ago pushed 3d ago agent-memoryaiai-agents
    +2.0/wk
    measured
    ★ 124
    27
  9. 259
    A data platform for investment research. Ingests financial data sources and computes point-in-time financial metrics for valuation and screening. Delivered as an MCP server.
    Python created 2mo ago pushed 5d ago alpha-researchbacktestingduckdb
    +5.0/wk
    measured
    ★ 329
    27
  10. 260
    WCDB is a cross-platform database framework developed by WeChat.
    C created 9y ago pushed 30d ago androiddatabaseios
    +2.0/wk
    measured
    ★ 11.6k
    27
  11. 261
    Supplementary Materials for the The Complete dbt (Data Build Tool) Bootcamp Udemy course
    Python created 4y ago pushed 34d ago analytics-engineeringdata-engineeringdbt
    +2.0/wk
    measured
    ★ 841
    27
  12. 262
    Autonomous backend platform. Your coding agent builds it over MCP, Backenly keeps it running: Postgres, REST APIs, auth, storage, realtime, and functions, with every change governed, verified, and reversible.
    TypeScript created 3mo ago pushed 5d ago agent-toolsai-agentsauthentication
    +1.0/day
    lifetime avg
    ★ 107
    26
  13. 263
    Declarative text based tool for data analysts and engineers to extract, load, transform and orchestrate their data pipelines.
    Scala created 5y ago pushed 3d ago bigquerydata-engineeringdata-integration
    +1.0/wk
    measured
    ★ 217
    26
  14. 264
    An async ORM for Python and TypeScript with one Rust core: one Prisma schema, typed models, migrations, PostgreSQL and SQLite.
    Rust created 37d ago pushed 7d ago asyncdatabasemigrations
    +1.3/day
    lifetime avg
    ★ 45
    26
  15. 265
    🐚 Python-powered shell. Full-featured, cross-platform and AI-friendly.
    Python created 11y ago pushed 4d ago anthropicclaudeclaude-code
    −2.0/wk
    measured
    ★ 9.7k
    26
  16. 266
    Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
    Rust created 5y ago pushed 4d ago aickancsv
    +1.0/wk
    measured
    ★ 3.8k
    26
  17. 267
    A fast, native, keyboard-driven database client for macOS. 20 databases, an AI assistant that shows its work, free and open source.
    Rust created 40d ago pushed 13d ago ai-assistantclickhousedatabase
    +5.0/wk
    measured
    ★ 419
    26
  18. 268
    Versioned DuckDB databases, locally or replicated through S3, for people and agents
    C++ created 29d ago pushed 5d ago agentsbranchingduckdb
    +1.2/day
    measured
    ★ 48
    26
  19. 269
    YARD is an editor plugin for Godot to manage resources and query them at runtime, with a table view in the editor as the cherry on top.
    GDScript created 10mo ago pushed 3d ago databasegame-developmentgame-engine
    +4.0/wk
    measured
    ★ 371
    26
  20. 270
    Support agile Ontology DataOps Based on Flink, DataX and Flink-CDC with Web-UI
    Java created 7y ago pushed 4d ago cdcchatbidataops
    +2.0/wk
    measured
    ★ 1.4k
    26
  21. 271
    🐬 Beginner-to-advanced MySQL course — filtering, joins, subqueries, window functions, CTEs & more. Hands-on guides with copy-paste SQL on one sample database. 🎥 Full video walkthrough on YouTube.
    created 5mo ago pushed 13d ago cruddatabasedbms
    +4.0/wk
    measured
    ★ 121
    26
  22. 272
    A desktop application for viewing and analyzing tabular data
    TypeScript created 10y ago pushed 20mo ago csvdata-analysisdata-science
    +3.0/wk
    measured
    ★ 3.5k
    26
  23. 273
    Visualize and share your data. All in SQL. Powered by DuckDB.
    Go created 18mo ago pushed 3d ago analyticsdashboardsdata
    +2.0/wk
    measured
    ★ 1.3k
    26
  24. 274
    An open-source inference engine for AI functions in SQL
    Python created 3mo ago pushed 3d ago ai-sqlarrowdatabase
    +5.0/wk
    measured
    ★ 179
    26
  25. 275
    Job board aggregator indexing 1,000,000+ active positions from 20,000+ companies across Greenhouse, Lever, Ashby, Workday, and other major ATS platforms. Multithreaded Python ETL pipeline, daily automated refreshes via GitHub Actions, filterable client-side search.
    Python created 10mo ago pushed 3d ago atsdata-engineeringdata-pipeline
    +5.0/wk
    measured
    ★ 185
    26
  26. 276
    Zero-config entity resolution feeding a durable identity layer: messy records from any source become stable golden entities, a Customer 360 with provenance, merge/split and audit. Fellegi-Sunter beats hand-tuned Splink. Arrow-native/Rust, 250M rows in 11.2 min. Python + edge TypeScript (WASM), SQL-native in Postgres & DuckDB, 97 MCP tools + REST.
    Python created 7mo ago pushed 3d ago data-cleaningdata-engineeringdata-matching
    +1.0/wk
    measured
    ★ 135
    26
  27. 277
    🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
    Python created 4y ago pushed 5mo ago aiapicli
    +1.0/wk
    measured
    ★ 2.3k
    26
  28. 278
    Local MCP server for DuckDB and MotherDuck
    Python created 23mo ago pushed 8d ago aiduckdbmotherduck
    +3.0/wk
    measured
    ★ 539
    26
  29. 279
    A comprehensive list of 180+ YouTube Channels for Data Science, Data Engineering, Machine Learning, Deep learning, Computer Science, programming, software engineering, etc.
    created 5y ago pushed 3mo ago aiartificial-intelligenceawesome
    +2.4/day
    measured
    ★ 1.7k
    26
  30. 280
    AxData 是一个开源量化数据库框架,覆盖通达信、巨潮、腾讯财经、新浪财经、东方财富、财联社、开盘红等公开源接口。
    Python created 4mo ago pushed 52d ago a-sharechina-stockdata-platform
    +1.4/day
    measured
    ★ 325
    25
  31. 281
    🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
    Ruby created 3y ago pushed 4d ago bigquerycdpcustomer-data-platform
    +1.0/wk
    measured
    ★ 1.7k
    25
  32. 282
    Turns Data and AI algorithms into production-ready web applications in no time.
    Python created 4y ago pushed 3mo ago automationdata-engineeringdata-integration
    +6.0/wk
    measured
    ★ 19.5k
    25
  33. 283
    Fast and efficient unstructured data extraction. Written in Rust with bindings for many languages.
    Rust created 2y ago pushed 22mo ago data-pipelinesdocxetl
    +3.0/wk
    measured
    ★ 1.8k
    25
  34. 284
    Python Streaming DataFrames for Kafka
    Python created 3y ago pushed 4d ago data-engineeringdata-intensive-applicationsdata-science
    +1.0/wk
    measured
    ★ 1.6k
    25
  35. 285
    《Easy Data x AI》是一门面向所有 AI 爱好者的数据与 AI 基础入门课程。通过双轨学习路径,从数据驱动视角学习构建 AI Agent。
    Python created 8mo ago pushed 31d ago agentic-aiaiai-agents
    +4.0/wk
    measured
    ★ 200
    25
  36. 286
    Bindings and ADO.NET Provider for DuckDB
    C# created 6y ago pushed 4d ago ado-netdata-scienceduckdb
    +2.0/wk
    measured
    ★ 722
    25
  37. 287
    pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
    Python created 7y ago pushed 6d ago amazon-athenaamazon-sagemaker-notebookapache-arrow
    −2.0/wk
    measured
    ★ 4.1k
    25
  38. 288
    A curated list of awesome PostgreSQL software, libraries, tools and resources, inspired by awesome-mysql
    created 11y ago pushed 2mo ago databasepostgrespostgresql
    +4.0/wk
    measured
    ★ 12.1k
    25
  39. 289
    An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
    Python created 6y ago pushed 6y ago airflowairflow-dagapache-airflow
    +3.0/wk
    measured
    ★ 1.6k
    25
  40. 290
    MAC address manufacturer lookup database combining IEEE, Nmap, Wireshark and HDM Mac-Tracker, with downloadable formats and a browser lookup. https://ringmast4r.github.io/OUI-Master-Database/
    JavaScript created 12mo ago pushed 2d ago cross-platformcybersecuritydatabase
    +4.0/wk
    measured
    ★ 149
    25
  41. 291
    AI-native SQL client for humans and agents. Query, explore, visualize, and open agent database work as editable SQL workspaces.
    TypeScript created 10mo ago pushed 4d ago aiclickhouseclickhouse-ui
    +3.0/wk
    measured
    ★ 309
    25
  42. 292
    Open-source data contract testing for Snowflake, Databricks, BigQuery, Kafka & 15 more. Built on the Open Data Contract Standard (ODCS). MIT.
    JavaScript created 3y ago pushed 4d ago bigquerydata-contractdata-contracts
    +1.0/wk
    measured
    ★ 1.1k
    25
  43. 293
    Ergonomic bindings to duckdb for Rust
    Rust created 5y ago pushed 2d ago arrowdatabaseduckdb
    +1.0/wk
    measured
    ★ 983
    25
  44. 294
    The leader in Customer Data Infrastructure
    Scala created 14y ago pushed 19d ago analyticsdatadata-collection
    −1.0/wk
    measured
    ★ 7k
    25
  45. 295
    Pivot is a high-performance real time analytics database on open data formats.
    Rust created 7mo ago pushed 2d ago aianalyticsbig-data
    +3.8/wk
    lifetime avg
    ★ 128
    25
  46. 296
    🥤 RAGLite is a Python toolkit for Retrieval-Augmented Generation (RAG) with DuckDB or PostgreSQL
    Python created 2y ago pushed 6d ago chainlitcolbertduckdb
    +1.0/wk
    measured
    ★ 1.2k
    25
  47. 297
    A collection of AI skills for working with Dagster
    Python created 9mo ago pushed 3d ago ai-toolsclaude-codedagster
    +2.0/wk
    measured
    ★ 218
    25
  48. 298
    The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.
    HTML created 5y ago pushed 3d ago analytics-engineerbigquerydata-analysis
    flat
    measured
    ★ 2.4k
    25
  49. 299
    Örnek SQL sorguları, T-SQL pratikleri ve Northwind veritabanı çalışmaları.
    TSQL created 8mo ago pushed 7mo ago codingdatabasenorthwind
    +1.6/day
    measured
    ★ 284
    24
  50. 300
    A fast, minimal, cross-platform database client built with C++ and ImGui
    C++ created 8mo ago pushed 3d ago cppdatabaseimgui
    +3.0/wk
    measured
    ★ 186
    24
← previous page 6 of 12 next →