9.3k repos tracked
updated 2d ago
Trending

Data

Pipelines, warehouses, orchestration and analytics engineering.

Trending New Most starred 551 repositories · topics: data-engineering, etl, data-pipeline, database, duckdb
  1. 151
    The most comprehensive SQL guide from a real-world expert! Learn everything from basics to advanced queries, optimizations, and real-world SQL
    TSQL created 20mo ago pushed 23d ago datadata-analyticsdata-engineering
    +3.4/day
    measured
    ★ 1.7k
    35
  2. 152
    Official home of the DB Browser for SQLite (DB4S) project. Previously known as "SQLite Database Browser" and "Database Browser for SQLite". Website at:
    C++ created 12y ago pushed 6d ago c-plus-pluscross-platformdatabase
    +1.4/day
    measured
    ★ 24.8k
    35
  3. 153
    Token Efficiency For AI Agents
    Python created 7mo ago pushed 6d ago ai-agentsautonomous-agentscli
    +1.0/wk
    measured
    ★ 125
    35
  4. 154
    Home of the Open Data Contract Standard (ODCS).
    Shell created 3y ago pushed 9d ago datadata-contractdata-contracts
    +1.7/day
    measured
    ★ 1.2k
    35
  5. 155
    The open-source reactive database for app developers
    TypeScript created 2y ago pushed 3d ago backendconvexdatabase
    +1.6/day
    measured
    ★ 12.8k
    35
  6. 156
    Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
    Python created 6y ago pushed 3d ago data-matchingdata-sciencededuplicate-data
    +2.3/day
    measured
    ★ 2.6k
    35
  7. 157
    A fast, keyboard-first database client built with Rust and GPUI.
    Rust created 10mo ago pushed 2d ago awsbeekeeper-alternativecross-platform
    +2.6/day
    measured
    ★ 514
    35
  8. 158
    One browser tab for PostgreSQL, MySQL, Oracle, SQL Server, MongoDB, Redis, SQLite, Couchbase, ClickHouse, Druid, DuckDB, Turso and more. An open-source SQL IDE with SSO, audit trail and AI-assisted queries MIT licensed, with nothing held back behind an enterprise wall.
    TypeScript created 10mo ago pushed 3d ago cassandraclickhousecouchbase
    +2.6/day
    measured
    ★ 1.3k
    35
  9. 159
    Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, code-like database branching, and scale to zero.
    Rust created 5y ago pushed 2mo ago databasepostgrespostgresql
    +3.4/day
    measured
    ★ 23.3k
    35
  10. 160
    The memory & storage database engine for AI agents. Multi-model, edge-to-cloud, PostgreSQL-compatible. Supports graph, vector, array, timeseries.
    Rust created 7mo ago pushed 3d ago agent-memoryai-agentscolumnar
    +1.0/day
    measured
    ★ 243
    34
  11. 161
    The fastest business intelligence tool for humans and agents.
    Go created 4y ago pushed 2d ago aiai-chatbotbi
    +1.9/day
    measured
    ★ 3k
    34
  12. 162
    High-performance, scalable time-series database designed for Industrial IoT (IIoT) scenarios
    C created 7y ago pushed 3d ago bigdatacloud-nativecluster
    +1.1/day
    measured
    ★ 25.2k
    34
  13. 163
    Business intelligence as code: build fast, interactive data visualizations in SQL and markdown
    TypeScript created 5y ago pushed 3d ago analyticsbusiness-intelligencedashboard
    +1.6/day
    measured
    ★ 7.1k
    34
  14. 164
    Python Stream Processing
    Python created 4y ago pushed 8d ago data-engineeringdata-processingdata-science
    +6.0/wk
    measured
    ★ 2.1k
    34
  15. 165
    A list of useful resources to learn Data Engineering from scratch
    created 7y ago pushed 9d ago cloud-providersdata-engineeringdata-pipeline
    +3.0/wk
    measured
    ★ 4k
    34
  16. 166
    Modern SQL and NoSQL database client for Neovim
    Rust created 58d ago pushed 4d ago clickhousedatabasedatabase-client
    +6.0/wk
    measured
    ★ 166
    34
  17. 167
    Always know what to expect from your data.
    Python created 9y ago pushed 2d ago cleandatadata-engineeringdata-profilers
    +1.3/day
    measured
    ★ 11.9k
    34
  18. 168
    REST API for any Postgres database
    Haskell created 12y ago pushed 3d ago apiautomatic-apidatabase
    +1.0/day
    measured
    ★ 27.7k
    34
  19. 169
    The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
    C++ created 3y ago pushed 9d ago agentic-aiclickhousecontext
    +5.0/wk
    measured
    ★ 2.3k
    34
  20. 170
    Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
    Rust created 2y ago pushed 2d ago apache-icebergapache-sparkarrow
    +1.7/day
    measured
    ★ 3.5k
    34
  21. 171
    Pure-Go SQLite, no cgo. Official mirror of gitlab.com/cznic/sqlite — issues and PRs welcome here.
    Go created 11mo ago pushed 5d ago cgo-freedatabasego
    +5.0/wk
    measured
    ★ 123
    34
  22. 172
    A super fast Graph Database uses GraphBLAS under the hood for its sparse adjacency matrix graph representation. Our goal is to provide the best Knowledge Graph for LLM (GraphRAG).
    Rust created 3y ago pushed 2d ago cloud-databasedatabasedatabase-as-a-service
    +1.3/day
    measured
    ★ 10.5k
    34
  23. 173
    A curated list of awesome big data frameworks, ressources and other awesomeness.
    created 12y ago pushed 8d ago awesomeawesome-listbigdata
    +1.3/day
    measured
    ★ 14.7k
    33
  24. 174
    Data streaming runtime focused on performance, consistency, and extensibility. Write plugins in Rust or WASM and process data with data guarantees.
    Rust created 5mo ago pushed 3d ago arrowdata-sciencedatafusion
    +3.0/wk
    measured
    ★ 107
    33
  25. 175
    Chat2DB is a free, cross-platform, local-first database client and SQL workspace for developers, DBAs, analysts, and data teams. Connect to 40+ databases, manage data, edit and run SQL, and use your own AI model to generate, explain, and optimize queries. Available on desktop, web, Docker, and CLI, with MCP support.
    Java created 3y ago pushed 6d ago aiclickhousedatabase
    +1.0/day
    measured
    ★ 28.4k
    33
  26. 176
    MiniBank - Simulation bancaire en C++
    C++ created 23d ago pushed 6d ago bankingbnpbourse
    +3.6/day
    measured
    ★ 93
    33
  27. 177
    The Data Engineering Cookbook
    Python created 7y ago pushed 22d ago best-practicesbig-datacookbook
    +1.4/day
    measured
    ★ 15.5k
    33
  28. 178
    Logical Replication extension for PostgreSQL 17, 16, 15, 14, 13, 12, 11, 10, 9.6, 9.5, 9.4 (Postgres), providing much faster replication than Slony, Bucardo or Londiste, as well as cross-version upgrades.
    C created 10y ago pushed 3mo ago cdcdata-transformationdata-transport
    +2.0/wk
    measured
    ★ 1.2k
    33
  29. 179
    libSQL is a fork of SQLite that is both Open Source, and Open Contributions.
    C created 4y ago pushed 22d ago databaseembedded-databaserust
    +1.3/day
    measured
    ★ 17.3k
    33
  30. 180
    ClickBench: a Benchmark For Analytical Databases
    Shell created 4y ago pushed 2d ago analyticsawsbenchmark
    +1.1/day
    measured
    ★ 1.2k
    32
  31. 181
    A Minimalistic Wrapper for IndexedDB
    TypeScript created 12y ago pushed 8d ago databaseindexeddbjavascript
    +1.0/day
    measured
    ★ 14.7k
    32
  32. 182
    In-memory key-value server in Rust that speaks RESP, so redis-cli and Redis clients connect. Locks, rate limits, job queues and a change feed are single commands; Raft-based failover. Clients for Rust, .NET, Go, Java, Python and Node.js.
    Rust created 23d ago pushed 3d ago cachedatabasedistributed-lock
    +2.6/day
    measured
    ★ 215
    32
  33. 183
    Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
    Go created 3y ago pushed 2d ago bigquerycdcclickhouse
    +5.0/wk
    measured
    ★ 3.3k
    32
  34. 184
    The fully open-source DuckDB workbench that runs in your browser. SQL editor, notebooks, charts, AI assistant. No install, no signup, no backend — your data never leaves the tab.
    TypeScript created 22mo ago pushed 15d ago analyticsdata-analysisdata-visualization
    +6.0/wk
    measured
    ★ 652
    32
  35. 185
    Fast, distributed per-aggregate event log with strict ordering and optimistic concurrency control.
    Rust created 11mo ago pushed 40d ago databaseevent-drivenevent-sourcing
    +1.0/wk
    measured
    ★ 116
    32
  36. 186
    Postgres intelligence for ai agents
    Go created 2mo ago pushed 5d ago aidata-analysisdatabase
    +1.6/day
    measured
    ★ 1.5k
    32
  37. 187
    Writing a sqlite clone from scratch in C
    C created 9y ago pushed 2y ago database
    +6.0/wk
    measured
    ★ 10.5k
    32
  38. 188
    OceanBase is the unified distributed database for the AI era — open-source, multi-model, one engine for your most demanding workloads.
    C++ created 5y ago pushed 3d ago analyticscloud-nativedatabase
    +6.0/wk
    measured
    ★ 10.4k
    32
  39. 189
    Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
    Rust created 4y ago pushed 2d ago apache-icebergdata-engineeringdatabase
    +6.0/wk
    measured
    ★ 9.4k
    32
  40. 190
    High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
    Rust created 4y ago pushed 3d ago ai-engineeringai-pipelinearrow
    +1.0/day
    measured
    ★ 5.8k
    32
  41. 191
    DuckDB extension to read and write to SQLite databases
    C created 5y ago pushed 3d ago duckdbsqlsqlite
    +2.0/wk
    measured
    ★ 297
    31
  42. 192
    Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
    Python created 5y ago pushed 3d ago connectorsdatadata-engineering
    +1.0/day
    measured
    ★ 2.7k
    31
  43. 193
    ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
    Go created 2y ago pushed 2d ago bigquerycopy-databasedata-ingestion
    +1.0/day
    measured
    ★ 4k
    31
  44. 194
    Supercharge BigQuery with BigFunctions
    Python created 4y ago pushed 2mo ago bigquerydatadata-analytics
    +1.0/wk
    measured
    ★ 761
    31
  45. 195
    📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
    created 6y ago pushed 2y ago applied-data-scienceapplied-machine-learningcomputer-vision
    +7.9/day
    measured
    ★ 30.8k
    31
  46. 196
    A data engineering project with Kafka, Spark Streaming, dbt, Docker, Airflow, Terraform, GCP and much more!
    Python created 4y ago pushed 4y ago airflowdata-engineeringdbt
    +1.4/day
    measured
    ★ 931
    31
  47. 197
    EF Core is a modern object-database mapper for .NET. It supports LINQ queries, change tracking, updates, and schema migrations.
    C# created 12y ago pushed 2d ago aspnet-productc-sharpdatabase
    +4.0/wk
    measured
    ★ 14.8k
    31
  48. 198
    🥑 ArangoDB is a native multi-model database with flexible data models for documents, graphs, and key-values. Build high performance applications using a convenient SQL-like query language or JavaScript extensions.
    C++ created 15y ago pushed 2d ago arangodbdatabasedistributed-database
    +4.0/wk
    measured
    ★ 14.3k
    31
  49. 199
    Dadbod Grip turns database tables into editable Vim buffers, with schema browsing, staged mutations, generated SQL, relationship navigation, and cross-database federation inside Neovim.
    Lua created 8mo ago pushed 4d ago aicross-databasedatabase
    +2.0/wk
    measured
    ★ 182
    31
  50. 200
    Postgres CLI with autocompletion and syntax highlighting
    Python created 12y ago pushed 3d ago databasepostgrespostgresql
    +4.0/wk
    measured
    ★ 13.4k
    31
← previous page 4 of 12 next →