LLM & Inference

A collection of libraries to optimise AI model performances

The chat decision assistant: before you reply, Jev reads the chat, judges intent and risk, and drafts replies you fill in with one tap. You press send. Android · Windows · macOS · iOS | 聊天辅助决策工具：先看懂对方再回复，发不发由你。

The paper list of the 86-page SCIS cover paper "The Rise and Potential of Large Language Model Based Agents: A Survey" by Zhiheng Xi et al.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

SoTA production-ready AI retrieval system. Agentic Retrieval-Augmented Generation (RAG) with a RESTful API.

Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.

A series of large language models trained from scratch by developers @01-ai

Infrastructure for continually self‑improving agents

ODS V3 Pre-Release: Public testing and refinement ahead of the official V3 launch. Turn your PC, Mac, or Linux box into a private AI server.

Open-source implementation of AlphaEvolve

Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts)

Using Low-rank adaptation to quickly fine-tune diffusion models.

GLM-5: From Vibe Coding to Agentic Engineering

《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

An Open-Ended Embodied Agent with Large Language Models

Reverse Engineering: Decompiling Binary Code with Large Language Models

Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.

OpenSquilla — Token-Efficient AI Agent with same budget, higher intelligence density

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)

Open-source Grok Bot alternative. Choose your own model and sandbox.

TSP自托管、零运维的 A 股「自建策略 + 监控 + 回测」量化工作台 | LLM能力驱使策略定制+个股分析+复盘 | 自由接入第三方数据源与个性化扩展数据 | 个人开源

Self-hosted TypeScript agent runtime with durable approvals and verifiable run records. Own it, approve it, audit it.

An embeddable, portable, branchable virtual machine to safely run Agents locally.

An on-device AI agent for Android that operates apps and the web to get things done for you.

FlashInfer: Kernel Library for LLM Serving

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

The agent-native LLM router for autonomous agents. Every frontier model behind one wallet, <1ms local routing, USDC payments on Base & Solana via x402.

The free AI already on your Mac. CLI tool, OpenAI-compatible server, and interactive chat — all on-device via Apple Intelligence. No API keys, no cloud, no downloads.

AI가 쓴 한글을 사람 글처럼 윤문하는 Claude 스킬 — Korean AI-text humanizer: detects and rewrites translationese, mechanical parallelism, and 71 other AI tells

Local-first AI agent workspace for coding, writing, design, research, and automation — one runtime for desktop GUI and TUI.

The most comprehensive Claude Code guide: agentic workflows, hooks, skills, MCP servers, quizzes, and production-ready templates. 430K+ lines.

Tutorials, Guides and Agent Skills Directories

Building AI agents, atomically

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

Chronos: Pretrained Models for Time Series Forecasting

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

Open-source, desktop-grade AI agent that gets real work done — data analysis, slides, docs, video & web research. Built on OpenClaw; runs tools on your real desktop and takes commands from your phone via WeChat, Feishu, DingTalk & Telegram.

Learn AI and LLMs from scratch using free resources

Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

[NeurIPS 2023] Tree of Thoughts: Deliberate Problem Solving with Large Language Models

The Harness of Harnesses • Super Agent: a trusted, persistent, self-evolving multi-agent ecosystem for all-domain collaboration.

🚀 Efficient implementations for emerging model architectures

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

多模型协作工作流引擎 — /ccg:go 一个命令，AI 自动分析意图、选择策略、编排 Codex + Gemini + Claude 协作执行

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

Apache Maka (Incubating) is a high-performance agent workspace that keeps a complete record of everything it did.

🚀 2026 最系统的 AI Agent 速成指南｜智能体实战教程 · 完整学习路径 + 实战项目 + 面试题库 · 对标大模型应用开发工程师岗位 · 覆盖LangChain / LangGraph / Coze / Dify / MCP / skills / LLM / RAG / 提示词 · 企业级部署与微调 · 从0到企业级落地 + 从学习到上线项目 + 面试准备一体化