LLM & Inference

One-click Portable Windows installation of 'AI-Toolkit by Ostris'

⚡FlashRAG: A Python Toolkit for Efficient RAG Research (WWW2025 Resource)

结合多大模型分析与确定性代码风控的自主量化交易系统 ｜ Autonomous crypto trading system with multi-LLM analysis, deterministic Python risk control & OKX execution.

You type. It already knows. Local LLMs that read and think while you type.

An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that drafts and verifies its own tokens.

Confucius4-TTS: a Multilingual and Cross-Lingual Zero-Shot TTS Engine

Open desktop AI agent for Windows, macOS and Linux, written from scratch: its own agent engine, its own browser tooling and its own drawing engine that shows what it explains.

Long-term memory runtime for AI agents — plain Markdown as the source of truth, local ranked retrieval, and an independent sleep-time Manage layer. Claude Code and Codex share one store. No API key.

Pinned Intel Arc Pro B60/B70 inference recipes, engine patches and measurement records: Qwen3.8-27B EXL3 on one card (262,144 context, 63 tok/s; MTP3 measured lossy on 27% of a fixed 329-item set), Nemotron Bolt 74B, Flash-Next, dual-card routes, and a machine-checked catalog where each speculative row states whether the drafter changes the answer.

A design system for AI applications. Material, physical, tactile, technical.

Feel how fast a local LLM runs before you buy the hardware

This repository provides programs to build Retrieval Augmented Generation (RAG) code for Generative AI with LlamaIndex, Deep Lake, and Pinecone leveraging the power of OpenAI and Hugging Face models for generation and evaluation.

Pure Rust Inference Engine

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

DeepSeek-V4.1-Flash-1017 EXL3 2.9bpw on 2x NVIDIA DGX Spark (GB10) with TensorFold: 2.1-2.3x the vLLM kit's decode speed.

dgx-spark-qwen38 is an open-source installer that turns an NVIDIA DGX Spark, or any GB10 machine (ASUS, Dell, HP, Lenovo, MSI, GIGABYTE, Acer), into a private AI server in one command: measured model lanes, OpenAI and Anthropic APIs, and a local cockpit.

Local AI desktop app: a Chromium browser with a GPU-aware llama.cpp runtime, OpenAI-compatible API, AI chat and agents, Stable Diffusion image/video, TTS and music, plus an MCP server so Claude Code, Codex and Cline can drive your browser. Windows, macOS, Linux.

👋 Hello Agent System — learn enterprise AI agent system design by building every production layer from scratch. 17 bilingual lessons (EN/中文) with tested exercises: tools, architectures, reliability, security, evals, distributed execution, cost, RAG, release ops. 企业级 Agent 系统设计训练营

Blitz Strike — a universal MCP security-audit toolbelt. Reconnaissance at speed. Analysis in depth. Validation before report.

Custom Diffusion: Multi-Concept Customization of Text-to-Image Diffusion (CVPR 2023)

A linguistic linter for Traditional Chinese (zh-TW)

Resource, examples & tutorials for multimodal AI, RAG and agents using vector search and LLMs

聊天悬浮窗助手（macOS）：屏幕感知 + 本地小模型判断意图与风险，按话术生成回复候选。纯只读。

Every front-end GUI client for ChatGPT, Claude, and other LLMs

AI sovereignty for the world's most trusted runtime.

An offline coding agent for local models. One command sets up the best model your GPU can run, and on 24 GB+ RTX 30/40/50-series cards Qwen3.8-27B runs up to 1.9x faster than llama.cpp.

精酿 BrewReel：让 DeepSeek 这类便宜模型也能做出好看的竖版宣传片。写一份产品简报，AI 挑镜头、写文案，一条命令出片。3 种配方、6 个行业、广告法校验，开源可商用。

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

Learn LLM internals step by step - from tokenization to attention to inference optimization.

Video-MME-v2: Towards the Next Stage in Benchmarks for Comprehensive Video Understanding

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

An agentic system that auto-optimizes LLM workloads on AMD GPUs.

A curated collection of papers and resources on On-Policy Distillation for Large Language Models.

A curated list of open-source AI agent platforms: AI coworkers and teammates, agent builders and frameworks, workflow automation platforms, browser agents, and coding agents, with licenses.

Can AI Agents Build Bespoke Systems?

Flash weight streaming for MLX: run massive models larger than your RAM on Apple Silicon.

An intentionally vulnerable OWASP LLM Top 10 training platform for AI Security, Prompt Injection, RAG Security, Agent Security, and GenAI penetration testing.

The AI chat app that sweats the details. Every top model, real memory, living documents, and your own tools.

Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)

NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness.

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

Jev-Mem: System-One Controlled Agentic Memory

Open-source control panel for the real llama.cpp: see where a model lands in your VRAM before loading, every llama-server flag per model, one-click llama.cpp updates, several models at once, and Embers (local agents whose quotes are checked by code). Windows · Linux · macOS.

Adapting Meta AI's Segment Anything to Downstream Tasks with Adapters and Prompts

clodfarm (say it out loud): a farm of Claude Code agents. Plant a mission, they split it into sub-agents, open the work, and pace themselves on each account's real 5-hour and weekly usage. Steer it from the Claude app.

Sno Station Skills: 31 tested skills that turn Claude Code and Codex into one working team. Cross-vendor peer review, automatic handoff when an agent hits its quota, a one-page morning brief, work orders with proof, and a nightly loop that learns from your own sessions. Open source, installed with one sno setup.

The unofficial OpenAI SDK for Dart & Flutter. Full API coverage + OpenAI-compatible providers (Azure, DeepSeek, LM Studio, Ollama): Responses, Chat, Realtime, Videos, Batch, Fine-tuning.

The production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in chat UI, and native MCP.

Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses | 500+ Papers | Perception, Cognition, Planning, Interaction, Agentic System

LLMs as Copilots for Theorem Proving in Lean