LLM & Inference

An open-source take on ChatGPT's Intelligent UI: answers stream as interactive UI (maps, photos, charts, drawings and working tools). Built with OpenUI.

Fold papers, web pages, files, and AI replies into five levels of detail. Zoom from an overview to the original text.

This repo contains a PyTorch implementation of a pretrained BERT model for multi-label text classification.

A highly optimized LLM inference acceleration engine for Llama and its variants.

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

NEO Series: Native Vision-Language Models from First Principles

LeaderWorkerSet & DisaggregatedSet: Kubernetes APIs for Distributed and Disaggregated AI Serving

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

Generate High-Quality Synthetics, Train, Measure, and Evaluate in a Single Pipeline

Train and serve LLMs at extreme speed and massive throughput.

LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.

Official repository of my book "A Hands-On Guide to Fine-Tuning LLMs with PyTorch and Hugging Face"

Toolkit for fine-tuning, ablating and unit-testing open-source LLMs.

JevChat-Windows：聊天窗口旁挂的回复辅助。窗口截图 + 本地离线 OCR 读对方消息 → Jev 判断意图 → 3 条候选一键填入，发送永远手动

Sno Station Skills: 31 tested skills that turn Claude Code and Codex into one working team. Cross-vendor peer review, automatic handoff when an agent hits its quota, a one-page morning brief, work orders with proof, and a nightly loop that learns from your own sessions. Open source, installed with one sno setup.

Magick is a cutting-edge toolkit for a new kind of AI builder. Make Magick with us!

Confucius4-TTS: a Multilingual and Cross-Lingual Zero-Shot TTS Engine

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

Halo is an open-source framework built by White Circle for training large language and multimodal models

Self-hosted metasearch and grounding engine for AI agents, based on SearXNG. Rust, JSON API, optional page text.

AI System Design - Learn how to design AI systems built on LLMs, RAG, and AI Agents step by step.

♾️ Private Agent Fleet with Spec Coding. Each agent gets their own GPU-accelerated desktop. Run Claude, Codex, Gemini and open models on a full private AI Stack ♾️

免 ROOT 免 Termux，一个 APK 在安卓手机上跑 DeepSeek Harness。引擎内嵌 App 进程 · 读屏操作 App · 改系统设置 · 定时/条件自动化 · 可选 Shizuku

The most easy-to-understand tutorial for using LoRA (Low-Rank Adaptation) within diffusers framework for AI Generation Researchers🔥

Llama 3+ inference in pure Java

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

RDNA-native LLM inference engine in Rust.

[MICCAI 2019 Young Scientist Award] [MedIA Best Paper Award] Models Genesis: self-supervised pre-training for 3D medical images. Learns transferable representations from unlabeled CT and MRI volumes, then fine-tunes for downstream segmentation and classification. Keras and PyTorch weights included.

Single-GPU LLM fine-tuning preflight: memory estimates, runnable LoRA/QLoRA recipes, and local measurements.

ToolOrchestra is an end-to-end RL training framework for orchestrating tools and agentic workflows.

Framework for enhancing LLMs for RAG tasks using fine-tuning.

Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.

A curated list of tools built for Jev — TypeSafe AI's System One model for typed decisions.

Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more. Cross-platform: Linux, macOS, Windows, Android, iOS.

The platform for continuously improving AI agents.

A unified framework for easy fine-tuning in Flow-Matching models

Hands-on, framework-free Colab notebooks for the AI Engineer / Forward Deployed Engineer (FDE) skill set — model APIs, structured output, tool calling, RAG, evals-as-the-spine, agents (loop from scratch, tool design, guardrails, MCP, Skills), fine-tuning vs LoRA, prompt-injection/security, LLMOps, and customer craft. Runs on the free Groq API.

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

LLM-PowerHouse: Unleash LLMs' potential through curated tutorials, best practices, and ready-to-use code for custom training and inferencing.

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

The official implementation of MARS: Unleashing the Power of Variance Reduction for Training Large Models

Pure Rust Inference Engine

⚡ Pi, the coding agent, compiled ahead of time to one native executable. 10× faster to start, 7× less CPU and 4.5× less memory than Pi on Node. Linux, macOS, Windows, ARM64 and Android (Termux).

LLMFlows - Simple, Explicit and Transparent LLM Apps

A self-evolving agentic OS. A new architecture for agent execution, built on operating-system principles, with the harness as a path toward superintelligence.

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conversations}. Don't let the poverty limit your imagination! Train your own 8B/14B LLaVA-training-like MLLM on RTX3090/4090 24GB.

A curated list of decision models (System One / typed decision models): hosted APIs, open-weight models, runtimes, SDKs, applications, benchmarks, and papers.

Repo for "Adaptation of Agentic AI"

Autonomous development for real products. Governed agents build from specs you approve, follow the rules you set, and nothing merges until another agent proves it. Your decisions, judgement and taste are the bar that work must clear. Declare your vision and let agents build it.