LLM & Inference

🧪 Awesome list of AI Scientist papers, systems, benchmarks, datasets and open-source platforms.

An LLM training framework built from the ground up, featuring a custom BumbleBee architecture and end-to-end support for multiple open-source models across Pretraining → SFT → RLHF/DPO.

Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.

STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization

Honest local LLM deployment planning and benchmarking for high unified-memory Macs and future Linux/NVIDIA rigs.

构屎 GOUJI — 纯 C++/CUDA 的三值 mini LLM。2 bit/权重，零 Python；联想记忆工具路由，零训练参数；143 个缺陷与 29 个变异测试记录在案。

LLM inference engine written from scratch in HIP for AMD Strix Halo (gfx1151) and Qwen3.8-Flash-Next - ~50 tokens/s out to 492k context

An open-source virtual AI Infra team that discovers, benchmarks, and safely upgrades local LLM inference—with independent quality gates, a stable OpenAI-compatible API, and automatic rollback.

You type. It already knows. Local LLMs that read and think while you type.

DeepSeek-V4.1-Flash-1017 EXL3 2.9bpw on 2x NVIDIA DGX Spark (GB10) with TensorFold: 2.1-2.3x the vLLM kit's decode speed.

An environment where open models work as agents: a first-level harness with routing, skills, memory and sandboxed tools, on an engine that drafts and verifies its own tokens.

llama.cpp optimized for NVIDIA Tesla V100 (SM70). Qwen3.8-Flash-Next (MoE) with MTP on 1/2/3/6x V100-16GB at 262K context (6 GPUs: prefill 3402 tok/s at 200K; 2 GPUs on one socket: 2152 tok/s). Qwen3.8-27B with DFlash2 on 1-6 GPUs, up to 512K context. Multimodal, thinking, concurrent serving, built-in web UI.

Compile organizational policy into verifiable decision training data.

GLM-5.3-Flash EXL3 (320B MoE) on 2x NVIDIA DGX Spark — production serving kit, 1M context, 97%+ multi-session prefix caching, DFlash2 spec decode

Self-hosted decision model (selfjev-4b) with a Jev-compatible API: Python SDK, server, fine-tuning and RLCD.

Low VRAM SDXL Fine-Tuning Toolkit 2026: Train Full-Quality AI Models on 12GB GPUs

Best Open Source NLP LLM Fine-Tuning Trainer Framework 2026

Fine-Tuning GPT-2 for Math Riddles Generation with Transformers and TRL 2026

Zero-Config LoRA Fine-Tuning Studio for Diffusers 2026: The Ultimate Lightweight Trainer

Best AI Workload Orchestrator for Kubernetes Python 2026

A database-agnostic Agentic GraphRAG framework using swappable System One models (local Laya / cloud Jev). A plug-and-play intelligence layer featuring a complete 4-phase pipeline, continuous evaluation and custom A* traversal for any graph database.

Streamline AI Workloads on Kubernetes: Universal Python SDK Guide 2026

Modern Multi-Task NLP Fine-Tuning Framework with ModernBERT 2026

Open Source Instruction Tuning LLM Training Library 2026

Fine-Tuning Transformers in 2026: Hands-On LLM Training, LoRA, and PEFT Guide

An Introduction to reinforcement learning in the era of large language models

OpenAI-compatible speech serving in Rust: streaming TTS and full-duplex voice over GPT-Live. 128 concurrent PersonaPlex sessions on one GPU.

Strata Manager makes Niko1221/Strata, the local LLM inference engine, easier to set up and use. Manage local models and services, monitor hardware and API usage, chat and tune inference. Open-source desktop app for Windows / Linux x64.

Uncensored DeepSeek-V4.1-Flash IQ2_XS on Google Colab

《高级机器学习》：面向生成式人工智能时代的机器学习教材，涵盖强化学习、大模型训练与对齐、生成建模和多模态学习等内容。

Memory efficient and high performance Mixture-of-Experts inference for Apple Silicon

Switchless RoCE collectives, NCCL profiles and fabric tooling for DGX Spark inference recipes