LLM & Inference

A curated collection of top-tier penetration testing tools and productivity utilities across multiple domains. Join us to explore, contribute, and enhance your hacking toolkit!

The Best TypeScript framework for fine-tuning

Assessing the reliability, accuracy, and safety of large language models, RAG pipelines, and AI agents.

Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE

🚀 200倍速！AI时代的下载神器 | Docker/PyPI/HuggingFace/CRAN 全加速 | 并行分片+智能缓存，让下载飞起来

DeepSeek-V4-Flash-0731 284B inference in ~25 GB of RAM / Qwen3.8-Next-Flash inference in ~15 GB of RAM on any M-series MacBook

An Efficient Heterogeneous Co-Design for Fine-Tuning on a Single GPU (DAC ‘26)

Unofficial skill that teaches coding agents to build with TypeSafe AI's Jev: typed decisions, calibrated confidence, and prior art from 150+ community projects.

从 MiniMind 源码读起,再延伸到现代大模型技术体系的中文学习笔记。主线逐行精读预训练 / SFT / DPO / PPO / GRPO 与训练机制;附录 17 篇进阶卷覆盖量化、投机解码、RLHF 全景、模型代际史等 MiniMind 没涉及、但进阶绕不开的主题。

[ICML 2026] A neuro-symbolic framework for abductive reasoning that grounds multi-agent collaboration in structured belief states using a causal graph and state machine.

[COLMW'26 Spotlight] AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters

LLM infrastructure cost reduction via NUMA-aware weight banking: 147 t/s (8.8x stock llama.cpp) on refurbished enterprise POWER8. Self-hosted inference, no cloud APIs. Part of the Proof of Physical AI stack.

[NeurIPS 2026] Official implementation of paper "VLM³: Vision Language Models Are Native 3D Learners".

Awesome Pretrained Chinese NLP Models，高质量中文预训练模型&大模型&多模态模型&大语言模型集合

AI novel generation with world building, interactive choices, and persistent character state. Next.js + Go. AI 小说创作台。

A bio-inspired cognitive memory engine — a new paradigm for Graph RAG.

VibeVoiceFusion is a full-stack, multi-speaker voice generation web system featuring LoRA fine-tuning, batch generation, and VRAM optimization. Based on Microsoft's VibeVoice (AR + diffusion architecture)

🧪 Awesome list of AI Scientist papers, systems, benchmarks, datasets and open-source platforms.

Nano vLLM with vLLM v1's request scheduling strategy and chunked prefill

Self-hosted decision model (selfjev-4b) with a Jev-compatible API: Python SDK, server, fine-tuning and RLCD.

Best Open Source NLP LLM Fine-Tuning Trainer Framework 2026

Hand-written NVFP4 W4A16 CUDA kernels for Volta

Fine-Tuning GPT-2 for Math Riddles Generation with Transformers and TRL 2026

Low VRAM SDXL Fine-Tuning Toolkit 2026: Train Full-Quality AI Models on 12GB GPUs

Defend sound research against LLM reviewer bias through meaning-preserving rewrites.

[ICML 2026] effGen: Enabling Small Language Models as Capable Autonomous Agents

German-OCR is specifically trained to extract text from German documents including invoices, receipts, forms, and other business documents.

Fine-Tuning Transformers in 2026: Hands-On LLM Training, LoRA, and PEFT Guide

Modern Multi-Task NLP Fine-Tuning Framework with ModernBERT 2026

Open Source Instruction Tuning LLM Training Library 2026

An Open-Ended Embodied Agent with Large Language Models

The inference engine the open-source world built for itself.

STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization

LLM agent 的有界自纠错长期记忆层：把连续、含矛盾的项目交互日志蒸馏成有界、可自更新的记忆库。Bounded, self-correcting long-term memory for LLM agents — candidate pool + value dynamics + tension adjudication + causal-replay evals.

A Markdown convention and linter for project docs written by AI and read by humans and agents

[Public preview] Externally scored agentic ML research benchmark: 60 tasks, real competition ground truth. Open protocol, operated evaluation.

Deploy, monitor, and tune LLM through a visual interface — plus image / video generation on top of ComfyUI.

PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers

An agent harness that compiles a model into one provably-correct, self-retargeting CUDA megakernel and self-tunes it past cuBLAS at batch-1 LLM decode, paper: https://arxiv.org/abs/2606.09682

Official Codebase for "Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights" (ICML 2026 Spotlight)

Switchless RoCE collectives, NCCL profiles and fabric tooling for DGX Spark inference recipes

Curated papers, taxonomy, benchmarks, and decision guides for credit assignment in reasoning and agentic LLM reinforcement learning.

TLive-Omni: An Omni-Modal Understanding Model for E-Commerce Live Streaming

An LLM-powered multi-agent framework for simulating interactive, evolving story worlds.

[Official] Prima.cpp: Scale Your Local AI Beyond One Device.

USP: Unified (a.k.a. Hybrid, 2D) Sequence Parallel Attention for Long Context Transformers Model Training and Inference

动手学大模型全栈：CS336 中文精讲 · PyTorch 手搓 Transformer · 单卡复现 Pretrain/SFT/LoRA/DPO/GRPO/RLVR · PEFT/Agent/RAG 落地（含真实实验、曲线与复现脚本）

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.

Explore a comprehensive collection of resources, tutorials, papers, tools, and best practices for fine-tuning Large Language Models (LLMs). Perfect for ML practitioners and researchers!

[ICLR 2025] LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs