LLM & Inference

Fast ML inference & training for ONNX models in Rust

MiniMax-M2, a model built for Max coding & agentic workflows.

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

An open coding agent for your terminal, built by a community collective rather than a company. Bring your own model, keep your code on your machine, and owe nothing to anyone.

A curated list of public projects, integrations, and discussions built on Jev — TypeSafe AI's System One model for typed decisions.

Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

A simple, performant, and scalable Jax LLM!

Mastering Applied AI, One Concept at a Time

A unified orchestration layer for heterogeneous AI compute. It standardizes how to manage compute and run training and inference on GPU clouds, Kubernetes, VMs, or bare-metal clusters.

Developer Asset Hub for NVIDIA Nemotron — A one-stop resource for training recipes, usage cookbooks, datasets, and full end-to-end reference examples to build with Nemotron models

HRM-Text is a 1B text generation model based on the HRM architecture, strengthened by task completion and latent space reasoning.

Distributed AI Model Training and LLM Fine-Tuning on Kubernetes

MoBA: Mixture of Block Attention for Long-Context LLMs

The Agentic Framework of the PHP ecosystem to build production-ready AI driven applications. Connect components (LLMs, Tools, vector DBs, memory) to agents that interact with your data and UI.

Your Automatic Prompt Engineering Assistant for GenAI Applications

A local inference engine for Apple silicon, built around the model.

AICI: Prompts as (Wasm) Programs

A design system for AI applications. Material, physical, tactile, technical.

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

A repository that contains models, datasets, and fine-tuning techniques for DB-GPT, with the purpose of enhancing model performance in Text-to-SQL

Your Cheat Sheet For AI Engineering Interviews at Top AI Companies - Questions and Answers.

Custom Diffusion: Multi-Concept Customization of Text-to-Image Diffusion (CVPR 2023)

Open Source AI Accelerator Research Platform Standard / 开源推理研究平台

Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V

Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.

Minimal reproduction of OneRec

[ICLR 2025] LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs

A comprehensive guide to building RAG-based LLM applications for production.

Scalable data pre processing and curation toolkit for LLMs

The non profit fostering AI in India through credits, grants, resources

Ultrafast serverless GPU inference, sandboxes, and background jobs

A Multilingual, Multimodal Translation Model Family

High-performance, privacy-first on-device AI inference library for React Native, powered by PyTorch's ExecuTorch runtime

Learn LLM internals step by step - from tokenization to attention to inference optimization.

Synthetic data curation for post-training and structured data extraction

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness.

LLM Inference Engine for Metal, CUDA and Vulkan.

Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.

Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request)

大模型（LLM）全栈学习路线与中文教程🔥：覆盖 Prompt Engineering、RAG、AI Agent、MCP、微调、模型部署、Transformer、AI 编程与大厂面试，从入门到生产实践。

Adapting Meta AI's Segment Anything to Downstream Tasks with Adapters and Prompts

Open-source AI computer builds, from 2-GPU desktops to 8-GPU servers: CAD, BOM, BIOS and build guides. Get the case kit with PCIe risers, or ask us for a free fit check.

Home for "How To Scale Your Model", a short blog-style textbook about scaling LLMs on TPUs

List of software that allows searching the web with the assistance of AI: https://hf.co/spaces/felladrin/awesome-ai-web-search

LuaN1aoAgent is a fully autonomous AI-driven penetration testing agent powered by graph-based cognitive reasoning.

Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.