λLLM Wiki
Posts

Posts

Browse the LLM Wiki by domain and most recent update.

40 posts

TypeTitleLast modified
appendix
FSDP2 Qwen3 Decoder Layer Walkthrough
appendix
knowledge
Fully Sharded Data Parallel
knowledge
knowledge
Proximal Policy Optimization
knowledge
knowledge
Value Model
knowledge
appendix
Derivation: GRPO's Unbiased Non-Negative KL Estimator
appendix
appendix
Derivation: From Policy Gradient to PPO
appendix
appendix
PPO+RLHF Training Loop with Value Model — Walkthrough
appendix
knowledge
Group Relative Policy Optimization
knowledge
knowledge
Reward ModelStale
knowledge
knowledge
Zero-RL for Reasoning ModelsStale
knowledge
knowledge
DTensor
knowledge
example
FSDP Transformer Block Sharding Example
example
appendix
FSDP Training Step WalkthroughVerified
appendix
appendix
Ring AllReduce Walkthrough
appendix
knowledge
AllReduce
knowledge
knowledge
Multi-Turn Reinforcement LearningVerifiedPublished
knowledge
knowledge
Speculative DecodingVerifiedPublished
knowledge
knowledge
KL DivergenceVerifiedPublished
knowledge
appendix
DFlash KV Injection Walkthrough
appendix
entity
DFlash
entity
entity
SGLang
entity
entity
Slime
entity
guide
Online Training of EAGLE MTP for RL Speculative Decoding
guide
knowledge
Agent Harness
knowledge
knowledge
Data Parallelism
knowledge
knowledge
Expert Parallelism
knowledge
knowledge
Flash Attention
knowledge
knowledge
KV Cache
knowledge
knowledge
LLM Parallelism StrategiesVerified
knowledge
knowledge
Model Quantization
knowledge
knowledge
On-Policy and Off-Policy Reinforcement Learning
knowledge
knowledge
Principal Component Analysis
knowledge
knowledge
Pipeline Parallelism
knowledge
knowledge
Rotary Position Embedding
knowledge
knowledge
Online Weight Updates in SGLang
knowledge
knowledge
SGLang Scheduler
knowledge
knowledge
Tensor Parallelism
knowledge
knowledge
Transformer
knowledge
lecture
Reading Guide to Zhaochenyang20's RL Systems Blog Series
lecture
tip
Megatron 3D Parallel Rank Layout Example
tip