Concept maps for how models and learning systems work, and the ideas behind them.
These notes explain the problems and design choices behind learning systems. The current series develops reinforcement learning from interaction and Markov models through policy gradients, PPO, GRPO, and DPO. Follow its reading path for the detailed progression. Representation learning, sequence models, and generative models are future directions for this collection.
A bilingual series building reinforcement learning from the ground up in nine chapters: from what makes it a distinct learning problem, through the Markov ladder and the Bellman machinery, to a full derivation of policy gradient, the sample-reuse machinery of PPO, GRPO's removal of the critic, and DPO's direct formulation of preference learning as policy optimization.
9 notes