From your YouTube video
Reinforcement learning from scratch
Your channel · 5 figures
Reinforcement learning
How a machine learns by trying things and keeping score.
1. The agent loop
- An agent sees a state st, picks an action at, and gets a reward rt+1.
- Nobody tells it the right move. It only learns which actions pay off.

2. Rewards over time
- The agent maximises the return: every future reward, with later ones counting less.
- The discount γ (between 0 and 1) sets how far ahead it plans.

3. Values and policies
- The value V(s) is the return you can expect from a state. The Bellman equation defines it one step at a time:
- Repeat that update everywhere (value iteration) and values spread out from the goal. The policy just follows the arrows uphill.

4. Explore or exploit
- Always picking the best-known action can miss something better. ε-greedy explores with a small probability ε.
- Learning from each step with Q-learning:

5. RL for language models (RLVR)
- Reinforcement learning with verifiable rewards trains a language model on questions whose answers can be checked, like maths problems or code with tests.
- A simple checker gives reward 1 for a correct final answer and 0 otherwise. No human judge or learned reward model is needed.
- The term comes from the Tülu 3 paper (Lambert et al., 2024). DeepSeek-R1 trained its reasoning with rule-based rewards like these and GRPO, which scores each answer against the others in its group:

End of notes · back to the video at 18:24
