Deep Reinforcement Learning
How an agent learns to act well purely from reward: the technique behind game-mastering AIs and robot control.
AI tutor is turned off for this class
Use the CS224R lectures, notes, and assignments below to keep learning.
Big Idea
Learning
Grade bands
K-2 · 3-5 · 6-8 · 9-12
AI literacy pillar
How AI works · Ethics
Lesson overview
How an agent learns to act well purely from reward: the technique behind game-mastering AIs and robot control. This module climbs from an everyday intuition to the real mechanism, then names the Stanford course it descends from.
Teacher script · ~45 min
- 0–5
Hook
Train a dog: good behavior gets a treat, bad behavior doesn't. The dog isn't told the rules; it figures out which actions lead to treats by trying things. Reinforcement learning is exactly this, for software: act, get reward, do more of what paid off.
- 5–15
Explore
Students do the activity in pairs: With a friend, hide candy under one of three cups and let them guess repeatedly. Watch them balance 'stick with the winner' vs 'check the others.'
- 15–30
Explain
The hard part of RL is that rewards come late: a chess move that loses the game might have been the move 20 turns earlier. The agent must spread credit and blame backward across time onto the actions that really mattered. A 'value function' estimates how good a situation is long-term, not just right now.
- 30–40
Connect to the summit
Show students this is the real thing professionals build: CS224R, the real thing. How an agent learns to act well purely from reward: the technique behind game-mastering AIs and robot control.
- 40–45
Check
Run the formative check below. Anyone who can explain a key term in their own words has it.
Student activity
With a friend, hide candy under one of three cups and let them guess repeatedly. Watch them balance 'stick with the winner' vs 'check the others.'
Slides
Formative check
- 1.In your own words, what is "Reward"? (Looking for: A number the environment hands back telling the agent how good its last action was.)
- 2.In your own words, what is "Policy"? (Looking for: The agent's strategy: what action to take in each situation.)
- 3.In your own words, what is "Value function"? (Looking for: An estimate of total future reward from a given situation, not just the immediate payoff.)
Carry-away concepts
- Reward
- A number the environment hands back telling the agent how good its last action was.
- Policy
- The agent's strategy: what action to take in each situation.
- Value function
- An estimate of total future reward from a given situation, not just the immediate payoff.
- Exploration vs exploitation
- The trade-off between trying new things and cashing in on what already works.
From the summit · the Stanford source
You implement policy gradients, Q-learning with deep networks, actor-critic methods, and offline RL, and wrestle with why they're so unstable to train.
This module descends from CS224R at Stanford. Students who climb the full ladder arrive here.