Skip to main content

Course Schedule

wkLectureNotesLinksReading material
1/26Introduction to Deep Decision MakingPart 1: What is this class about?
Part 2: Why is decision making hard?

1. A framework for behavioural cloning, Bain and Sommut, 1999.
2. A reduction of imitation learning and structured prediction to no-regret online learning, Ross et al., 2011.
2/2Supervised Learning for Decision MakingPart 1: Training Neural Networks
Part 2: Variants of Behavior Cloning policies

1. Behavior Cloning (ALVINN)
2. Variational Autoencoder
3. Generative Adversarial Networks
4. Case study papers: VINN, RT-1, Dobb-E, Implicit BC, BeT, C-BeT, Diffusion Policy
2/9[Tutorial] Supervised Learning for Decision MakingSetting up decision making environments and model training
2/16Case studies of supervised decision makingExamples of supervised learning working in the real world
2/23Decision making without expert dataPart 1: Formalism for Bandit problem
Part 2: Algorithms for Bandit problems
3/1Sequential Decision makingPart 1: Motivation and formalism
Part 2: Core concepts of value and policy iteration
3/8Q-learning: from tables to AtariPart 1: Why Q function?
Part 2: Deep Q functions: What goes wrong and how to make them work?
Part 3: Variants of DQN
3/15Policy OptimizationPart 1: MC-based optimization (CEM)
Part 2: Differentiable versions (REINFORCE)
Part 3: Trust region / proximal policy optimization
3/29[Tutorial] Visual and Temporal Policy Learning
4/5Guest Lecture - Mahi Shafiullah
4/12Decision making with world modelsPart 1: Classical approaches (LQR / iLQR / DDP)
Part 2: Model-based RL
Part 3: case study: Dreamer v3
4/19Decision making with Tree SearchMCTC (AlphaGo, AlphaZero)
4/26Revisiting Decision making with expert dataInverse RL and offline RL
5/3Course Project Presentations