Skip to main content

Class Logistics

Course Overview#

Deep Reinforcement Learning (RL) has made massive strides in the last decade for sequential decision making problems such as playing Atari games, mastering GO, and continuous control of robots. This course serves as a graduate-level introduction to RL, with an emphasis on applications and recent research. Students will be introduced to a broad set of topics in RL: Basic formalisms; Exploration vs exploitation; Imitation learning; Model-free RL; Model-based control and planning; Unsupervised learning for RL; Applications to games, robotics, industry; Current frontiers. This course will involve several coding home-works where you will implement various algorithms, and a final project. Other alternative titles for this course are Adaptive control and learning, Dynamic optimization.

Target Audience#

This course is aimed at MSc and PhD level students in computer science / data science.

Prerequisites#

CS/Data Science students#

This is NOT a basic course in reinforcement learning, deep learning or AI. If you are unsure about whether you are ready to take the class, go through Assignment 0 to test yourself.

  • Graduate level knowledge of machine learning, computer vision, and deep learning is assumed.
  • Proficiency in Python and PyTorch is assumed, and will be necessary to complete the assignments.

For non-DS/CS Students#

Please contact Rosemary Amico amico@cs.nyu.edu.

Logistical Overview#

Lecture#

5.10pm-7.00pm on Fridays at 60FA 110 (Class calendar).

  • Each week consists of a lecture.
  • Lectures will be held in person.
  • The class will be recorded, we will post the link to the recorded lectures to the class campuswire.

Grading#

  1. Homeworks (60%)
  2. Discussions, both in class and in campuswire (10%)
  3. Final Project (30%)

Assignments#

  • Assignments will use Python 3 and PyTorch; we will provide you a conda environment to install all dependencies.
  • To complete the assignemnt, you would need GPU access. Use Greene if you need access to one.
  • We strongly recommended using Python for the project as well.

Final Project#

  • Project proposals (1 page) will be due on 10/9.
  • Maximum (and recommended) team size is 2.
  • Final presentations of all projects will take place on 12/10/2021.

Course Textbook#

We highly recommend the following e-book: Sutton and Barto. Reinforcement Learning.. Reading materials for each class is posted in the schedule.

Course Staff#

Office Hours#

  • Lerrel: TBD
  • Mahi: TBD

Remarks#

  • A student in this course is expected to act professionally. Please also follow the GSAS regulations on academic integrity found here: http://gsas.nyu.edu/page/academic.integrity
  • Academic accommodations are available for students with disabilities. Please contact the Moses Center for Students with Disabilities (212-998-4980 or mosescsd@nyu.edu) for further information. Students who are requesting academic accommodations are advised to reach out to the Moses Center as early as possible in the semester for assistance.
Last updated on