Move 37

Course Objective

This is the syllabus for "Move 37", Siraj Raval's free reinforcement learning course, as part of School of AI. This course can be taken for free on Youtube in the form of a playlist, or at School of AI for a more immersive learning experience. Reinforcement learning is driving some of the latest advances in AI, from DeepMind's AlphaGo to OpenAI's DOTA bots. Although these AIs are designed for video games, reinforcement learning is a powerful branch of AI that can be applied to endless applications in the real world. In this course, we'll cover various RL techniques in order of increasing complexity, applying them to both simulated and real world problems. Students will develop an intuition around when to use certain RL algorithms and by the end of the course will have the practical skills necessary to apply RL to a problem they are passionate about to make a positive impact in the world.

Prerequisites

• Understand Basic Python Syntax
• Understand how the Backpropagation algorithm works.

Components

• Midterm Project
• Final Project
• Educational Videos
• Quizzes
• Reading Assignments
• Coding Assignments
• Interviews
• Group Discussion in Slack

Course Length

• 10 Weeks
• 10-15 hours of dedicated study per week
• Starts September 10 at 12 PM PST

Tools Used

• Pytorch & Tensorflow (Deep Learning Libraries in Python)
• OpenAI Gym (Reinforcement Learning Library)
• Google Colab (for free GPUs, no need to install/configure dependencies)

Week 1 - Introduction

• Trailer
• Introduction
• Sensor Networks
• Google Dopamine (Live Stream)

Topics Covered

Markov Decision Processes, Policy Functions, Value Functions, and the Bellman Equation

Week 2 - Dynamic Programming

• Route Planning
• Options Pricing
• Scheduling
• Operating Systems

Topics Covered

Iterative Policy Evaluation, Policy improvement, Policy iteration, Value iteration

Week 3 - Monte Carlo Methods

• Interview #1
• Medical Diagnosis
• Energy Efficiency
• Physics Research

Topics Covered

Monte Carlo prediction, Monte Carlo control, Greedy & Epsilon-Greedy Policies , Exploration vs Exploitation Dilemma

Week 4 - Model-Free Learning

• Delivery Management
• Automated Trading
• Backgammon
• Dopamine in Neuroscience

Topics Covered

Temporal Difference Learning, SARSA, Q-Learning, Model vs Model Free Intuition

Week 5 - Reinforcement Learning in Continuous Spaces

• Self Driving Cars
• Delivery Drones
• Rescue Robots
• Assembly Robots

Topics Covered

Control Theory, Imitation Learning, The Hamilton-Jacobi-Bellman Equation, Kalman Filters

Midterm Project

• Train a bipedal humanoid robot to walk in simulation!

Week 6 - Deep Reinforcement Learning

• Traffic Optimization
• Gaming
• Meta Learning
• Interview #2

Topics Covered

DQN + Double DQN Networks, Dueling DQN, Prioritized Replay, Value-based Methods for Robotics

Week 7 - Policy Based Methods

• Web System Configuration
• Text Summarization
• AI Assisted Design
• Portfolio Optimization

Topics Covered

Evolutionary Algorithms, Stochastic Policy Search, Policy Gradients, REINFORCE

Week 8 - Policy Gradient Methods

• Dialogue Systems
• Photo Editing
• Language Translation
• Tutoring Systems

Topics Covered

Evolved Policy Gradients, Generalized Advantage Estimation (GAE), Trust Region Policy Optimization, Proximal Policy Optimization (PPO)

Week 9 - Actor Critic Methods

• Advanced Trading Techniques
• Human-Machine Cooperation
• Insurance Cost Analysis
• Interview #3

Topics Covered

Actor Critic Algorithms, Asynchronous Advantage Actor Critic, Deep Deterministic Policy Gradients (DDPG), Bayesian Actor-Critic

Week 10 - Multi Agent Reinforcement Learning

• Move 37
• Transportation Networks
• Decentralized Autonomous Organizations
• The Future of AI

Topics Covered

Cooperation, Competiton, Parallelism, Inverse Reinforcement Learning

Final Project

• Develop a multi-agent network to solve a real world problem!