Introduction and overview of RL 

For the first topic, we will learn about the foundations of RL and Markov Decision Processes.  Your reading for this topic is Section 22.1-22.2 (Introduction through Passive RL).

Although this topic has more videos than usual for our topics, they are relatively short.  I broke it into lots of videos to focus on each concept.

Link to my slides

Notation and problem setup

As with any family of algorithms, we need to discuss common notation and terms.  This video will do exactly that for RL!

Link to my slides

Markov Decision Processes

RL algorithms are based on Markov Decision Processes.  These may look familiar to you from previous classes where you did finite state machines.  There are differences though.  The video below discusses MDPs!

Link to my slides

How do you choose an action?

Given a value estimate (which we will discuss later how to learn this estimation), how do you choose an action?  There are several common approaches for RL (and many others not covered in this video, remember we are doing a short overview of RL, not a deep dive as the ML class will do!).  If you choose to implement RL for your project, you will probably want to implement e-greedy as discussed here as it is the simplest approach and works well.

Link to my slides

State-value functions and action-value functions

Before discussing how to learn a state or action value function, we need to discuss what they are.  This video overviews both state value functions (typically represented using V(S)) and action-value functions (typically represented using Q(s,a)).

Link to my slides

Exercise

Complete the exercise on the introduction to RL