The following notes?contain?Lesson 4 and 5 of the David Silver's lecture [1]?and Chapter 7?of Shiyu Zhao's Mathematical Foundation of Reinforcement Learning [2].

Reference

[1] https://www.davidsilver.uk/teaching/

[2] https://github.com/MathFoundationRL/Book-Mathmatical-Foundation-of-Reinforcement-Learning