List of resources about reinforcement learning organized into two sections; Textbooks & Resources, and Papers. I update the list on an ongoing basis. See Neural Networks for general resources on deep learning.

Textbooks & Resources

• Deep Reinforcement Learning: Good 90 minute video overview by John Schulman. Assumes some knowledge of neural networks. If you are not familiar with neural networks, then start with Sutton and Barto’s book.
• Reinforcement Learning: An Introduction: Classic textbook by Sutton and Barto. A good place to go next after watching John Schulman’s talk. Link is to the in progress 2nd edition. First edition published in 1998 is available here.
• David Silver’s Reinforcement Learning Course: David Silver is a fantastic teacher. The canonical source for reinforcement learning.
• The Open AI Lab: An experimentation system for Reinforcement Learning using OpenAI Gym, Tensorflow, and Keras. The Lab handles the basic RL environment and algorithm setups, provides implementations of standard components for deep reinforcement learning algorithms, such Q-Learning or Sarsa, and provides automated plots and analytics for testing new RL algorithms. Makes it easier both for beginners to get started with RL and for experienced practitioners to develop and compare new algorithms
• Denny Britz’ implementations of the major reinforcement learning algorithms.
• The OpenAI gym: Open source project which provides a simple interface to various reinforcement learning problems. This project deals with the simulating environments so that you can focus on implementing and testing algorithms.
• Unity ML Agents: SDK for designing environments, games, and simulations where agents can be trained using reinforcement learning, evolutionary algorithms, or any other machine learning method.

Papers

• Playing Atari with Deep Reinforcement Learning, 2013: Classic paper describing the convolutional neural network architecture which learned to play seven different Atari games, achieving superhuman performance in three of them.
• Mastering the game of Go with deep neural networks and tree search: Paper explaining AlphaGo, the algorithm that beat the world’s greatest Go player, Lee Sedol.
• Reinforcement Learning with Unsupervised Auxiliary Tasks, 2016: Interesting paper from DeepMind exploring the effect of adding auxiliary tasks. These additional tasks were motivated by the issue of sparse rewards – what do agents learn in the meantime? And even if rewards are frequent, auxiliary tasks may encourage the agent to learn beneficial representations. These tasks were indeed effective, and not only led to state of the art results, but were shown to improve data efficiency and robustness to hyper-parameter settings.
• On Actor-Critic Algorithms
• Continuous Control with Deep Reinforcement Learning: Introduces the Deep Deterministic Policy Gradient algorithm
• Prioritized Experience Replay: Tackles the problem of how to sample from memory. Of everything an agent has seen, what should it be focusing on the learn from?
• Neural Architecture Search with Reinforcement Learning: Trains neural networks to propose optimal model architectures and optimizers. An initial investigation into what neural networks can learn to learn.

I’m always looking to learn more. If you have any comments or suggestions, please send me an email on contact [at] learningmachinelearning [dot] org