Multi-agent Learning Seminar
Mondays, 14h00 CoMo Lab (10G711)
Prior experience with reinforcement learning / machine learning such as from the Machine Learning, Techniques of AI or Computational Game Theory courses is recommended.
In this course we will look at different learning algorithms for intelligent agents. After an initial introduction to some of the more avanced learning methods students will be able to select a research project to work on. The focus of these projects will be on applying learing methods in robotics or large simulated problems.
This year the course will focus on applying learning in the OpenAI gym, a community driven AI benchmark suite. The sessions will take the form of regular (but not weekly) workshops. After the initial introduction to reinforcement learning, the students will be required to independently read research papers introducing various more advanced techniques. At the workshop one student will be tasked with presenting the paper, after which it will be discussed in group. Students will then implement and test the technique in a benchmark problem by the next session.
Students will be evaluated based on following criteria:
- Course participation (presentation of paper + participation in discussions)
- Learning agent project (2nd semester)
- Project defense & report (2nd semester exam period)
- 03/10/2016: Course Introduction
- 13/10/2016: Basics of Reinforcement Learning Note: this session will be combined with the learning dynamics/ computational game theory course and will be held exceptionally on Thursday at 13h in 10F720.
- 24/10/2016: Advanced Reinforcement Learning
- 31/10/2016: Choose papers from list. Mail top 3 to Peter.
- 7/11/2016: Introduction Deep RL
- Human-level control through deep reinforcement learning Presenter: Yannick
- Asynchronous Methods for Deep Reinforcement Learning Presenter: Fouad
- 15/11/2016 15-17h in 10F720: Invited lectures by Matthew Taylor and Enda Howley. (note different time & place!)
- Learning from others: speeding up sequential decision making (Matt Taylor)
- TBA (Enda Howley)
- Policy Distillation Presenter: Florentin
- Continuous control with deep reinforcement learning Presenter: Xinyu
- Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning Presenter: David
- Progressive Neural Networks Presenter: Andreas
- Assignment: get dqn running
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning Presenter: Fabio
- Deep Exploration via Bootstrapped DQN Presenter: Jens
- Mastering the game of Go with deep neural networks and tree search Presenter Arnau
- Dueling Network Architectures for Deep Reinforcement Learning Presenter: Kenzo
- Hybrid computing using a neural network with dynamic external memory Presenter: Nassim
- Project announcement
- Assignment: Experiment with Deep Policy Gradients (see github)
- 27/3/2017: mail project choice
- 24/4/2017: Project Progress session
- 5/6/2017: PROJECT DEADLINE (see presentation for details)
Basic Reinforcement Learning book:
Examples of Papers to read:
- Minih et al, 2014: Playing atari with deep reinforcement learning.
- Ng et al.,1999. Policy invariance under reward transformations: Theory and application to reward shaping
- S. Kalyanakrishnan and P. Stone, 2007. Batch reinforcement learning in a complex domain.
- Sutton, PRecup et Sing,1999. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
- The OpenAI Gym will be use as a testbed for learning algorithms
- Experiments will be coded in Python. We recommend installation of the Anaconda framework to get all the scientific computing libraries.