The repository contains useful downloadable material related to my research and teaching, including Matlab software, presentations, and demonstration movies. Presentations are selectively chosen for tutorial value. If an item has a "»" button to its right, this button can be clicked to reveal more information; the "«" button then hides this information again (requires Javascript).


  • Approximate RL and DP toolbox, latest snapshot, including bugfixes and new, work-in-progress algorithms and experiments - possibly with their own, new bugs. (9 January 2016, 1.9 MBytes). »
  • Approximate RL and DP toolbox, July 2013 release. (13 July 2013, 1.6 MBytes). »
  • Optimistic planning, a selection of algorithms as a stand-alone package. (13 July 2013, 79.3 KBytes). »
  • MARL toolbox ver. 1.3, a Matlab multi-agent reinforcement learning toolbox (4 August 2010, 336.9 KBytes). »
  • MARL toolbox documentation, the documentation files for the MARL toolbox (4 August 2010, 223.1 KBytes). »
  • Approximate RL and DP toolbox, developed in Matlab. (6 June 2010, 967.6 KBytes). »
  • makepdf, a Windows XP batch script to automate the creation of PDF files from DVI (21 November 2008, 2.4 KBytes).


  • Optimistic planning for continuous actions, latest talk on an interesting algorithm for near-optimal control in general nonlinear systems with continuous actions (6 July 2016, 759.1 KBytes).
  • Reinforcement learning and planning algorithms, a high-level overview talk I gave at the IROS 2015 workshop on Machine Learning in Planning and Control of Robot Motion (2 October 2015, 3.0 MBytes). »
  • Nonlinear near-optimal control using optimistic planning, (Algorithms, Networked Control Systems, Real-Time Control), presented at the Italian Institute of Technology (25 September 2014, 4.6 MBytes). »
  • Optimistic planning for near-optimal control in MDPs, an in-depth description of the optimistic planning algorithm for MDPs and its analysis (1 December 2011, 1.1 MBytes). »
  • Reinforcement learning lectures, introducing classical and approximate RL (3 March 2010, 2.1 MBytes). »

Demonstration Movies

  • Assistive robot demo using online POMDP planning, Cyton Gamma 1500 robot arm, with Pioneer3AT mobile base and end-effector camera, flips off electrical switches forgotten on. Uses an online planning algorithm called AEMS2 for partially-observable Markov decision processes. With Elod Pall and Levente Tamas, see our IROS paper for details. (7 July 2016).
  • Planning to swing up a rotary pendulum in real time, using the continuous-action simultaneous optimistic optimization for planning (SOOP) algorithm. With Elod Pall. (24 November 2014).
  • Learning to swing up an inverted pendulum, using online least-squares policy iteration. (8 January 2009, 51.8 MBytes). »
  • Final swingup solution, after the online LSPI learning experiment was completed. (8 January 2009, 864.9 KBytes).
  • Robot goalkeeper learning to catch the ball, using approximate online RL and experience replay (demo by Sander Adam). (1 October 2008, 13.3 MBytes).