The repository contains useful downloadable material related to my research and teaching, including Matlab software, presentations, and demonstration movies. Presentations are selectively chosen for tutorial value. If an item has a "»" button to its right, this button can be clicked to reveal more information; the "«" button then hides this information again (requires Javascript).


  • Approximate RL and DP toolbox, latest snapshot, including bugfixes and new, work-in-progress algorithms and experiments - possibly with their own, new bugs. (9 January 2016, 1.9 MBytes). »
  • Optimistic planning, a selection of algorithms as a stand-alone package. (13 July 2013, 79.3 KBytes). »
  • Approximate RL and DP toolbox, July 2013 release. (13 July 2013, 1.6 MBytes). »
  • MARL toolbox ver. 1.3, a Matlab multi-agent reinforcement learning toolbox (4 August 2010, 336.9 KBytes). »
  • MARL toolbox documentation, the documentation files for the MARL toolbox (4 August 2010, 223.1 KBytes). »
  • Approximate RL and DP toolbox, developed in Matlab. (6 June 2010, 967.6 KBytes). »
  • makepdf, a Windows XP batch script to automate the creation of PDF files from DVI (21 November 2008, 2.4 KBytes).


  • Basics of Reinforcement Learning, a very condensed introduction to basic dynamic programming and RL methods. Taught at the Transylvanian Summer School on Machine Learning, in Cluj-Napoca, Romania (20 July 2018, 4.5 MBytes).
  • AI Planning with Applications to Switched Systems, discussing, in addition to some planning techniques, their adaptations for switched system control. Keynote at the IFAC CESCIT conference (6 June 2018, 5.4 MBytes).
  • Online, Optimistic Planning for Markov Decision Processes, an in-depth course mainly on my recent research into optimistic planning algorithms, with a practical session. Taught at the ACAI Summer School on RL, in Nieuwpoort, Belgium (10 October 2017).
  • Approximate Dynamic Programming and Reinforcement Learning for Control, an invited, three-day intensive Master course at the Polytechnic University of Valencia, Spain (21 June 2017). »

Demonstration Movies

  • Fall detection using a quadrotor, A Parrot AR.Drone 2 monitors a person for falls while flying at a set distance and orientation. The location of the person, as well as falls, are detected with deep-learning vision algorithms. With Paul Dragan and Cristi Iuga, see our conference paper for details. (1 December 2017).
  • Assistive robot demo using online POMDP planning, Cyton Gamma 1500 robot arm, with Pioneer3AT mobile base and end-effector camera, flips off electrical switches forgotten on. Uses an online planning algorithm called AEMS2 for partially-observable Markov decision processes. With Elod Pall and Levente Tamas, see our IROS paper for details. (7 July 2016).
  • Planning to swing up a rotary pendulum in real time, using the continuous-action simultaneous optimistic optimization for planning (SOOP) algorithm. With Elod Pall. (24 November 2014).
  • Learning to swing up an inverted pendulum, using online least-squares policy iteration. (8 January 2009, 51.8 MBytes). »
  • Robot goalkeeper learning to catch the ball, using approximate online RL and experience replay (demo by Sander Adam). (1 October 2008, 13.3 MBytes).