TOC   *   Intro   *   Agent behaviour   *   Worlds   *   Running and replaying   *   Batch experiments

Important note: This documentation has been written for version 1.0 of the toolbox. Many things have changed since that version. Please see the Matlab reference for up-to-date information about how to use the toolbox, and the change log for a list of changes since version 1.0.

Introduction

The Multiagent Reinforcement Learning Toolbox (MARL Toolbox) is a set of Matlab functions that can be used to develop and test multiagent reinforcement learning algorithms. The toolbox is written for tasks that have the following properties:

The platform currently supports only episodic tasks, but hooks are in place for continuing tasks. The learning mechanism is generic and works for any task that satisfies the requirements above. New tasks can be easily added by implementing a required set of functions.

Agents can be homogeneous or heterogeneous in behaviour. Their behaviour is specified by three components: learning, action selection, and exploration strategy. New agent types can be added by implementing these components and linking them together with the agent constructor.

Learning experiments can be run and their results processed either individually, via Matlab commands, or in batches, via dedicated functions. The facilities for running and processing the results of batch experiments are useful in evaluating and cross-comparing the performance of various multiagent reinforcement learning algorithms.

This documentation is organized in the form of a tutorial. Each chapter explains an important piece of toolbox functionality by means of an example. This chapter continues with installation instructions, a detailed table of contents, and a list with the known issues of the toolbox. The following two chapters explain how to implement new agent behaviours and new task types. Chapter 4 shows how to run individual learning experiments, and how to replay learned agent behaviour. Chapter 5 demonstrates the use of the batch experiment facilities. Demonstration scripts implementing the tutorials in Chapters 4 and 5 can be found in the rldemo subdirectory of the toolbox. A Matlab functions reference in HTML form is also provided. Wherever the documentation refers to a toolbox function, a direct link is provided to its documentation page.

Jump to section:

  1. Installation
  2. Table of contents
  3. Work-in-progress. Known issues

Installation

Installation is easy:

  1. Unzip the rl-mfiles.zip archive containing the toolbox into a directory of your choice.
  2. Startup Matlab and navigate to the root directory of the toolbox, rl.
  3. Type startuprl at the Matlab prompt.
After this, you can run the demos by typing learndemo or batchdemo at the Matlab prompt.

Table of Contents

  1. Introduction
  2. Defining agent behaviour
  3. Creating new tasks
  4. Running learning and replaying learned behaviour
  5. Running and processing batch experiments

Work-in-progress. Known issues

This software is currently under development. Please send any reports of bugs, inconsistencies or missing items in the documentation, and the like, to the author. Thank you.

Known issues:

TOC   *   Intro   *   Agent behaviour   *   Worlds   *   Running and replaying   *   Batch experiments