Reinforcement Learning 2024

Lecturer: Lucian Busoniu. TA: Florin Gogianu.

Navigation: [Lecture slides|Labs|Contact]          [Back to the lecturer's webpage]

This course provides an introduction to reinforcement learning (RL), geared mostly towards control engineers. After introducing the RL problem, the dynamic programming algorithms that sit at the foundation of RL are described in the discrete-variable context. Then, classical RL algorithms are introduced in the same context. In the second part of the course, the dynamical programming and RL algorithms are extended with approximation techniques, in order to make them applicable to continuous-variable control, as well as to large-scale discrete-variable problems. We dedicate significant space to deep neural networks and their application to RL, leading to reinforcement learning techniques.

This course is part of the Master program CPS (Cyberphysical Systems) of the Automation Department, UTCluj (2nd year 1st semester). As prerequisites, basic knowledge of analysis and linear algebra is needed, together with notions of discrete-time dynamical systems. The teacher responsible is Lucian Busoniu.

The lecture and lab sessions take place on Tuesdays. A detailed schedule is given next:

Grading rules:

• 50% exam.
• 10% lecture quizzes.
• 50% labs.

The slides are made available here in time for each lecture. The slides are required material for the exam.

• Part 1: The reinforcement learning problem (covered in lecture 1).
• Part 2: The optimal solution. Dynamic programming (covered in lectures 2 and 3). You may also download the code for the examples in parts 2 and 3.

In the lab classes, a set of assignments must be solved. A solution consists of a brief report in PDF and associated code, and must be submitted by a specified deadline. For each lab, the full code or a specified part of it should be completed during the lab session itself. Each lab is graded up to 10, reduced to 5 if handed in late. Lab solutions are required to participate in the exam.

There is zero tolerance for copying; any copied solution means immediate forfeiture of the discipline.

A discussion session with mandatory participation will be organized before the exam, where the teachers will discuss the solutions separately with each student group. In this session, detailed questions will be asked to clearly assess whether the assignment solution is original, and the contribution of each student to this solution.

Contact

Comments, suggestions, questions etc. related to this course or website are welcome; please contact the lecturer.