Heuristically Accelerated Reinforcement Learning

.the.project

Reinforcement Learning (RL) techniques have been attracting a great deal of attention in the context of robotic, control and AI systems. The reasons frequently cited for such attractiveness are: the existence of strong theoretical guarantees on convergence, they are easy to use, and they provide model-free learning of adequate control strategies. Besides that, they also have been successfully applied to solve a wide variety of control and planning problems.

However, one of the main problems with RL algorithms is that they typically suffers from very slow learning rates, requiring a huge number of iterations to converge on a good solution. This problem becomes worse in tasks with high dimensional or continuous state spaces and when the system is given sparse rewards. One of the reasons for the slow learning rates is that most RL algorithms assumes that neither an analytical model nor a sampling model of the problem is available a priori, when, in some cases, there is domain knowledge that could be used to speed up the learning process: ``Without an environment model or additional guidance from the programmer, the agent may literally have to keep falling off the edge of a cliff in order to learn that this is bad behavior'' (Hasinoff, 2003).

As a way to add domain knowledge to help in the solution of the RL problem, the Heuristically Accelerated Reinforcement Learning (HARL) algorithms were proposed in 2004 (Bianchi, 2004). These algorithms allows the use of heuristics to speed up well-known Reinforcement Learning algorithms, using a heuristic function that influences the choice of the actions. Several HRL algoritms have been proposed:

This project investigates the use of the HARL to speed up the learning process of several types of domains, including mobile robots acting in a unknown environment, teams of mobile autonomous robotic agents acting in a concurrent multiagent environment like the RoboCup 2D Simulator and FIRA MiroSot and SimuroSot.

.students

Three MsC Students are working on HRL now:

.project.publications

List with all my publications can be found here.

Papers in Journals

Papers in International Conferences

created february 15th, 1994
last updated february 27th, 2007
webdesign by tri-star