Design, Evaluation and Comparison of Evolution and Reinforcement Learning Models.

Abstract

This thesis presents the design, evaluation and comparison of evolution and reinforcement learning models with a view towards their adaptability in environments of changing stability and a specific focus on the interaction between learning and evolution in Darwinian and Lamarckian frameworks. Our ultimate objective is to determine whether hybrid models of evolution and learning can demonstrate adaptive qualities beyond those of such models when applied in isolation. This work demonstrates the limitations of evolution, reinforcement learning and Lamarckian models in dealing with increasingly unstable environments, while noting the effective adaptive nature of a Darwinian model to assimilate increasing levels of instability. This is shown to be a result of the Darwinian evolution model's ability to separate learning at two levels, the population's experience of the environment over the course of many generations and the individual's experience of the environment over the course of its lifetime. Thus, knowledge relating to the general characteristics of the environment over many generations can be maintained in the population's genotypes with phenotype (reinforcement) learning being utilized to adapt a particular agent to the particular characteristics of its environment. Lamarckian evolution, though, is shown to demonstrate adaptive characteristics that are highly effective in response to the stable environments. Selection and reproduction combined with reinforcement learning creates a model that has the ability to utilize useful knowledge produced by reinforcements, as opposed to random mutations, to accelerate the search process. Since learnt solutions and evolved solutions remain consistent in stable environments, the influence of individual learning on the population's evolution is shown to be more successful when applied in the more direct, Lamarckian form. Based on our results demonstrating the success of Lamarckian strategies in stable environments and Darwinian strategies in unstable environments, hybrid Darwinian/Lamarckian models are created with a view towards combining the advantages of both forms of evolution to provide a superior adaptive capability. Our investigation demonstrates that such hybrid models can effectively combine the adaptive advantageous of both Darwinian and Lamarckian evolution to provide a more effective capability of adapting to a range of conditions, from stable to unstable, appropriately adjusting the degree of inheritance in response to the requirements of the environment.

Participants

Masters Theses

[1] Clinton Mclean. Design, evaluation and comparison of evolution and reinforcement learning models. Master's thesis, Computer Science Department, Rhodes University, Grahamstown, South Africa, April 2001. [PDF] [BibTeX]

Images