Abstract
In this paper we present a new method for robot real time policy adaptation by combining learning and evolution. The robot adapts the policy as the environment conditions change. In our method, we apply evolutionary computation to find the optimal relation between reinforcement learning parameters and robot performance. The proposed algorithm is evaluated in the simulated environment of the Cyber Rodent (CR) robot, where the robot has to increase its energy level by capturing the active battery packs. The CR robot lives in two environments with different settings that replace each other four times. Results show that evolution can generate an optimal relation between the robot performance and exploration-exploitation of reinforcement learning, enabling the robot to adapt online its strategy as the environment conditions change.
Chapter PDF
Similar content being viewed by others
References
Sutton, R.S., Barto, A.G.: Reinforcement learning. MIT Press, Cambridge (1998)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Minato, T., Asada, M.: Environmental change adaptation for mobile robot navigation. Journal of Robotics Society of Japan 18(5), 706–712 (2000)
Matsui, T., Inuzuka, N., Seki, H.: Adapting to subsequent changes of environment by learning policy preconditions. Int. Journal of Computer and Information Science 3(1), 49–58 (2002)
Doya, K.: Reinforcement learning in continuous time and space. Neural Computation 12, 219–245 (2000)
Belew, R.K., McInerney, J., Schraudolph, N.N.: Evolving networks: using the genetic algorithm with connectionist learning. In: Langton, C.G., et al. (eds.) Artificial Life II, pp. 511–547. Addison Wesley, Reading (1990)
Unemi, T.: Evolutionary differentiation of learning abilities - a case study on optimizing parameter values in Q-learning by a genetic algorithm. In: Brooks, R.A., Maes, P. (eds.) Artificial Life IV - Proceedings of the Fourth International Workshop on the Synthesis and Simulation of Living Systems, pp. 331–336. MIT Press, Cambridge (1994)
French, R.M., Messinger, A.: Genes, phenes and the baldwin effect: learning and evolution in a simulated population. In: Proc. of Forth Int. Workshop on the Synthesis and Simulation of Living Systems, pp. 277–282 (1977)
Nolfi, S., Parisi, D.: Learning to adapt to changing environments in evolving neural networks. Adaptive Behavior 5(1), 75–98 (1996)
Niv, Y., Joel, D., Meilijson, I., Ruppin, E.: Evolution of reinforcement learning in uncertain environments: A simple explanation for complex foraging behaviors. Adaptive Behavior 10(1), 5–24 (2002)
Capi, G., Doya, K.: Application of evolutionary computation for efficient reinforcement learning. Applied Artificial Intelligence 20(1), 1–20 (2005)
Eriksson, A., Capi, G., Doya, K.: Evolution of metaparameters in reinforcement learning. In: IROS 2003 (2003)
Capi, G.: Multiobjective Evolution of Neural Controllers and Task Complexity. IEEE Transactions on Robotics 23(6), 1225–1234 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 IFIP International Federation for Information Processing
About this paper
Cite this paper
Capi, G., Toda, H., Kaneko, SI. (2011). Real Time Robot Policy Adaptation Based on Intelligent Algorithms. In: Iliadis, L., Maglogiannis, I., Papadopoulos, H. (eds) Artificial Intelligence Applications and Innovations. EANN AIAI 2011 2011. IFIP Advances in Information and Communication Technology, vol 364. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23960-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-23960-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23959-5
Online ISBN: 978-3-642-23960-1
eBook Packages: Computer ScienceComputer Science (R0)