Journal of Electrical Engineering and Technology

The Korean Institute of Electrical Engineers (대한전기학회)
권호 표지
DOI QR코드

DOI QR Code

Collision Prediction based Genetic Network Programming-Reinforcement Learning for Mobile Robot Navigation in Unknown Dynamic Environments

  • Findi, Ahmed H.M. (Control and Systems Engineering Department, University of Technology) ;
  • Marhaban, Mohammad H. (Department of Electrical and Electronic Engineering, Faculty of Engineering, Universiti Putra Malaysia) ;
  • Kamil, Raja (Department of Electrical and Electronic Engineering, Faculty of Engineering, Universiti Putra Malaysia) ;
  • Hassan, Mohd Khair (Department of Electrical and Electronic Engineering, Faculty of Engineering, Universiti Putra Malaysia)
  • Received : 2015.06.27
  • Accepted : 2016.10.12
  • Published : 2017.03.01
Download PDF
⟨ Previous Next ⟩

Abstract

The problem of determining a smooth and collision-free path with maximum possible speed for a Mobile Robot (MR) which is chasing a moving target in a dynamic environment is addressed in this paper. Genetic Network Programming with Reinforcement Learning (GNP-RL) has several important features over other evolutionary algorithms such as it combines offline and online learning on the one hand, and it combines diversified and intensified search on the other hand, but it was used in solving the problem of MR navigation in static environment only. This paper presents GNP-RL based on predicting collision positions as a first attempt to apply it for MR navigation in dynamic environment. The combination between features of the proposed collision prediction and that of GNP-RL provides safe navigation (effective obstacle avoidance) in dynamic environment, smooth movement, and reducing the obstacle avoidance latency time. Simulation in dynamic environment is used to evaluate the performance of collision prediction based GNP-RL compared with that of two state-of-the art navigation approaches, namely, Q-Learning (QL) and Artificial Potential Field (APF). The simulation results show that the proposed GNP-RL outperforms both QL and APF in terms of smooth movement and safer navigation. In addition, it outperforms APF in terms of preserving maximum possible speed during obstacle avoidance.

Keywords

References

  1. Zhang, Y., L. Zhang, and X. Zhang. Mobile Robot path planning base on the hybrid genetic algorithm in unknown environment. in Intelligent Systems Design and Applications, 2008. ISDA'08. Eighth International Conference on. 2008. IEEE.
  2. Belkhouche, F., Reactive path planning in a dynamic environment. Robotics, IEEE Transactions on, 2009. 25(4): p. 902-911. https://doi.org/10.1109/TRO.2009.2022441
  3. Du Toit, N.E. and J.W. Burdick, Robot motion planning in dynamic, uncertain environments. Robotics, IEEE Transactions on, 2012. 28(1): p. 101-115. https://doi.org/10.1109/TRO.2011.2166435
  4. Parhi, D.R., Navigation of mobile robots using a fuzzy logic controller. Journal of intelligent and robotic systems, 2005. 42(3): p. 253-273.
  5. Li, W. Fuzzy logic-basedperception-action'behavior control of a mobile robot in uncertain environments. in Fuzzy Systems, 1994. IEEE World Congress on Computational Intelligence., Proceedings of the Third IEEE Conference on. 1994. IEEE.
  6. Jaradat, M.A.K., M.H. Garibeh, and E.A. Feilat, Autonomous mobile robot dynamic motion planning using hybrid fuzzy potential field. Soft Computing, 2012. 16(1): p. 153-164. https://doi.org/10.1007/s00500-011-0742-z
  7. Mobadersany, P., S. Khanmohammadi, and S. Ghaemi. An efficient fuzzy method for path planning a robot in complex environments. in Electrical Engineering (ICEE), 2013 21st Iranian Conference on. 2013. IEEE.
  8. Mendonca, M., L.V.R. de Arruda, and F. Neves Jr, Autonomous navigation system using event drivenfuzzy cognitive maps. Applied Intelligence, 2012. 37(2): p. 175-188. https://doi.org/10.1007/s10489-011-0320-1
  9. Farooq, U., et al. A two loop fuzzy controller for goal directed navigation of mobile robot. in Emerging Technologies (ICET), 2012 International Conference on. 2012.
  10. Er, M. J. and C. Deng, Obstacle avoidance of a mobile robot using hybrid learning approach. Industrial Electronics, IEEE Transactions on, 2005. 52(3): p. 898-905. https://doi.org/10.1109/TIE.2005.847576
  11. Hui, N.B. and D.K. Pratihar, A comparative study on some navigation schemes of a real robot tackling moving obstacles. Robotics and Computer-Integrated Manufacturing, 2009. 25(4): p. 810-828. https://doi.org/10.1016/j.rcim.2008.12.003
  12. Hui, N.B., V. Mahendar, and D.K. Pratihar, Timeoptimal, collision-free navigation of a car-like mobile robot using neuro-fuzzy approaches. Fuzzy Sets and Systems, 2006. 157(16): p. 2171-2204. https://doi.org/10.1016/j.fss.2006.04.004
  13. Pratihar, D.K., K. Deb, and A. Ghosh, A genetic-fuzzy approach for mobile robot navigation among moving obstacles. International Journal of Approximate Reasoning, 1999. 20(2): p. 145-172. https://doi.org/10.1016/S0888-613X(98)10026-9
  14. Dinham, M. and G. Fang. Time optimal path planning for mobile robots in dynamic environments. in Mechatronics and Automation, 2007. ICMA 2007. International Conference on. 2007. IEEE.
  15. Vukosavljev, S.A., et al. Mobile robot control using combined neural-fuzzy and neural network. in Computational Intelligence and Informatics (CINTI), 2011 IEEE 12th International Symposium on. 2011. IEEE.
  16. Singh, M.K., D.R. Parhi, and J.K. Pothal. ANFIS Approach for Navigation of Mobile Robots. in Advances in Recent Technologies in Communication and Computing, 2009. ARTCom'09. International Conference on. 2009. IEEE.
  17. Dominguez-Lopez, J.A., et al., Adaptive neurofuzzy control of a robotic gripper with on-line machine learning. Robotics and Autonomous Systems, 2004. 48(2): p. 93-110. https://doi.org/10.1016/j.robot.2004.06.001
  18. Kareem Jaradat, M.A., M. Al-Rousan, and L. Quadan, Reinforcement based mobile robot navigation in dynamic environment. Robotics and Computer-Integrated Manufacturing, 2011. 27(1): p. 135-149. https://doi.org/10.1016/j.rcim.2010.06.019
  19. Ratering, S. and M. Gini, Robot navigation in a known environment with unknown moving obstacles. Autonomous Robots, 1995. 1(2): p. 149-165. https://doi.org/10.1007/BF00711254
  20. Ge, S.S. and Y.J. Cui, Dynamic motion planning for mobile robots using potential field method. Autonomous Robots, 2002. 13(3): p. 207-222. https://doi.org/10.1023/A:1020564024509
  21. Sgorbissa, A. and R. Zaccaria, Planning and obstacle avoidance in mobile robotics. Robotics and Autonomous Systems, 2012. 60(4): p. 628-638. https://doi.org/10.1016/j.robot.2011.12.009
  22. Agirrebeitia, J., et al., A new APF strategy for path planning in environments with obstacles. Mechanism and Machine Theory, 2005. 40(6): p. 645-658. https://doi.org/10.1016/j.mechmachtheory.2005.01.006
  23. Yaonan, W., et al., Autonomous mobile robot navigation system designed in dynamic environment based on transferable belief model. Measurement, 2011. 44(8): p. 1389-1405. https://doi.org/10.1016/j.measurement.2011.05.010
  24. Li, G., et al., Effective improved artificial potential field-based regression search method for autonomous mobile robot path planning. International Journal of Mechatronics and Automation, 2013. 3(3): p. 141-170. https://doi.org/10.1504/IJMA.2013.055612
  25. Wilkie, D., J. van den Berg, and D. Manocha. Generalized velocity obstacles. in Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on. 2009. IEEE.
  26. Chunyu, J., et al. Reactive target-tracking control with obstacle avoidance of unicycle-type mobile robots in a dynamic environment. in American Control Conference (ACC), 2010. 2010. IEEE.
  27. Mucientes, M., et al., Fuzzy temporal rules for mobile robot guidance in dynamic environments. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, 2001. 31(3): p. 391-398. https://doi.org/10.1109/5326.971667
  28. Chang, C.C. and K.-T. Song, Environment prediction for a mobile robot in a dynamic environment. Robotics and Automation, IEEE Transactions on, 1997. 13(6): p. 862-872. https://doi.org/10.1109/70.650165
  29. Mabu, S., A. Tjahjadi, and K. Hirasawa, Adaptability analysis of genetic network programming with reinforcement learning in dynamically changing environments. Expert Systems with Applications, 2012. 39(16): p. 12349-12357. https://doi.org/10.1016/j.eswa.2012.04.038
  30. Sendari, S., S. Mabu, and K. Hirasawa. Fuzzy genetic Network Programming with Reinforcement Learning for mobile robot navigation. in Systems, Man, and Cybernetics (SMC), 2011 IEEE International Conference on. 2011. IEEE.
  31. Li, X., et al., Probabilistic Model Building Genetic Network Programming Using Reinforcement Learning. 2011. 2(1): p. 29-40.
  32. Mabu, S., et al. Evaluation on the robustness of genetic network programming with reinforcement learning. in Systems Man and Cybernetics (SMC), 2010 IEEE International Conference on. 2010. IEEE.
  33. Mabu, S., et al. Genetic Network Programming with Reinforcement Learning Using Sarsa Algorithm. in Evolutionary Computation, 2006. CEC 2006. IEEE Congress on. 2006. IEEE.
  34. Sendari, S., S. Mabu, and K. Hirasawa. Two-Stage Reinforcement Learning based on Genetic Network Programming for mobile robot. in SICE Annual Conference (SICE), 2012 Proceedings of. 2012. IEEE.
  35. Li, X., S. Mabu, and K. Hirasawa, Towards the maintenance of population diversity: A hybrid probabilistic model building genetic network programming. Trans. of the Japanese Society for Evol. Comput, 2010. 1(1): p. 89-101.
  36. Sutton, R.S. and A.G. Barto, Reinforcement learning: An introduction. Vol. 1. 1998: Cambridge Univ Press.

Cited by

  1. A Q-learning approach based on human reasoning for navigation in a dynamic environment pp.1469-8668, 2018, https://doi.org/10.1017/S026357471800111X