한국군사과학기술학회지 (Journal of the Korea Institute of Military Science and Technology)

  • 제26권4호
  • /
  • Pages.361-370
  • /
  • 2023
  • /
  • 1598-9127(pISSN)
  • /
  • 2636-0640(eISSN)
한국군사과학기술학회 (The Korea Institute of Military Science and Technology)
권호 표지
DOI QR코드

DOI QR Code

전이학습을 활용한 군집제어용 강화학습의 효율 향상 방안에 관한 연구

Study on Enhancing Training Efficiency of MARL for Swarm Using Transfer Learning

  • 이슬기 ((주)씨피프티원 기술연구소) ;
  • 김권일 ((주)씨피프티원 기술연구소) ;
  • 윤석민 (국방과학연구소 국방첨단과학기술연구원 인공지능자율센터)
  • Seulgi Yi (C51 Inc., R&D Department) ;
  • Kwon-Il Kim (C51 Inc., R&D Department) ;
  • Sukmin Yoon (AI Autonomy Technology Center, Advanced Defense Science & Technology Research Institute, Agency for Defense Development)
  • 투고 : 2023.03.09
  • 심사 : 2023.07.13
  • 발행 : 2023.08.05
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Swarm has recently become a critical component of offensive and defensive systems. Multi-agent reinforcement learning(MARL) empowers swarm systems to handle a wide range of scenarios. However, the main challenge lies in MARL's scalability issue - as the number of agents increases, the performance of the learning decreases. In this study, transfer learning is applied to advanced MARL algorithm to resolve the scalability issue. Validation results show that the training efficiency has significantly improved, reducing computational time by 31 %.

키워드

과제정보

이 논문은 2023년 정부(방위사업청)의 재원으로 국방과학연구소의 지원을 받아 수행된 연구임.(UG190055RD)

참고문헌

  1. https://www.newspim.com/news/view/20230106000810
  2. https://www.bbc.com/korean/international-49705340
  3. https://www.darpa.mil/work-with-us/offensive-swarmenabled-tactics
  4. Lowe, R. et al., "Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments," Proceedings of 31st Conference on Neural Information Processing Systems(NIPS 2017), Long Beach, CA, USA, 2017, arXiv preprint arXiv:1706.02275
  5. Vinyals, O. et al., "Grandmaster Level in StarCraft II using Multi-Agent Reinforcement Learning," Nature, Vol. 575, pp. 350-369, 2019. https://doi.org/10.1038/s41586-019-1724-z
  6. Hasan, Y. W. et al., "Defensive Escort Teams for Navigation in Crowds via Multi-Agent Deep Reinforcement Learning," Proceeding of IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS), Las Vegas, USA, 2020.
  7. Rahman, A. et al., "AdverSAR: Adversarial Search and Rescue via Multi-Agent Reinforcement Learning," 2022. arXiv preprint arXiv: 2212.10064v1
  8. Wang, J. et al., "Pattern RL: Multi-robot Cooperative Pattern Formation via Deep Reinforcement Learning," Proceeding of 18th IEEE International Conference on Machine Learning and Applications, 2019.
  9. Reynolds, C. W., "Flocks, Herds, and Schools: A Distributed Behavioral Model," Computer Graphics, Vol. 21(4), pp. 25-34, 1987. https://doi.org/10.1145/37402.37406
  10. Liu, Y. et al., "A Survey of Formation Control and Motion Planning of Multiple Unmanned Vehicles," Robotica, Vol. 36(7), pp. 1019-1047, 2018. https://doi.org/10.1017/S0263574718000218
  11. Huttenrauch, M. et al., "Deep Reinforcement Learning for Swarm Systems," Journal of Machine Learning Research, Vol. 20, pp. 1-31, 2019.
  12. Zhou, J., "Adversarial Swarm Defense with Decentralized Swarms," M.S. Thesis, Univ. of California, Berkeley, 2021.
  13. Zhang, K. et al., "Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms," Handbook of Reinforcement Learning and Control. Studies in Systems, Decision and Control, Vol. 325, Springer, pp. 321-384, 2021. arXiv preprint arXiv:1911.10635 1911
  14. Liu, X. et al., "Attentive Relational State Representation in Decentralized Multiagent Reinforcement Learning," IEEE Transactions on Cybernetics, Vol. 52(1), pp. 252-264, 2022. https://doi.org/10.1109/TCYB.2020.2979803
  15. Wang, W. et al., "From Few to More: Large-scale Dynamic Multiagent Curriculum Learning," Proceedings of AAAI Conference on Artificial Intelligence(AAAI), 2020. arXiv preprint arXiv: 1909.02790
  16. Rashid T. et al., "QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning," Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden, PMLR 80, 2018. arXiv preprint arXiv: 1803.11485
  17. Sunehag, P. et al., "Value-Decomposition Networks for Cooperative Multi-Agent Learning," 2017. arXiv preprint arXiv:1706.05296
  18. Zheng, L. et al., "MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence," NIPS 2017 & AAAI 2018 Demo, 2017. arXiv preprint arXiv:1712.00600
  19. Lu, M. et al., "Is Deep Reinforcement Learning Ready for Practical Applications in Healthcare? A Sensitivity Analysis of Duel-DDQN for Hemodynamic  Management in Sepsis Patients," AMIA Annual Symposium Proceedings, pp. 773-782, 2020. arXiv preprint arXiv: 2005.04301
  20. Boggess, K. et al., "Toward Policy Explanations for Multi-Agent Reinforcement Learning," 2022. arXiv preprint arXiv:2204.12568v1