ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

IRSML: An intelligent routing algorithm based on machine learning in software defined wireless networking

  • Duong, Thuy-Van T. (Faculty of Information Technology, Ton Duc Thang University) ;
  • Binh, Le Huu (Faculty of Information Technology, University of Sciences, Hue University)
  • Received : 2021.06.22
  • Accepted : 2022.04.26
  • Published : 2022.10.10
Download PDF
⟨ Previous Next ⟩

Abstract

In software-defined wireless networking (SDWN), the optimal routing technique is one of the effective solutions to improve its performance. This routing technique is done by many different methods, with the most common using integer linear programming problem (ILP), building optimal routing metrics. These methods often only focus on one routing objective, such as minimizing the packet blocking probability, minimizing end-to-end delay (EED), and maximizing network throughput. It is difficult to consider multiple objectives concurrently in a routing algorithm. In this paper, we investigate the application of machine learning to control routing in the SDWN. An intelligent routing algorithm is then proposed based on the machine learning to improve the network performance. The proposed algorithm can optimize multiple routing objectives. Our idea is to combine supervised learning (SL) and reinforcement learning (RL) methods to discover new routes. The SL is used to predict the performance metrics of the links, including EED quality of transmission (QoT), and packet blocking probability (PBP). The routing is done by the RL method. We use the Q-value in the fundamental equation of the RL to store the PBP, which is used for the aim of route selection. Concurrently, the learning rate coefficient is flexibly changed to determine the constraints of routing during learning. These constraints include QoT and EED. Our performance evaluations based on OMNeT++ have shown that the proposed algorithm has significantly improved the network performance in terms of the QoT, EED, packet delivery ratio, and network throughput compared with other well-known routing algorithms.

Keywords

References

  1. A. Abdelaziz, A. Fong, A. Gani, S. Khan, F. Alotaibi, and M. Khan, On software-defined wireless network (SDWN) network virtualization: Challenges and open issues, Comput. J. 60 (2017), 1510-1519. https://doi.org/10.1093/comjnl/bxx063
  2. A. Abujoda, D. Dietrich, P. Papadimitriou, and A. Sathiaseelan, Software-defined wireless mesh networks for internet access sharing, Comput. Netw. 93 (2015), 359-372. https://doi.org/10.1016/j.comnet.2015.09.008
  3. M. Bano, S. S. A. Gilani, and A. Qayyum, A comparative analysis of hybrid routing schemes for SDN based wireless mesh networks, (IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems, Exeter, UK), 2018, pp. 1189-1194.
  4. C. J. Bernardos, A. de la Oliva, P. Serrano, A. Banchs, L. M. Contreras, H. Jin, and J. C. Zuniga, An architecture for software defined wireless networking, IEEE Wirel. Commun. 21 (2014), no. 3, 52-61.
  5. T. M. Mitchell, Machine Learning, Science - Engineering - Math, McGraw-Hill, 1997.
  6. Y. Wang, M. Martonosi, and L.-S. Peh, Predicting link quality using supervised learning in wireless sensor networks, Mobile Comput. Commun. Rev. 11 (2007), no. 3, 71-83.
  7. A. Woo, T. Tong, and D. Culler, Taming the underlying challenges of reliable multihop routing in sensor networks, (Proceedings of The 1st ACM International Conference on Embedded Networked Sensor Systems, Los Angeles, CA, USA), 2003, pp. 14-27.
  8. K. Singh and J. Kaur, Machine learning based link cost estimation for routing optimization in wireless sensor networks, Adv. Wirel. Mobile Commun. 10 (2017), no. 1, 39-49.
  9. M. Boushaba, A. Hafid, and A. Belbekkouche, Reinforcement learning-based best path to best gateway scheme for wireless mesh networks, (IEEE 7th International Conference on Wireless and Mobile Computing, Networking and Communications, Shanghai, China), 2011, pp. 373-379.
  10. M. Boushaba, A. Hafid, A. Belbekkouche, and M. Gendreau, Reinforcement learning based routing in wireless mesh networks, Wirel. Netw. 19 (2013), no. 8, 2079-2091. https://doi.org/10.1007/s11276-013-0592-y
  11. T.-V. T. Duong, L. H. Binh, and V. M. Ngo, Reinforcement learning for QoS-guaranteed intelligent routing in Wireless Mesh Networks with heavy traffic load, ICT Express 8 (2022), no. 1, 18-24. https://doi.org/10.1016/j.icte.2022.01.017
  12. Z. Mammeri, Reinforcement learning based routing in networks: Review and classification of approaches, IEEE Access 7 (2019), 55916-55950. https://doi.org/10.1109/ACCESS.2019.2913776
  13. DARPA, The network simulator NS2. [Online]. Available: http://www.isi.edu
  14. A. R. Syed, K. A. Yau, J. Qadir, H. Mohamad, N. Ramli, and S. L. Keoh, Route selection for multi-hop cognitive radio networks using reinforcement learning: An experimental study, IEEE Access 4 (2016), 6304-6324. https://doi.org/10.1109/ACCESS.2016.2613122
  15. M. Yin, J. Chen, X. Duan, B. Jiao, and Y. Lei, QEBR: Q-learning based routing protocol for energy balance in wireless mesh networks, (IEEE 4th International Conference on Computer and Communications, Chengdu, China), 2018, pp. 280-284.
  16. C. Yu, J. Lan, Z. Guo, and Y. Hu, Drom: Optimizing the routing in software-defined networks with deep reinforcement learning, IEEE Access 6 (2018), 64533-64539. https://doi.org/10.1109/ACCESS.2018.2877686
  17. T. Lillicrap, J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, Continuous control with deep reinforcement learning, arXiv preprint, 2015. https://doi.org/10.48550/arXiv.1509.02971
  18. J. Rischke, P. Sossalla, H. Salah, F. H. P. Fitzek, and M. Reisslein, QR-SDN: Towards reinforcement learning states, actions, and rewards for direct flow routing in software-defined networks, IEEE Access 8 (2020), 174773-174791. https://doi.org/10.1109/ACCESS.2020.3025432
  19. C. Cung Trong, V. Tu, and N. Hai, An innovative solution of DSR routing mechanism based on mobile agent in MANET networks, J. Comput. Sci. Cybern. 29 (2013), no. 1, 31-42.
  20. C. T. Cuong, V. T. Tu, and N. T. Hai, MAR-AODV: Innovative routing algorithm in MANET based on mobile agent, (27th International Conference on Advanced Information Networking and Applications Workshops, Barcelona, Spain), 2013, pp. 62-66.
  21. L. H. Binh and T.-V. T. Duong, Load balancing routing under constraints of quality of transmission in mesh wireless network based on software defined networking, J. Commun. Netw. 23 (2021), no. 1, 12-22. https://doi.org/10.23919/JCN.2021.000004
  22. L. H. Binh and V. T. Tu, QTA-AODV: An improved routing algorithm to guarantee quality of transmission for mobile ad hoc networks using cross-layer model, J. Commun. 13 (2018), no. 7, 338-349.
  23. T. Liu and W. Liao, Location-dependent throughput and delay in wireless mesh networks, IEEE Trans. Vehic. Technol. 57 (2008), no. 2, 1188-1198. https://doi.org/10.1109/TVT.2007.905389
  24. V. Ramamurthi, A. S. Reaz, D. G. S. Dixit, and B. Mukherjee, Channel, capacity, and flow assignment in wireless mesh networks, Comput. Netw. 55 (2011), 2241-2258. https://doi.org/10.1016/j.comnet.2011.03.007
  25. A. Varga, Omnet++ discrete event simulation system, release 4.6, 2015. [Online]. Available: http://www.omnetpp.org
  26. A. Virdis and M. Kirsche, Recent advances in network simulation-The OMNeT++ environmentand its ecosystem, Springer Nature Switzerland AG, 2019.
  27. S. Khan, A.-S. K. Pathan, and N. A. Alrajeh, Wireless sensor networks - current status and future trends, CRC Press, 2012.
  28. A.-S. K. Pathan, M. M. Monowar, and S. Khan, Simulation technologies in networking and communications-Selecting the best tool for the test, CRC Press, Taylor & Francis Group, LLC, 2015.