International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

Analysis and study of Deep Reinforcement Learning based Resource Allocation for Renewable Powered 5G Ultra-Dense Networks

  • Received : 2024.01.05
  • Published : 2024.01.30
Download PDF
⟨ Previous Next ⟩

Abstract

The frequent handover problem and playing ping-pong effects in 5G (5th Generation) ultra-dense networking cannot be effectively resolved by the conventional handover decision methods, which rely on the handover thresholds and measurement reports. For instance, millimetre-wave LANs, broadband remote association techniques, and 5G/6G organizations are instances of group of people yet to come frameworks that request greater security, lower idleness, and dependable principles and correspondence limit. One of the critical parts of 5G and 6G innovation is believed to be successful blockage the board. With further developed help quality, it empowers administrator to run many systems administration recreations on a solitary association. To guarantee load adjusting, forestall network cut disappointment, and give substitute cuts in case of blockage or cut frustration, a modern pursuing choices framework to deal with showing up network information is require. Our goal is to balance the strain on BSs while optimizing the value of the information that is transferred from satellites to BSs. Nevertheless, due to their irregular flight characteristic, some satellites frequently cannot establish a connection with Base Stations (BSs), which further complicates the joint satellite-BS connection and channel allocation. SF redistribution techniques based on Deep Reinforcement Learning (DRL) have been devised, taking into account the randomness of the data received by the terminal. In order to predict the best capacity improvements in the wireless instruments of 5G and 6G IoT networks, a hybrid algorithm for deep learning is being used in this study. To control the level of congestion within a 5G/6G network, the suggested approach is put into effect to a training set. With 0.933 accuracy and 0.067 miss rate, the suggested method produced encouraging results.

Keywords

References

  1. Najm, M. Ismail, T. Rahm, and A. Al Razak, "Wireless implementation selection in higher institution learning environment," Journal of Theoretical and Applied Information Technology, vol. 67, pp. 477-484, 2014.
  2. T. Rahem, M. Ismail, I. A. Najm, and M. Balfaqih, "Topology sense and graph-based TSG: efficient wireless ad hoc routing protocol for WANET," Telecommunication Systems, vol. 65, no. 4, pp. 739-754, 2017. https://doi.org/10.1007/s11235-016-0242-7
  3. M. Y. Aalsalem, W. Z. Khan, W. Gharibi, M. K. Khan, and Q. Arshad, "Wireless Sensor Networks in oil and gas industry: recent advances, taxonomy, requirements, and open challenges," Journal of Network and Computer Applications, vol. 113, pp. 87-97, 2018. https://doi.org/10.1016/j.jnca.2018.04.004
  4. M. Saleem, S. Abbas, T. M. Ghazal, M. A. Khan, N. Sahawneh, and M. Ahmad, "Smart cities: fusion-based intelligent traffic congestion control system for vehicular networks using machine learning techniques," Egyptian Informatics Journal, vol. 23, 2022, In Press.
  5. R. Dong, C. She, W. Hardjawana, Y. Li, and B. Vucetic, "Deep learning for hybrid 5G services in mobile edge computing systems: learn from a digital twin," IEEE Transactions on Wireless Communications, vol. 18, no. 10, pp. 4692-4707, 2019. https://doi.org/10.1109/TWC.2019.2927312
  6. A. Sunny, S. Panchal, N. Vidhani et al., "A generic controller for managing TCP transfers in IEEE 802.11 infrastructure WLANs," Journal of Network and Computer Applications, vol. 93, pp. 13-26, 2017. https://doi.org/10.1016/j.jnca.2017.05.006
  7. Y. Chen, J. Li, W. Chen, Z. Lin, and B. Vucetic, "Joint user association and resource allocation in the downlink of heterogeneous networks," IEEE Transactions on Vehicular Technology, vol. 65, no. 7, pp. 5701-5706, 2016. https://doi.org/10.1109/TVT.2015.2452953
  8. A. Khalili, S. Akhlaghi, H. Tabassum, and D. W. K. Ng, "Joint user association and resource allocation in the uplink of heterogeneous networks," IEEE Wireless Communications Letters, vol. 9, no. 6, pp. 804-808, 2020. https://doi.org/10.1109/LWC.2020.2970696
  9. N. Zhao, Y.-C. Liang, D. Niyato, Y. Pei, M. Wu, and Y. Jiang, "Deep reinforcement learning for user association and resource allocation in heterogeneous cellular networks," IEEE Transactions on Wireless Communications, vol. 18, no. 11, pp. 5141-5152, 2019.
  10. V. Mnih, K. Kavukcuoglu, D. Silver et al., "Human-level control through deep reinforcement learning," Nature, vol. 518, no. 7540, pp. 529-533, 2015. https://doi.org/10.1038/nature14236
  11. Sharma, N., & Kumar, K. (2022). Energy Efficient Clustering and Resource Allocation Strategy for Ultra-Dense Networks: A Machine Learning Framework. IEEE Transactions on Network and Service Management.
  12. Anzaldo, A., Rodriguez, M. D., & Andrade, A. G. (2023). Intelligence-learning driven resource allocation for B5G Ultra-Dense Networks: A structured literature review.
  13. Zhao, S. (2023). Energy efficient resource allocation method for 5G access network based on reinforcement learning algorithm. Sustainable Energy Technologies and Assessments, 56, 103020.
  14. Peng, T., Guo, Y., Wang, Y., Chen, G., Yang, F., & Chen, W. (2022). An interference-oriented 5G radio resource allocation framework for ultradense networks. IEEE Internet of Things Journal, 9(22), 22618-22630. https://doi.org/10.1109/JIOT.2022.3183930
  15. Gao, S., Dong, P., Pan, Z., & Li, G. Y. (2020). Reinforcement learning based cooperative coded caching under dynamic popularities in ultra-dense networks. IEEE Transactions on Vehicular Technology, 69(5), 5442-5456. https://doi.org/10.1109/TVT.2020.2979918
  16. Tinh, B. T., Nguyen, L. D., Kha, H. H., & Duong, T. Q. (2022). Practical optimization and game theory for 6G ultra-dense networks: Overview and research challenges. IEEE Access, 10, 13311-13328. https://doi.org/10.1109/ACCESS.2022.3146335
  17. Mughees, A., Tahir, M., Sheikh, M. A., &Ahad, A. (2020). Towards energy efficient 5G networks using machine learning: Taxonomy, research challenges, and future research directions. Ieee Access, 8, 187498-187522. https://doi.org/10.1109/ACCESS.2020.3029903
  18. Gorla, P., Keerthivasan, V., Chamola, V., &Guizani, M. (2022). A Novel Framework of Federated and Distributed Machine Learning for Resource Provisioning in 5G and Beyond using Mobile-Edge SCBS. IEEE Transactions on Network and Service Management.
  19. Nguyen, V. D., Duong, T. Q., &Vien, Q. T. (2020). Emerging techniques and applications for 5G networks and beyond. Mobile Networks and Applications, 25, 1984-1986. https://doi.org/10.1007/s11036-020-01547-x
  20. Chen, J., Cao, X., Yang, P., Xiao, M., Ren, S., Zhao, Z., & Wu, D. O. (2022). Deep reinforcement learning based resource allocation in multi-uav-aided MEC networks. IEEE Transactions on Communications, 71(1), 296-309. https://doi.org/10.1109/TC.2020.3047972
  21. J. Qiu, J. Lyu, and L. Fu, "Placement optimization of aerial base stations with deep reinforcement learning," in ICC 2020-2020 IEEE International Conference on Communications (ICC),, pp. 1-6, Dublin, Ireland, 2020.
  22. J. Pei, P. Hong, M. Pan, J. Liu, and J. Zhou, "Optimal VNF placement via deep reinforcement learning in SDN/NFV-enabled networks," IEEE Journal on Selected Areas in Communications, vol. 38, no. 2, pp. 263- 278, 2020. https://doi.org/10.1109/JSAC.2019.2959181
  23. Jiang and X. Zhu, "Reinforcement learning based capacity management in multi-layer satellite networks," IEEE Transactions on Wireless Communications, vol. 19, no. 7, pp. 4685-4699, 2020.
  24. H.-X. Peng and X. Shen, "Multi-agent reinforcement learning based resource management in MEC- and UAV-assisted vehicular networks," IEEE Journal on Selected Areas in Communications, vol. 39, no. 1, pp. 131-141, 2021. https://doi.org/10.1109/JSAC.2020.3036962
  25. R. Arshad, H. ElSawy, S. Sorour, T. Y. Al-Naffouri, and M.-S. Alouini, "Velocity-aware handover management in two-tier cellular networks," IEEE Transactions on Wireless Communications, vol. 16, no. 3, pp. 1851-1867, 2017. https://doi.org/10.1109/TWC.2017.2655517
  26. Y. Z. H. Wang, X. Yang, and C. Wei, "METRE measurement task recommendation for energy-efficient handover in dense networks," in GLOBECOM 2020 - 2020 IEEE Global Communications Conference, pp. 1-6, Taipei, Taiwan, 2020.
  27. L. W. W. Sun, J. Liu, N. Kato, and Y. Zhang, "Movement aware CoMP handover in heterogeneous ultra-dense networks," IEEE Transactions on Communications, vol. 69, no. 1, pp. 340-352, 2021. https://doi.org/10.1109/TCOMM.2020.3019388
  28. Q. Liu, C. F. Kwong, S. Wei, L. Li, and S. Zhang, "Intelligent handover triggering mechanism in 5G ultra-dense networks via clustering-based reinforcement learning," Mobile Networks and Applications, vol. 26, pp. 27-39, 2021. https://doi.org/10.1007/s11036-020-01718-w
  29. M. Cicioglu, "Multi-criteria handover management using entropy-based SAW method for SDN-based 5G small cells," Wireless Networks, vol. 27, no. 4, pp. 2947-2959, 2021. https://doi.org/10.1007/s11276-021-02625-y
  30. G. Godor, Z. Jako, A. Knapp, and S. Imre, "A survey of handover management in LTE-based multi-tier femtocell networks: requirements, challenges and solutions," Computer Networks, vol. 76, pp. 17-41, 2015  https://doi.org/10.1016/j.comnet.2014.10.016