International Journal of High-Rise Buildings (국제초고층학회논문집)

Council on Tall Building and Urban Habitat Korea (한국초고층도시건축학회)
권호 표지
DOI QR코드

DOI QR Code

Transfer Learning Based Real-Time Crack Detection Using Unmanned Aerial System

  • Yuvaraj, N. (Department of Architectural Engineering, Kyungil University) ;
  • Kim, Bubryur (Department of Architectural Engineering, Kyungil University) ;
  • Preethaa, K. R. Sri (Department of Artificial Intelligence and Data Science, KPR Institute of Engineering and Technology)
  • Published : 2020.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

Monitoring civil structures periodically is necessary for ensuring the fitness of the structures. Cracks on inner and outer surfaces of the building plays a vital role in indicating the health of the building. Conventionally, human visual inspection techniques were carried up to human reachable altitudes. Monitoring of high rise infrastructures cannot be done using this primitive method. Also, there is a necessity for more accurate prediction of cracks on building surfaces for ensuring the health and safety of the building. The proposed research focused on developing an efficient crack classification model using Transfer Learning enabled EfficientNet (TL-EN) architecture. Though many other pre-trained models were available for crack classification, they rely on more number of training parameters for better accuracy. The TL-EN model attained an accuracy of 0.99 with less number of parameters on large dataset. A bench marked METU dataset with 40000 images were used to test and validate the proposed model. The surfaces of high rise buildings were investigated using vision enabled Unmanned Arial Vehicles (UAV). These UAV is fabricated with TL-EN model schema for capturing and analyzing the real time streaming video of building surfaces.

Keywords

References

  1. Abdel-Qader, Abudayyeh, O. and Kelly, (2003). Analysis of Edge-Detection Techniques for Crack Identification in Bridges. Journal of Computing in Civil Engineering, 17, 255-63. https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255)
  2. Ali, R., Gopal L.G. and Cha Y.J. (2018). Vision-based concrete crack detection technique using cascade features, Proc. SPIE, Sensors and Smart Structures Technologies for Civil Mechanical, and Aerospace Systems, 10598.
  3. Alipour-Fanid, Amir, Dabaghchian, Monireh, Wang, Ning, Wang, Pu & Zhao, Liang & Zeng, Kai. (2019). Machine Learning-Based Delay-Aware UAV Detection and Operation Mode Identification over Encrypted Wi-Fi Traffic.
  4. Asakura, T and Kojima, Y. (2003). Tunnel maintenance in Japan, Tunnelling and Underground Space Technology. 18, 161-169. https://doi.org/10.1016/S0886-7798(03)00024-5
  5. ASCE, 2017, American Society of Civil Engineers (ASCE), Infrastructure Report Card.
  6. Asharul Islam Khan and Yaseen Al-Mulla. (2019). Unmanned Aerial Vehicle in the Machine Learning Environment, Procedia Computer Science, 160, pp. 46-53. https://doi.org/10.1016/j.procs.2019.09.442
  7. Billie F Spencer, Vedhus Hoskere and Yasutaka Narazaki (2019). Advances in Computer Vision-Based Civil Infrastructure Inspection and Monitoring. Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030
  8. Bishop, C.M. (2006) Pattern recognition and machine learning. New York City, Springer-Verlag.
  9. Cao Vu Dung and Le Duc Anh (2019). Autonomous concrete crack detection using deep fully convolutional neural network. Automation in Construction, 52-58.
  10. Cha, Y.J. and Wooram Choi. (2017). Deep learning-based crack damage detection using convolutional neural networks, Computer-Aided Civil and Infrastructure Engineering, 32, 361-378. https://doi.org/10.1111/mice.12263
  11. Cha, Y.J., Choi, W., Suh, G., Mahmoudkhani, S and Buyukozturk, O. (2017) Autonomous structural visual inspection using region based deep learning for detecting multiple damage types, Computer Aided Civil Infrastructure Engineering, 33(9), 731-747.
  12. Chen, F. and Jahanshahi, M. R. (2018). NB-CNN: deep learning-based crack detection using convolutional neural network and Naive Bayes data fusion. IEEE Transactions on Industrial Electronics, 65(5), 4392-4400. https://doi.org/10.1109/TIE.2017.2764844
  13. Christian Koch, Kristina Georgieva, Varun Kasireddy, Burcu Akinci and Paul Fieguth. (2015). A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Advanced Engineering Informatics, 29(2), 196-210. https://doi.org/10.1016/j.aei.2015.01.008
  14. Chuan-Wei Zhang , Meng-Yue Yang, Hong-Jun Zeng, Jian-Ping Wen. (2019). Pedestrian detection based on improved LeNet-5 convolutional neural network. Journal of Algorithms & Computational Technology, 13, 1-10.
  15. Dinh, T.H. ,Ha, Q. and La, H. M. (2016) "Computer vision-based method for concrete crack detection." Proc. 14th International Conference on Control, Automation, Robotics and Vision, August, pp.1-6.
  16. Dongho Kang and Young-Jin Cha (2018). Autonomous UAVs for structural health monitoring using deep learning and an ultrasonic beacon system with geo-tagging. Computer-Aided Civil and Infrastructure Engineering, 33 885-902. https://doi.org/10.1111/mice.12375
  17. Douglas, Reward K., Said Nawar, M Carmen Alamar, AM Mouazen, and Frederic Coulon. (2018). Rapid Assessment of Petroleum Hydrocarbon Contamination in Soils: Spectroscopy Approach. Journal of Engineerng and Applied Sciences, 4(3), 66-73.
  18. Fan, Z., Wu, Y., Lu, J. and Li, W. (2018) Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network, Computer Visions and Pattern Recognition, arXiv:1802.02208.
  19. Fathi, H., Fei Dai and Manolis Lourakis. (2015). Automated as-built 3D reconstruction of civil infrastructure using computer vision: Achievements, opportunities, and challenges. Advanced Engineering Informatics, 29, 49-161.
  20. Gibb, S., La H. M. and Louis, S. (2018). A genetic algorithm for convolutional network structure optimization for concrete crack detection. IEEE Congress on Evolutionary Computation (CEC), Rio de Janeiro, 1-8.
  21. Islam, M. M. M., Kim J. M. (2019). Vision-Based Autonomous Crack Detection of Concrete Structures Using a Fully Convolutional Encoder-Decoder Network, Sensors, 19.
  22. Khairunniza-Bejo, Siti, Samihah Mustaffha, and Wan Ishak Wan Ismail. (2014). Application of artificial neural network in predicting crop yield: A review. Journal of Food Science and Engineering, 4(1), p. 1.
  23. Kim, I.H, Jeon, H., Baek, S.C.,Hong W.H and Jung H.J. (2018). Application of Crack Identification Techniques for an Aging Concrete Bridge Inspection Using an Unmanned Aerial Vehicle. Sensors, 18.
  24. L. Zhang, F. Yang, Y. Daniel Zhang and Y. J. Zhu, "Road crack detection using deep convolutional neural network," 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, 2016, pp. 3708-3712, doi:10.1109/ICIP.2016.7533052.
  25. LeCun, Y., Bottou, L., Bengio, Y. and Haffner, P. (1998) "Gradient-based learning applied to document recognition." Proc. the IEEE, 86, pp. 2278-2324. https://doi.org/10.1109/5.726791
  26. Lecun,Y., Bottou, L., Bengio, Y. and Haffner, P. (1998) "Gradient-based learning applied to document recognition." Proc of the IEEE, 86(11), 2278-2324. https://doi.org/10.1109/5.726791
  27. Li, G., Zhao, X., Du, K., Ru, F. and Zhang, Y. (2017). Recognition and evaluation of bridge cracks with modified active contour model and greedy search-based support vector machine. Automation in Construction, 78, 51-61. https://doi.org/10.1016/j.autcon.2017.01.019
  28. Nahar, P., Wu, K., Mei, S., Ghoghari, H., Srinivasan, P., Lee, Y., Gao, J.Z., & Guan, X. (2017). Autonomous UAV forced graffiti detection and removal system based on machine learning. 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation, 1-8.
  29. National Transportation Safety Board, Collapse of I-35W Highway Bridge, Minneapolis, Minnesota (2007).
  30. Noh, Y., Koo, D., Kang, Y., Park, D. and Lee. D. (2017) "Automatic crack detection on concrete images using segmentation via fuzzy C526 means clustering." Proc. of the 2017 International Conference on Applied System Innovation (ICASI), Sapporo, Japan, April, pp. 877-880.
  31. Pauly, L., Hogg, D., Fuentes, R., Peel, H. (2017) "Deeper networks for pavement crack detection." Proc. the 34th International Symposium in Automation and Robotics in Construction, IAARC, Taipei, Taiwan, pp. 479-485.
  32. Prasanna, P., Dana, K. J., Gucunski, N., Basily, B. B., La, H.M, Lim, R.S. and Parvardeh. (2016). Automated crack detection on concrete bridges. IEEE Transactions on Automation Science and Engineering, 13, 591-599. https://doi.org/10.1109/TASE.2014.2354314
  33. Prasanna, P., Kristin J. Dana, Nenad Gucunski, Basily B. Basily, Hung M. La, Ronny Salim Lim, and Hooman Parvardeh. (2016). Automated Crack Detection on Concrete Bridges, IEEE Transactions on Automation Science and Engineering, 13(2), 591-599. https://doi.org/10.1109/TASE.2014.2354314
  34. Rawat, W. and Wang, Z. (2017) Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review. Neural Computing, 29(9), 2352-2449. https://doi.org/10.1162/neco_a_00990
  35. Ren, S., He, K., Girshick, R. and Sun, J. (2017), Faster RCNN: towards real-time object detection with region proposal networks. IEEE Transactions Pattern Analysis and Machine Intelligence, 39, 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031
  36. Rey, N.J. (2016). Combining UAV-imagery and machine learning for wildlife conservation.
  37. Road Bureau, Ministry of Land, Infrastructure, Transportation, and Tourism, Roads in Japan, Retrieved from: http://www.mlit.go.jp/road/road_e/index_e.html.
  38. Siewert, Sam, Mehran Andalibi, Stephen Bruder, Iacopo Gentilini, Aasheesh Dandupally, Soumyatha Gavvala, Omkar Prabhu, Jonathan Buchholz, and Dakota Burklund. (2018). Drone Net, a passive instrument network driven by machine vision and machine learning to automate UAS traffic management. AUVSI Xponential, Denver, Colorado.
  39. Sri Preethaa, K. R. and Sabari, A. (2020), Intelligent video analysis for enhanced pedestrian detection by hybrid metaheuristic approach, Soft Computing, 24(16), 12303-12311. https://doi.org/10.1007/s00500-020-04674-5
  40. Stefania C. Radopoulou and Ioannis Brilakis. (2015). Patch detection for pavement assessment, Automation in Construction, 53, 95-104. https://doi.org/10.1016/j.autcon.2015.03.010
  41. Sudheer, K.P., Gosain, A.K., Mohana Rangan, D and Saheb, S.M. (2002) Modelling evaporation using an artificial neural network algorithm. Hydrological Processes, 16(16), 3189-3202. https://doi.org/10.1002/hyp.1096
  42. Tan, M. and Le, Q. (2019). "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks." Proceedings of the 36th International Conference on Machine Learning, in PMLR, Long Beach, California, 97, pp.6105-6114.
  43. Thikra Dawood, Zhenhua Zhu and Tarek Zayed. (2017). Machine vision-based model for spalling detection and quantification in subway networks. Automation in Construction, 81, 149-160. https://doi.org/10.1016/j.autcon.2017.06.008
  44. Wang, K.C., Zhang, A., Li, J.Q., Fei, Y., Chen, C. and Li, B. (2017). Deep learning for asphalt pavement cracking recognition using convolutional neural network, Airfeld and Highway Pavements, 166-177.
  45. Wu, Stephen Gang, Forrest Sheng Bao, Eric You Xu, Yu-Xuan Wang, Yi-Fan Chang, and Qiao-Liang Xiang. A leaf recognition algorithm for plant classification using probabilistic neural network. in 2007 IEEE international symposium on signal processing and information technology. 2007: IEEE.
  46. Yan, J., Downey, A., Cancelli, A., Laflamme, S., Chen, A., Li, J. and Ubertini, F. (2019). Concrete Crack Detection and Monitoring Using a Capacitive Dense Sensor Array. Sensors, 19, 1843. https://doi.org/10.3390/s19081843
  47. Young?Jin Cha, Wooram Choi and Oral Buyukozturk (2017). Deep learning-based crack damage detection using convolutional neural networks, Computer Aided Civil and Infrastructure Engineering, 32, 361-378. https://doi.org/10.1111/mice.12263
  48. Zaman, B., Jensen, A.M., and McKee, M. (2017). UAV, Machine Learning, and GIS for Wetland Mitigation in Southwestern Utah, USA.
  49. Zhang, C., Patras, P. and Haddadi, H. (2019). Deep Learning in Mobile and Wireless Networking: A Survey. IEEE Communications Surveys & Tutorials, 21(3), pp. 2224-2287. https://doi.org/10.1109/COMST.2019.2904897
  50. Zhang, L., Yang, F., Zhang, Y.D. and Zhu, Y.J. (2016). "Road crack detection using deep convolutional neural network." Proc. in IEEE International conference on Image Processing, Phoenix, AZ, pp. 3708-3712.