국제초고층학회논문집 (International Journal of High-Rise Buildings)

  • 제12권4호
  • /
  • Pages.321-333
  • /
  • 2023
  • /
  • 2234-7224(pISSN)
  • /
  • 2288-9930(eISSN)
한국초고층도시건축학회 (Council on Tall Building and Urban Habitat Korea)
권호 표지
DOI QR코드

DOI QR Code

Computer Vision-based Structural Health Monitoring: A Review

  • Jun Su Park (Department of Architecture and Architectural Engineering, Yonsei University) ;
  • Joohyun An (Department of Architecture and Architectural Engineering, Yonsei University) ;
  • Hyo Seon Park (Department of Architecture and Architectural Engineering, Yonsei University)
  • 발행 : 2023.12.29
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Structural health monitoring is a technology or research field that extends the service life of structures and contributes to the prevention of disaster accidents by continuously evaluating the safety, stability, and serviceability of structures as well as allowing timely and proper maintenance. However, the contact-type sensors used for it require considerable time, cost, and labor for installation and maintenance. As an alternative, computer vision has attracted attention recently. Computer vision has the potential to make quality, deformation, and damage monitoring for structures contactless and automated. In this study, research cases in which computer vision was utilized for structural health monitoring are introduced, and its effects and limitations are summarized. Therefore, the applicability and future research directions of computer vision-based structural health monitoring are discussed.

키워드

과제정보

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government Ministry of Science, ICT & Future Planning (MSIP) (No. 2021R1A2C3008989 and 2018R1A5A1025137).

참고문헌

  1. Alcantarilla, P. F., Bartoli, A. and Davison, A. J. (2012). "Kaze Features." Computer Vision - ECCV 2012, Berlin, Heidelberg, 2012//, 214-227.
  2. Azhar, A. S., Kudus, S. A., Jamadin, A., Mustaffa, N. K. and Sugiura, K. (2023). "Recent Vibration-Based Structural Health Monitoring on Steel Bridges: Systematic Literature Review." Ain Shams Engineering Journal, 102501, https://doi.org/10.1016/j.asej.2023.102501.
  3. Badrinarayanan, V., Kendall, A. and Cipolla, R. (2017). "Segnet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation." IEEE transactions on pattern analysis and machine intelligence, 39(12), 2481-2495. https://doi.org/10.1109/TPAMI.2016.2644615
  4. Bay, H., Tuytelaars, T. and Van Gool, L. (2006). "Surf: Speeded up Robust Features." Computer Vision - ECCV 2006, Berlin, Heidelberg, 2006//, 404-417.
  5. Bertinetto, L., Valmadre, J., Henriques, J. F., Vedaldi, A. and Torr, P. H. (2016). "Fully-Convolutional Siamese Networks for Object Tracking." Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part II 14, 850-865.
  6. Canny, J. (1986). "A Computational Approach to Edge Detection." IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8(6), 679-698, 10.1109/TPAMI.1986.4767851.
  7. Choi, J., Kang, D. S., Lee, M. G., Bae, S. G., Hong, T., Lee, D.-E. and Park, H. S. (2023). "Vibration Safety Evaluation Model and Sensor Network-Based Monitoring System for Coke Drums in Operation." Journal of Asian Architecture and Building Engineering, 22(3), 1399-1412, 10.1080/13467581.2022.2085719.
  8. Dalal, N. and Triggs, B. (2005). "Histograms of Oriented Gradients for Human Detection." 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 20-25 June 2005, 886-893 vol. 881, 10.1109/CVPR.2005.177.
  9. Dong, C.-Z. and Catbas, F. N. (2020). "A Review of Computer Vision-Based Structural Health Monitoring at Local and Global Levels." Structural Health Monitoring, 20(2), 692-743, 10.1177/1475921720935585.
  10. Duda, R. O. and Hart, P. E. (1972). "Use of the Hough Transformation to Detect Lines and Curves in Pictures." Commun. ACM, 15(1), 11-15, 10.1145/361237.361242.
  11. Guo, J., Liu, P., Xiao, B., Deng, L. and Wang, Q. (2024). "Surface Defect Detection of Civil Structures Using Images: Review from Data Perspective." Automation in Construction, 158(105186, https://doi.org/10.1016/j.autcon.2023.105186.
  12. Han, K. K. and Golparvar-Fard, M. (2017). "Potential of Big Visual Data and Building Information Modeling for Construction Performance Analytics: An Exploratory Study." Automation in Construction, 73(184-198, https:// doi.org/10.1016/j.autcon.2016.11.004.
  13. Harris, C. and Stephens, M. (1988). "A Combined Corner and Edge Detector." Alvey vision conference, 10-5244.
  14. He, K., Gkioxari, G., Dollar, P. and Girshick, R. (2017). "Mask R-Cnn." Proceedings of the IEEE international conference on computer vision, 2961-2969.
  15. He, K., Zhang, X., Ren, S. and Sun, J. (2016). "Deep Residual Learning for Image Recognition." Proceedings of the IEEE conference on computer vision and pattern recognition, 770-778.
  16. Isola, P., Zhu, J.-Y., Zhou, T. and Efros, A. A. (2017). "Image-to-Image Translation with Conditional Adversarial Networks." Proceedings of the IEEE conference on computer vision and pattern recognition, 1125-1134.
  17. Jianbo, S. and Tomasi (1994). "Good Features to Track." 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 21-23 June 1994, 593- 600, 10.1109/CVPR.1994.323794.
  18. Kardovskyi, Y. and Moon, S. (2021). "Artificial Intelligence Quality Inspection of Steel Bars Installation by Integrating Mask R-Cnn and Stereo Vision." Automation in Construction, 130(103850.
  19. Khuc, T. and Catbas, F. N. (2018). "Structural Identification Using Computer Vision-Based Bridge Health Monitoring." Journal of Structural Engineering, 144(2), 04017202, 10.1061/(ASCE)ST.1943-541X.0001925.
  20. Kim, Y., Park, J. S., Oh, B. K., Cho, T., Kim, J. M., Kim, S. H. and Park, H. S. (2019). "Practical Wireless Safety Monitoring System of Long-Span Girders Subjected to Construction Loading a Building under Construction." Measurement, 146(524-536, 10.1016/j.measurement.2019.05.110.
  21. Krizhevsky, A., Sutskever, I. and Hinton, G. E. (2012). "Imagenet Classification with Deep Convolutional Neural Networks." Advances in neural information processing systems, 25.
  22. Kwan Oh, B., Seon Park, H. and Glisic, B. (2023). "TimeDependent Structural Response Estimation Method for Concrete Structures Using Time Information and Convolutional Neural Networks." Engineering Structures, 275(115193, https://doi.org/10.1016/j.engstruct.2022.115193.
  23. LeCun, Y., Bengio, Y. and Hinton, G. (2015). "Deep Learning." nature, 521(7553), 436-444. https://doi.org/10.1038/nature14539
  24. Lee, H. M., Kim, J. M., Sho, K. and Park, H. S. (2010). "A Wireless Vibrating Wire Sensor Node for Continuous Structural Health Monitoring." Smart Materials and Structures, 19(5), 055004, 10.1088/0964-1726/19/5/055004.
  25. Longuet-Higgins, H. C. and Prazdny, K. (1980). "The Interpretation of a Moving Retinal Image." Proceedings of the Royal Society of London. Series B. Biological Sciences, 208(1173), 385-397. https://doi.org/10.1098/rspb.1980.0057
  26. Lowe, D. G. (1999). "Object Recognition from Local ScaleInvariant Features." Proceedings of the Seventh IEEE International Conference on Computer Vision, 20-27 Sept. 1999, 1150-1157 vol.1152, 10.1109/ICCV.1999.790410.
  27. Lucas, B., D and Kanade, T. (1981). "An Iterative Image Registration Technique with an Application to Stereo Vision." IJCAI'81: 7th international joint conference on Artificial intelligence, Vancouver, Canada, 1981-08-24, 674-679.
  28. Ma, Z., Choi, J. and Sohn, H. (2022). "Real-Time Structural Displacement Estimation by Fusing Asynchronous Acceleration and Computer Vision Measurements." Computer-Aided Civil and Infrastructure Engineering, 37(6), 688-703, https://doi.org/10.1111/mice.12767.
  29. Mirzaei, K., Arashpour, M., Asadi, E., Masoumi, H., Mahdiyar, A. and Gonzalez, V. (2023). "End-to-End Point Cloud-Based Segmentation of Building Members for Automating Dimensional Quality Control." Advanced Engineering Informatics, 55(101878.
  30. Mirzazade, A., Popescu, C., Gonzalez-Libreros, J., Blanksvard, T., Taljsten, B. and Sas, G. (2023). "Semi-Autonomous Inspection for Concrete Structures Using Digital Models and a Hybrid Approach Based on Deep Learning and Photogrammetry." Journal of Civil Structural Health Monitoring, 10.1007/s13349-023-00680-x.
  31. Mishra, M., Lourenco, P. B. and Ramana, G. V. (2022). "Structural Health Monitoring of Civil Engineering Structures by Using the Internet of Things: A Review." Journal of Building Engineering, 48(103954, https://doi.org/10.1016/j.jobe.2021.103954.
  32. Oh, B. K., Glisic, B., Kim, Y. and Park, H. S. (2020). "Convolutional Neural Network-Based Data Recovery Method for Structural Health Monitoring." Structural Health Monitoring, 19(6), 1821-1838, 10.1177/1475921719897571.
  33. Oh, B. K., Hwang, J. W., Kim, Y., Cho, T. and Park, H. S. (2015). "Vision-Based System Identification Technique for Building Structures Using a Motion Capture System." Journal of Sound and Vibration, 356(72-85, https://doi.org/10.1016/j.jsv.2015.07.011.
  34. Oh, B. K., Kim, K. J., Kim, Y., Park, H. S. and Adeli, H. (2017). "Evolutionary Learning Based Sustainable Strain Sensing Model for Structural Health Monitoring of HighRise Buildings." Applied Soft Computing, 58(576-585, https://doi.org/10.1016/j.asoc.2017.05.029.
  35. Oh, B. K. and Park, H. S. (2022). "Urban Safety Network for Long-Term Structural Health Monitoring of Buildings Using Convolutional Neural Network." Automation in Construction, 137(104225.
  36. Olaszek, P. (1999). "Investigation of the Dynamic Characteristic of Bridge Structures Using a Computer Vision Method." Measurement, 25(3), 227-236, https://doi.org/10.1016/S0263-2241(99)00006-8.
  37. Otsu, N. (1979). "A Threshold Selection Method from GrayLevel Histograms." IEEE Transactions on Systems, Man, and Cybernetics, 9(1), 62-66, 10.1109/TSMC.1979.4310076.
  38. Paneru, S. and Jeelani, I. (2021). "Computer Vision Applications in Construction: Current State, Opportunities & Challenges." Automation in Construction, 132(103940.
  39. Park, H. S., Lee, H. M., Adeli, H. and Lee, I. (2007). "A New Approach for Health Monitoring of Structures: Terrestrial Laser Scanning." Computer-Aided Civil and Infrastructure Engineering, 22(1), 19-30, https://doi.org/10.1111/j.1467-8667.2006.00466.x.
  40. Park, H. S. and Oh, B. K. (2018). "Real-Time Structural Health Monitoring of a Supertall Building under Construction Based on Visual Modal Identification Strategy." Automation in Construction, 85(273-289. https://doi.org/10.1016/j.autcon.2017.10.025
  41. Park, H. S., Park, J. S. and Oh, B. K. (2017). "Vision-Based Stress Estimation Model for Steel Frame Structures with Rigid Links." Measurement Science and Technology, 28(7), 075104, 10.1088/1361-6501/aa6f50.
  42. Park, J. S., Oh, B. K., Choi, S. W. and Park, H. S. (2016). "Sensor-Free Stress Estimation Model for Steel Beam Structures Using a Motion Capture System." IEEE Sensors Journal, 16(8), 2701-2713, 10.1109/JSEN.2016.2519033.
  43. Park, J. S. and Park, H. S. (2023). "Automated Reconstruction Model of a Cross-Sectional Drawing from Stereo Photographs Based on Deep Learning." Computer-Aided Civil and Infrastructure Engineering, n/a(n/a), https://doi.org/10.1111/mice.13083.
  44. Park, S. W., Park, H. S., Kim, J. H. and Adeli, H. (2015). "3d Displacement Measurement Model for Health Monitoring of Structures Using a Motion Capture System." Measurement, 59(352-362, https://doi.org/10.1016/j.measurement.2014.09.063.
  45. Paterson, A. M., Dowling, G. R. and Chamberlain, D. A. (1997). "Building Inspection: Can Computer Vision Help?" Automation in Construction, 7(1), 13-20, https://doi.org/10.1016/S0926-5805(97)00031-9.
  46. Prince, S. J. (2012). Computer Vision: Models, Learning, and Inference, Cambridge University Press.
  47. Qi, C. R., Su, H., Mo, K. and Guibas, L. J. (2017). "Pointnet: Deep Learning on Point Sets for 3d Classification and Segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition, 652-660.
  48. Redmon, J., Divvala, S., Girshick, R. and Farhadi, A. (2016). "You Only Look Once: Unified, Real-Time Object Detection." Proceedings of the IEEE conference on computer vision and pattern recognition, 779-788.
  49. Ren, S., He, K., Girshick, R. and Sun, J. (2015). "Faster RCnn: Towards Real-Time Object Detection with Region Proposal Networks." Advances in neural information processing systems, 28(
  50. Roberts, L. G. (1963). Machine Perception of ThreeDimensional Solids, Massachusetts Institute of Technology.
  51. Ronneberger, O., Fischer, P. and Brox, T. (2015). "U-Net: Convolutional Networks for Biomedical Image Segmentation." Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, 234-241.
  52. Rublee, E., Rabaud, V., Konolige, K. and Bradski, G. (2011). "Orb: An Efficient Alternative to Sift or Surf." 2011 International Conference on Computer Vision, 6-13 Nov. 2011, 2564-2571, 10.1109/ICCV.2011.6126544.
  53. Sabato, A., Dabetwar, S., Kulkarni, N. N. and Fortino, G. (2023). "Noncontact Sensing Techniques for Ai-Aided Structural Health Monitoring: A Systematic Review." IEEE Sensors Journal, 23(5), 4672-4684, 10.1109/JSEN.2023.3240092.
  54. Shapiro, L. G. (2020). "Computer Vision: The Last 50 Years." International Journal of Parallel, Emergent and Distributed Systems, 35(2), 112-117, 10.1080/17445760.2018.1469018.
  55. Simonyan, K. and Zisserman, A. (2014). "Very Deep Convolutional Networks for Large-Scale Image Recognition." arXiv preprint arXiv:1409.1556.
  56. Sinha, S. K. and Fieguth, P. W. (2006). "Automated Detection of Cracks in Buried Concrete Pipe Images." Automation in Construction, 15(1), 58-72, https://doi.org/10.1016/j.autcon.2005.02.006.
  57. Sobel, I. and Feldman, G. (1968). "A 3×3 Isotropic Gradient Operator for Image Processing." a talk at the Stanford Artificial Project in, 271-272.
  58. Spencer, B. F., Hoskere, V. and Narazaki, Y. (2019). "Advances in Computer Vision-Based Civil Infrastructure Inspection and Monitoring." Engineering, 5(2), 199-222, https://doi.org/10.1016/j.eng.2018.11.030.
  59. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V. and Rabinovich, A. (2015). "Going Deeper with Convolutions." Proceedings of the IEEE conference on computer vision and pattern recognition, 1-9.
  60. Tomasi, C. and Kanade, T. (1991). "Detection and Tracking of Point Features." Carnegie Mellon University Technical Report CMU-CS-91-132.
  61. Ullman, S. (1979). "The Interpretation of Structure from Motion." Proceedings of the Royal Society of London. Series B. Biological Sciences, 203(1153), 405-426.
  62. Xu, S., Wang, J., Shou, W., Ngo, T., Sadick, A.-M. and Wang, X. (2021). "Computer Vision Techniques in Construction: A Critical Review." Archives of Computational Methods in Engineering, 28(5), 3383-3397, 10.1007/s11831-020-09504-3.
  63. Ye, W., Ren, J., Zhang, A. A. and Lu, C. (2023). "Automatic Pixel-Level Crack Detection with Multi-Scale Feature Fusion for Slab Tracks." Computer-Aided Civil and Infrastructure Engineering, 38(18), 2648-2665, https://doi.org/10.1111/mice.12984.
  64. Yin, Y., Yu, Q., Hu, B., Zhang, Y., Chen, W., Liu, X. and Ding, X. (2023). "A Vision Monitoring System for Multipoint Deflection of Large-Span Bridge Based on Camera Networking." Computer-Aided Civil and Infrastructure Engineering, 38(13), 1879-1891, https://doi.org/10.1111/mice.13050.
  65. Yoneyama, S., Kitagawa, A., Iwata, S., Tani, K. and Kikuta, H. (2007). "Bridge Deflection Measurement Using Digital Image Correlation." Experimental Techniques, 31(1), 34-40, 10.1111/j.1747-1567.2006.00132.x.
  66. Yu, X., Fu, Y., Li, J., Mao, J., Hoang, T. and Wang, H. (2023). "Recent Advances in Wireless Sensor Networks for Structural Health Monitoring of Civil Infrastructure." Journal of Infrastructure Intelligence and Resilience, 100066, https://doi.org/10.1016/j.iintel.2023.100066.
  67. Yun, D. Y., Kim, D., Kim, M., Bae, S. G., Choi, J. W., Shim, H. B., Hong, T., Lee, D.-E. and Park, H. S. (2021). "Field Measurements for Identification of Modal Parameters for High-Rise Buildings under Construction or in Use." Automation in Construction, 121(103446, https://doi.org/10.1016/j.autcon.2020.103446.
  68. Zhu, J.-Y., Park, T., Isola, P. and Efros, A. A. (2017). "Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks." Proceedings of the IEEE international conference on computer vision, 2223-2232.