Journal of Platform Technology

ICT Platform Society (ICT플랫폼학회)
권호 표지

Deep Learning-based Approach for Visitor Detection and Path Tracking to Enhance Safety in Indoor Cultural Facilities

실내 문화시설 안전을 위한 딥러닝 기반 방문객 검출 및 동선 추적에 관한 연구

  • 신원섭 (중앙대학교 첨단영상대학원 엔터테인먼트-테크놀로지 전공) ;
  • 노승민 (중앙대학교 산업보안학과)
  • Received : 2023.07.11
  • Accepted : 2023.08.03
  • Published : 2023.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

In the post-COVID era, the importance of quarantine measures is greatly emphasized, and accordingly, research related to the detection of mask wearing conditions and prevention of other infectious diseases using deep learning is being conducted. However, research on the detection and tracking of visitors to cultural facilities to prevent the spread of diseases is equally important, so research on this should be conducted. In this paper, a convolutional neural network-based object detection model is trained through transfer learning using a pre-collected dataset. The weights of the trained detection model are then applied to a multi-object tracking model to monitor visitors. The visitor detection model demonstrates results with a precision of 96.3%, recall of 85.2%, and an F1-score of 90.4%. Quantitative results of the tracking model include a MOTA (Multiple Object Tracking Accuracy) of 65.6%, IDF1 (ID F1 Score) of 68.3%, and HOTA (Higher Order Tracking Accuracy) of 57.2%. Furthermore, a qualitative comparison with other multi-object tracking models showcased superior results for the model proposed in this paper. The research of this paper can be applied to the hygiene systems within cultural facilities in the post-COVID era.

포스트-코로나 시대에는 방역 조치의 중요성이 크게 강조되고 있으며, 이에 맞춰 딥러닝을 이용한 마스크 착용 상태 검출 및 다른 전염병 예방에 관련된 연구가 진행되고 있다. 그러나 질병 확산 방지를 위한 문화시설 관람객 탐지 및 추적 연구도 마찬가지로 중요하므로 이에 대한 연구가 진행되어야 한다. 본 논문에서는 사전 수집된 데이터 셋을 이용하여 컨볼루션 신경망 기반 객체 탐지 모델을 전이 학습시키고, 학습된 탐지 모델의 가중치를 다중 객체 추적 모델에 적용하여 방문객을 모니터링 한다. 방문객 탐지 모델은 Precision 96.3%, Recall 85.2% F1-Score 90.4%의 결과를 보여주었다. 추적 모델의 정량적 결과로 MOTA 65.6%, IDF1 68.3%. HOTA 57.2%의 결과를 보여주었으며, 본 논문의 모델과 다른 다중 객체 추적 모델 간의 정성적 비교에서 우수한 결과를 보여주었다. 본 논문의 연구는 포스트-코로나 시대의 문화시설 내 방역 시스템에 적용될 수 있을 것이다.

Keywords

Acknowledgement

이 논문은 2023 년도 중앙대학교 연구장학기금 지원에 의한 것임.

References

  1. LeCun, Y., Bengio, Y., and Hinton, G. "Gradient-based learning applied to document recognition." Proceedings of the IEEE, 86(11), pp. 2278-2324. doi: 10.1109/5.726791, 1998.
  2. J. Talukdar, S. Gupta, P. S. Rajpura, and R. S. Hegde, "Transfer Learning for Object Detection using State-ofthe-Art Deep Neural Networks," in Proceeding of the 5th International Conference on Signal Processing and Integrated Networks, Noida, pp. 78-83, doi: 10.1109/SPIN.2018.8474198, 2018.
  3. S. H. Lee, H. G. Kwon, Y. J. Kim, J. S. Jeong, and H. J. Seo, "Development of CCTV for Identification of Maskless Wearers based on Deep Learning," in Proceeding of the 28th Korea Society of Computer Information, Korea, pp. 317-318, doi: 10.6109/jkiice.2021.25.1.44, 2020.
  4. Sunwoo Shin, Woosung Jung, Taemin Lee and Sanghyun Seo. "A Study on Detecting Mask Wearing Status using Ensemble based on Deep Learning" Journal of Digital Contents Society. Vol 22, No.11 pp. 1931-1939, doi: 10.9728/dcs.2021.22.11.1931, November 2021.
  5. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation," 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, pp. 580-587, doi: 10.1109/CVPR.2014.81, 2014.
  6. M. A. Hearst, S.T. Dumais, E. Osuna, J. Platt, and B. Scholkopf, "Support vector machines," in IEEE Intelligent Systems and their Applications, vol. 13, no. 4, pp. 18-28, doi: 10.1109/5254.708428, July-Aug. 1998.
  7. R. Girshick, "Fast R-CNN," 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, pp. 1440-1448, doi: 10.1109/ICCV.2015.169, 2015.
  8. He, K., Gkioxari, G., Dollar, P., and Girshick, R. "Mask R-CNN" Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2980-2988, doi: 10.1109/ICCV.2017.322., 2017.
  9. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. "You only look once: Unified, real-time object detection." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 779-788, doi: 10.1109/CVPR.2016.91, 2016.
  10. Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. "Simple online and realtime tracking." Proceedings of the IEEE International Conference on Image Processing (ICIP), pp. 3464-3468, doi: 10.1109/ICIP.2016.7533003, 2016.
  11. Ren, S., He, K., Girshick, R., and Sun, J. "Faster R-CNN: Towards real-time object detection with region proposal networks." Proceedings of the Neural Information Processing Systems (NIPS) Conference, pp. 91-99, doi: 10.1109/ICCV.2015.169, 2015.
  12. Woehlke, S., Buttner, L., Kreuzinger, N, and Leal-Taixe, L. "DeepSORT: A Simple Online and Realtime Tracking with Re-identification." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2573-2582, doi: 10.1109/CVPR42600.2020.00264, 2020.
  13. Nir Aharon, Roy Orfaig, and Ben-Zion Bobrovsky, "BoT-SORT: Robust Associations Multi-Pedestrian Tracking" arXiv, doi: arXiv:2206.14651, 2022.
  14. Wang, Chien-Yao & Bochkovskiy, Alexey, and Liao, Hong-yuan. "YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors."arXiv, doi: 10.48550/arXiv.2207.02696, 2022.
  15. He, L., Liao, X., Liu, W., Liu, X., Cheng, P, and Mei, T. (2020). "FastReID: A Pytorch Toolbox for General Instance Re-identification. arXiv, abs/2006.02631, 2020.
  16. Bernardin, K., Stiefelhagen, R. "Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics." J Image Video Proc 2008, 246309, doi: 10.1155/2008/246309, 2008.
  17. Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C., "Performance Measures and a Data Set for Multi-target, Multi-camera Tracking." ECCV vol 9914, doi: 10.1007/978-3-319-48881-3_2, 2016.
  18. Luiten, J., Osep, A., Dendorfer, P., Torr, P., Geiger, A., Leal-Taixe, L., & Leibe, B, "HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking." International Journal of Computer Vision, 129(2), pp. 548-578, doi: 10.1007/s11263-020-01390-7, 2021