Journal of the Semiconductor & Display Technology (반도체디스플레이기술학회지)

The Korean Society Of Semiconductor & Display Technology (한국반도체디스플레이기술학회)
권호 표지

Design of Indoor Space Guidance System Using LiDAR and Camera on iPhone

iPhone의 LiDAR와 Camera를 이용한 실내 공간 안내를 위한 시스템 설계

  • 장준석 (상명대학교 소프트웨어학과) ;
  • 성광제 (상명대학교 소프트웨어학과)
  • Received : 2024.02.16
  • Accepted : 2024.03.20
  • Published : 2024.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

In indoor environments, since global positioning system (GPS) signals can be blocked by obstacles, such as building structure. the performance of GPS-based positioning methods can be degraded because of the loss of GPS signals. To solve this problem, various localization schemes using inertial measurement unit (IMU) sensors, such as gyroscope, accelerometer, and magnetometer, have been proposed to enhance the positioning accuracy in indoor environments. IMU-based positioning methods can estimate the location of the user by calculating the velocity and heading angle of the user without the help of GPS. However, low-cost MEMS IMUs may lead to drift error and large bias. In addition, positioning errors in IMU-based positioning approaches can be caused by the irrelevant motion of the pedestrian. In this study, we propose an enhanced indoor positioning method that provides more reliable localization results by using the camera, light detection and right (LiDAR), and ARKit framework on the iPhone. Through reliable positioning results and augmented reality (AR) experiences, our indoor positioning system can provide indoor space guidance services.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2022R1G1A1013168).

References

  1. Kim, H. Y. and Yang, O., "Design of Inter- Regional Instrument Group-B Decoder Based on FPGA for Time Synchronous," Journal of the Semiconductor & Display Technology, Vol. 18, No. 1, pp. 59-64, 2019.
  2. Lee, J.-M., Lee, Y.-H., Choi, N.-K., Park, H., and Kim, H.-C., "Deep-Learning-based Plant Anomaly Detection using a Drone," Journal of the Semiconductor & Display Technology, Vol. 20, No. 1, pp. 94-98, 2021.
  3. Ye, F., Pan, S., Gao, W., Wang, H., Liu, G., Ma, C., and Wang, Y., "An improved single-epoch GNSS/INS positioning method for urban canyon environment based on real-time DISB estimation," IEEE Access, Vol. 8, pp. 227566-227578, 2020.
  4. Yang, Z., Wu, C., Zhou, Z., Zhang, X., Wang, X., and Liu, Y., "Mobility Increases Localizability: A Survey on Wireless Indoor Localization Using Inertial Sensors," ACM Comput. Surv., Vol. 47, No. 3, pp. 1-34, 2015.
  5. Han, H., Yi, S., Li, Q., Shen, G., Liu, Y., and Novak, E., "AMIL: Localizing neighboring mobile devices through a simple gesture," Proc. INFOCOM, pp. 1-9, Apr. 2016.
  6. Oufqir, Z., El Abderrahmani, A., and Satori, K., "ARKit and ARCore in serve to augmented reality," Proc. Int. Conf. Intell. Syst. Comput. Vis. (ISCV), pp. 1-7, Jun. 2020.
  7. Apple, "ARKit 6," Retrieved November 20, 2023, from https://developer.apple.com/kr/augmented-reality/arkit.
  8. VMware, "Spring (Version 6.0.12)," Retrieved January 12, 2024, from https://spring.io.
  9. Apple, "Displaying a Point Cloud Using Scene Depth," Retrieved October 15, 2023, from https://developer.apple.com/documentation/arkit/arkit_in_ios/environmental_analysis/displaying_a_point_cloud_using_scene_depth.
  10. Apple, "Reality Composer," Retrieved October 20, 2023, from https://apps.apple.com/us/app/reality-composer/id1462358802.
  11. Sung, K., Lee, D.K.'R., and Kim, H., "Indoor Pedestrian Localization Using iBeacon and Improved Kalman Filter," Sensors, Vol. 18, No. 6, pp. 1722, 2018.
  12. Apple, "RealityKit," Retrieved October 20, 2023, from https://developer.apple.com/kr/augmentedreality/realitykit/
  13. Apple, "ARView," Retrieved October 20, 2023, from https://developer.apple.com/documentation/realitykit/arview
  14. Apple, "View Matrix," Retrieved October 23, 2023, from https://developer.apple.com/documentation/arkit/arcamera/2921672-viewmatrix/
  15. 3D Game Engine Programming, "Understanding the View Matrix," Retrieved October 23, 2023, from https://www.3dgep.com/understanding-the-view-matrix/
  16. WWDC 2018, "Understanding ARKit Tracking and Detection," Retrieved October 26, 2023, from https://nilotic.github.io/2018/08/13/UnderstandingARKit-Tracking-and-Detection.html
  17. Appleinsider, "Inside Apple's ARKit and Visual Inertial Odometry, new in iOS 11," Retrieved October 27, 2023, from https://appleinsider.com/articles/17/10/12/insideapples-arkit-and-visual-inertial-odometry-new-in-ios-11
  18. Tong Qin, Peiliang Li, Shaojie Shen. VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator. IE EE Transactions on Robotics, 34(4), 1004-1020. 2018.
  19. Christian Forster, Luca Carlone, Frank Dellaert, Davide Sc aramuzza. On-Manifold Preintegration for Real-Time Visu al--Inertial Odometry. IEEE Transactions on Robotics, 33 (1), 1-21. 2016
  20. Seiskari, O.; Rantalankila, P.; Kannala, J.; Ylilammi, J.; Rahtu, E.; Solin, A. HybVIO: Pushing the Limits of Real-Time Visual-Inertial Odometry. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 4-8 January 2022; pp. 701-710.
  21. Delmerico, J.; Scaramuzza, D. A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21-25 May 2018; pp. 2502-2509.
  22. Von Stumberg, L.; Usenko, V.; Cremers, D. Direct Sparse Visual-Inertial Odometry Using Dynamic Marginalization. In Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21-25 May 2018; pp. 2510-2517.
  23. Chen, C.; Zhu, H.; Li, M.; You, S. A Review of Visual-Inertial Simultaneous Localization and Mapping from Filtering-Based and Optimization-Based Perspectives. Robotics 2018, 7, 45.
  24. Xie, H.; Gu, T.; Tao, X.; Ye, H.; Lu, J. A Reliability-Augmented Particle Filter for Magnetic Fingerprinting Based Indoor Localization on Smartphone. IEEE Trans. Mob. Comput. 2016, 15, 1877-1892.
  25. Shu, Y.; Huang, Y.; Zhang, J.; Coue, P.; Cheng, P.; Chen, J.; Shin, K.G. Gradient-Based Fingerprinting for Indoor Localization and Tracking. IEEE Trans. Ind. Electron. 2016, 63, 2424-2433.
  26. He, S.; Chan, S.H.G.; Yu, L.; Liu, N. SLAC: Calibration-Free Pedometer-Fingerprint Fusion for Indoor Localization. IEEE Trans. Mob. Comput. 2018, 17, 1176-1189.
  27. Sung, K.; Lee, D.K.R.; Kim, H. Indoor Pedestrian Localization Using iBeacon and Improved Kalman Filter. Sensors 2018, 18, 1722.