2. Enhancing LiDAR Data Colorization with Image Semantics for Improved Graph-SLAM Techniques in Mobile Mapping

Enhancing LiDAR Data Colorization with Image Semantics for Improved Graph-SLAM Techniques in Mobile Mapping

Robotics
ACQUAL
Staff Involved
Ville Lehtola
Additional Remarks

Students should possess strong programming skills in Python and C++, and familiarity with Linux and Docker is beneficial, though not required. The multi-sensor backpack system is equipped with 2 x VLP16 lidars, Ladybug5+ HD camera, BD990 Trimble GNSS, and XSens MTI 630R IMU.

Topic description

While Graph-SLAM techniques have significantly contributed to mobile mapping advancements in indoor and urban environments, there remains a need to improve the quality of data generated. One such approach is the integration of image semantics to colorize LiDAR data, which could yield more accurate and informative maps. This thesis will explore the state of the art in Graph-SLAM techniques and develop novel methods for colorizing LiDAR data using image semantics in mobile mapping applications employing a multi-sensor system.

Topic objectives and methodology

The primary objective of this thesis is to enhance Graph-SLAM techniques by integrating image semantics [1] for LiDAR data colorization in mobile mapping for indoor and urban environments, see e.g. [2]. The multi-sensor system provided by the laboratory may be utilized for this purpose. The research will encompass:

  1. Reviewing state-of-the-art Graph-SLAM techniques and existing LiDAR data colorization methods.
  2. Developing a robust method for LiDAR data colorization using image semantics, and implementing it in C++ and Python.
  3. Assessing the performance of the proposed method against existing SLAM techniques and LiDAR data colorization methods using available datasets or new data collected by the student.

By integrating image semantics with LiDAR data colorization, the resulting maps will potentially provide richer information [2,3], enabling improved Graph-SLAM techniques for mobile mapping applications.

References for further reading

[1] https://github.com/CSAILVision/semantic-segmentation-pytorch

[2] Berrio, J. S., Shan, M., Worrall, S., & Nebot, E. (2021). Camera-LIDAR integration: Probabilistic sensor fusion for semantic mapping. IEEE Transactions on Intelligent Transportation Systems, 23(7), 7637-7652.

[3] Hughes, N., Chang, Y., & Carlone, L. (2022). Hydra: A real-time spatial perception engine for 3d scene graph construction and optimization. arXiv preprint arXiv:2201.13360.