The Journal of Korea Robotics Society (로봇학회논문지)

Korea Robotics Society (한국로봇학회)
권호 표지
DOI QR코드

DOI QR Code

Semi-Supervised Domain Adaptation on LiDAR 3D Object Detection with Self-Training and Knowledge Distillation

자가학습과 지식증류 방법을 활용한 LiDAR 3차원 물체 탐지에서의 준지도 도메인 적응

  • Jungwan Woo (Department of Electrical Engineering and Computer Science DGIST) ;
  • Jaeyeul Kim (Department of Electrical Engineering and Computer Science DGIST) ;
  • Sunghoon Im (Department of Electrical Engineering and Computer Science DGIST)
  • Received : 2023.05.02
  • Accepted : 2023.07.05
  • Published : 2023.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

With the release of numerous open driving datasets, the demand for domain adaptation in perception tasks has increased, particularly when transferring knowledge from rich datasets to novel domains. However, it is difficult to solve the change 1) in the sensor domain caused by heterogeneous LiDAR sensors and 2) in the environmental domain caused by different environmental factors. We overcome domain differences in the semi-supervised setting with 3-stage model parameter training. First, we pre-train the model with the source dataset with object scaling based on statistics of the object size. Then we fine-tine the partially frozen model weights with copy-and-paste augmentation. The 3D points in the box labels are copied from one scene and pasted to the other scenes. Finally, we use the knowledge distillation method to update the student network with a moving average from the teacher network along with a self-training method with pseudo labels. Test-Time Augmentation with varying z values is employed to predict the final results. Our method achieved 3rd place in ECCV 2022 workshop on the 3D Perception for Autonomous Driving challenge.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2023-00210908)

References

  1. P. Sun, H. Kretzschmar, X. Dotiwalla, A. Chouard, V. Patnaik, P. Tsui, J. Guo, Y. Zhou, Y. Chai, B. Caine, V. Vasudevan, W. Han, J. Ngiam, H. Zhao, A. Timofeev, S. Ettinger, M. Krivokon, A. Gao, A. Joshi, Y. Zhang, J. Shlens, Z. Chen, and D. Anguelov, "Scalability in Perception for Autonomous Driving: Waymo Open Dataset," Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, USA, 2020, DOI: 10.1109/CVPR42600.2020.00252. 
  2. A. Geiger, P. Lenz, C. Stiller, and R. Urtasun, "Vision meets robotics: The KITTI dataset," The International Journal of Robotics Research, vol. 32, no. 11, Aug., 2013, DOI: 10.1177/0278364913491297. 
  3. H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, "nuScenes: A Multimodal Dataset for Autonomous Driving," Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, USA, 2020, DOI: 10.1109/CVPR42600.2020.01164. 
  4. J. Houston, G. Zuidhof, L. Bergamini, Y. Ye, L. Chen, A. Jain, S. Omari, V. Iglovikov, and P. Ondruska, "One Thousand and One Hours: Self-driving Motion Prediction Dataset," Conference on Robot Learning (CoRL), Cambridge, USA, pp. 409-418, 2020, [Online], https://proceedings.mlr.press/v155/houston21a.html 
  5. M.-F. Chang, J. Lambert, P. Sangkloy, J. Singh, S. Bak, A. Hartnett, D. Wang, P. Carr, S. Lucey, D. Ramanan, and J. Hays, "Argoverse: 3D Tracking and Forecasting With Rich Maps," Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, USA, 2019, DOI: 10.1109/CVPR.2019.00895. 
  6. P. Xiao, Z. Shao, S. Hao, Z. Zhang, X. Chai, J. Jiao, Z. Li, J. Wu, K. Sun, K. Jiang, Y. Wang, and D. Yang, "PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving," International Conference on Intelligent Transportation, Indianapolis, USA, 2021, DOI: 10.1109/ITSC48978.2021.9565009. 
  7. R. Roriz, J. Cabral, and T. Gomes, "Automotive LiDAR Technology: A Survey," IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 6282-6297, Jul., 2022, DOI: 10.1109/TITS.2021.3086804. 
  8. L. Jie, Z. Jin, J. Wang, L. Zhang, and X. Tan, "A SLAM System with Direct Velocity Estimation for Mechanical and Solid-State LiDARs," Remote Sensing, vol. 14, no. 7, Apr., 2022, DOI: 10.3390/rs14071741. 
  9. Y. Wang, X. Chen, Y. You, L. E. Li, B. Hariharan, M. Campbell, K. Q. Weinberger, and W.-L. Chao, "Train in Germany, Test in the USA: Making 3D Object Detectors Generalize," IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, USA, 2020, DOI: 10.1109/CVPR42600.2020.01173. 
  10. T. Yin, X. Zhou, and P. Krahenbuhl, "Center-Based 3D Object Detection and Tracking," Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, USA, 2021, DOI: 10.1109/CVPR46437.2021.01161. 
  11. S. Shi, L. Jiang, J. Deng, Z. Wang, C. Guo, J. Shi, X. Wang, and H. Li, "PV-RCNN++: Point-Voxel Feature Set Abstraction with Local Vector Representation for 3D Object Detection," International Journal of Computer Vision, vol. 131, pp. 531-551, Nov., 2022, DOI: 10.1007/s11263-022-01710-9. 
  12. S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, "PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection," Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, USA, 2020, DOI: 10.1109/CVPR42600.2020.01054. 
  13. S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," International Conference on Machine Learning, Lille, France, pp. 448-456, 2015, DOI: 10.5555/3045118.3045167. 
  14. Y. Yan, Y. Mao, and B. Li, "SECOND: Sparsely Embedded Convolutional Detection," Sensors, vol. 18, no. 10, Oct., 2018, DOI: 10.3390/s18103337.
  15. J.-B. Grill, F. Strub, F. Altche, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko, "Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning," Advances in Neural Information Processing Systems, Red Hook, USA, pp. 21271-21284, 2020, DOI: 10.5555/3495724.3497510.