Journal of Intelligence and Information Systems (지능정보연구)

Korea Intelligent Information System Society (한국지능정보시스템학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning Acoustic Non-line-of-Sight Object Detection

음향신호를 활용한 딥러닝 기반 비가시 영역 객체 탐지

  • Ui-Hyeon Shin (Department of Artificial Intelligence, Sungkyunkwan University) ;
  • Kwangsu Kim (Department of Computer science and Engineering, Sungkyunkwan University)
  • 신의현 (성균관대학교 인공지능학과) ;
  • 김광수 (성균관대학교 소프트웨어학과)
  • Received : 2023.02.02
  • Accepted : 2023.02.19
  • Published : 2023.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, research on detecting objects in hidden spaces beyond the direct line-of-sight of observers has received attention. Most studies use optical equipment that utilizes the directional of light, but sound that has both diffraction and directional is also suitable for non-line-of-sight(NLOS) research. In this paper, we propose a novel method of detecting objects in non-line-of-sight (NLOS) areas using acoustic signals in the audible frequency range. We developed a deep learning model that extracts information from the NLOS area by inputting only acoustic signals and predicts the properties and location of hidden objects. Additionally, for the training and evaluation of the deep learning model, we collected data by varying the signal transmission and reception location for a total of 11 objects. We show that the deep learning model demonstrates outstanding performance in detecting objects in the NLOS area using acoustic signals. We observed that the performance decreases as the distance between the signal collection location and the reflecting wall, and the performance improves through the combination of signals collected from multiple locations. Finally, we propose the optimal conditions for detecting objects in the NLOS area using acoustic signals.

최근 관찰자의 직접적인 시야 밖의 숨겨진 공간의 물체를 탐지하는 비가시 영역 객체 탐지 연구가 주목받고 있다. 대부분의 연구들은 빛의 직진성을 활용한 광학장비를 사용하지만, 회절성과 직진성을 모두 갖춘 소리 또한 비가시 영역연구에 적합하다. 본 논문에서는 가청 주파수 범위의 음향 신호를 활용하여 비가시 영역의 객체를 탐지하는 새로운 방법을 제안한다. 음향 신호만을 입력하여 비가시 영역에서 정보를 추출하고 숨겨진 물체의 종류와 범위를 예측하는 딥러닝 모델을 설계한다. 또한 딥러닝 모델의 훈련 및 평가를 위해 총 11개 물체에 대한 신호의 송 수신 위치를 변경하여 데이터를 수집한다. 이를 통해, 입력 데이터 변화에 따른 물체의 분류 정확도 및 탐지 성능을 비교한다. 우리는 딥러닝 모델이 음향신호를 활용히여 비가시 영역 객체 탐지하는데 우수한 성능을 보임을 증명한다. 신호 수집 위치와 반사벽 사이 거리가 멀어질수록 성능이 저하되고, 여러 위치에서 수집된 신호의 결합을 통해 성능이 향상되는 것을 관찰한다. 마지막으로, 음향 신호를 활용하여 비가시 영역 객체 탐지를 위한 최적의 조건을 제시한다.

Keywords

Acknowledgement

이 논문은 2023년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원(No.2019-0-00421, 인공지능대학원지원(성균관대학교))과 정보통신산업진흥원의 지원(No.S0102-23-1012, 헬스케어 AI 융합 연구개발)을 받아 수행된 연구임

References

  1. 김기태, 이보미, 김종우. (2017). 이진 분류문제에서의 딥러닝 알고리즘의 활용 가능성 평가. 지능정보연구, 23(1), 95-108. https://doi.org/10.13088/JIIS.2017.23.1.095
  2. 김호준. (2010). CNN 모델과 FMM 신경망을 이용한 동적 수신호 인식 기법. 지능정보연구, 16(2), 95-108.
  3. 안성만. (2016). 딥러닝의 모형과 응용사례. 지능정보연구, 22(2), 127-142. https://doi.org/10.13088/JIIS.2016.22.2.127
  4. 장승우, & 김광수. (2021). 가청 주파수를 활용한 비가시 영역 복원 기초 연구. 한국통신학회 학술대회논문집, 1088-1089.
  5. Alakwaa, W., Nassef, M., & Badr, A. (2017). Lung cancer detection and classification with 3D convolutional neural network (3D-CNN). International Journal of Advanced Computer Science and Applications, 8(8).
  6. Boger-Lombard, J., Slobodkin, Y., & Katz, O. (2022). Non-Line-of-Sight Passive Acoustic Localization Around Corners. arXiv preprint arXiv:2211.03453.
  7. Grau Chopite, J., Hullin, M. B., Wand, M., & Iseringhausen, J. (2020). Deep non-line-of-sight reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 960-969).
  8. Hearst, M. A., Dumais, S. T., Osuna, E., Platt, J., & Scholkopf, B. (1998). Support vector machines. IEEE Intelligent Systems and their applications, 13(4), 18-28. https://doi.org/10.1109/5254.708428
  9. Kan, R., Wang, M., Zhou, Z., Zhang, P., & Qiu, H. (2022). Acoustic Signal NLOS Identification Method Based on Swarm Intelligence Optimization SVM for Indoor Acoustic Localization. Wireless Communications and Mobile Computing, 2022.
  10. Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  11. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90. https://doi.org/10.1145/3065386
  12. LeCun, Y. (2015). LeNet-5, convolutional neural networks. URL: http://yann.lecun.com/exdb/lenet, 20(5), 14.
  13. Lindell, D. B., Wetzstein, G., & Koltun, V. (2019). Acoustic non-line-of-sight imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6780-6789).
  14. Liu, X., Bauer, S., & Velten, A. (2020). Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems. Nature communications, 11(1), 1645.
  15. Mann, S., & Haykin, S. (1991, June). The chirplet transform: A generalization of Gabor's logon transform. In Vision interface (Vol. 91, pp. 205-212). Princeton, NJ, USA: Citeseer.
  16. Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., ... & Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499.
  17. Rao, C., & Liu, Y. (2020). Three-dimensional convolutional neural network (3D-CNN) for heterogeneous material homogenization. Computational Materials Science, 184, 109850.
  18. Raspaud, M., Viste, H., & Evangelista, G. (2009). Binaural source localization by joint estimation of ILD and ITD. Ieee transactions on audio, speech, and language processing, 18(1), 68-77. https://doi.org/10.1109/TASL.2009.2023644
  19. Tang, W., Long, G., Liu, L., Zhou, T., Jiang, J., & Blumenstein, M. (2020). Rethinking 1d-cnn for time series classification: A stronger baseline. arXiv preprint arXiv:2002.10061, 1-7.
  20. Thaler, L. and Goodale, M. A. (2016). Echolocation in humans: an overview. Wiley Interdisciplinary Reviews: Cognitive Science, 7(6), 382-393. https://doi.org/10.1002/wcs.1408
  21. Velten, A., Willwacher, T., Gupta, O., Veeraraghavan, A., Bawendi, M. G., & Raskar, R. (2012). Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature communications, 3(1), 745.
  22. Wu, C., Liu, J., Huang, X., Li, Z. P., Yu, C., Ye, J. T., ... & Pan, J. W. (2021). Non-line-of-sight imaging over 1.43 km. Proceedings of the National Academy of Sciences, 118(10), e2024468118.
  23. Xu, B., Wang, N., Chen, T., & Li, M. (2015). Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853.
  24. Xie, J., Hu, K., Zhu, M., Yu, J., & Zhu, Q. (2019). Investigation of different CNN-based models for improved bird sound classification. IEEE Access, 7, 175353-175361. https://doi.org/10.1109/access.2019.2957572
  25. Zhang, L., Huang, D., Wang, X., Schindelhauer, C., & Wang, Z. (2017). Acoustic NLOS identification using acoustic channel characteristics for smartphone indoor localization. Sensors, 17(4), 727.
  26. Zhang, Z., & Sabuncu, M. (2018). Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems, 31.
  27. Kim, C. S. and J. H. Lee, Syntactic Pattern Recognition, Prentice-Hall, New Jersey, 1985.
  28. Rohlf, F. J., tpsDig Version 2.10, Department of Ecology and Evolution, State University of New York at Stony Brook, 2006. Available at http://life.bio.sunysb.edu/morph/ (Downloaded 29 September, 2008).
  29. Wu, K.-L., C. C. Agarwal, and P. S. Yu, "Personalization with dynamic profiler," Proceedings of the Third International Workshop on Advanced Issues of E-commerce and Web-based Information Systems, (2001), 12~20.