The Journal of the Acoustical Society of Korea (한국음향학회지)

The Acoustical Society of Korea (한국음향학회)
권호 표지
DOI QR코드

DOI QR Code

Dilated convolution and gated linear unit based sound event detection and tagging algorithm using weak label

약한 레이블을 이용한 확장 합성곱 신경망과 게이트 선형 유닛 기반 음향 이벤트 검출 및 태깅 알고리즘

  • 박충호 (고려대학교 전기전자공학과 지능신호처리 연구실) ;
  • 김동현 (고려대학교 전기전자공학과 지능신호처리 연구실) ;
  • 고한석 (고려대학교 전기전자공학과 지능신호처리 연구실)
  • Received : 2020.07.24
  • Accepted : 2020.08.27
  • Published : 2020.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a Dilated Convolution Gate Linear Unit (DCGLU) to mitigate the lack of sparsity and small receptive field problems caused by the segmentation map extraction process in sound event detection with weak labels. In the advent of deep learning framework, segmentation map extraction approaches have shown improved performance in noisy environments. However, these methods are forced to maintain the size of the feature map to extract the segmentation map as the model would be constructed without a pooling operation. As a result, the performance of these methods is deteriorated with a lack of sparsity and a small receptive field. To mitigate these problems, we utilize GLU to control the flow of information and Dilated Convolutional Neural Networks (DCNNs) to increase the receptive field without additional learning parameters. For the performance evaluation, we employ a URBAN-SED and self-organized bird sound dataset. The relevant experiments show that our proposed DCGLU model outperforms over other baselines. In particular, our method is shown to exhibit robustness against nature sound noises with three Signal to Noise Ratio (SNR) levels (20 dB, 10 dB and 0 dB).

본 논문은 약한 레이블 기반 음향 이벤트 검출을 위한 시간-주파수 영역분할 맵 추출 모델에서 발생하는 희소성 및 수용영역 부족에 관한 문제를 완화 시키기 위해, 확장 게이트 선형 유닛(Dilated Convolution Gated Linear Unit, DCGLU)을 제안한다. 딥러닝 분야에서 음향 이벤트 검출을 위한 영역분할 맵 추출 기반 방법은 잡음 환경에서 좋은 성능을 보여준다. 하지만, 이 방법은 영역분할 맵을 추출하기 위해 특징 맵의 크기를 유지해야 하므로 풀링 연산 없이 모델을 구성하게 된다. 이로 인해 이 방법은 희소성과 수용영역의 부족으로 성능 저하를 보이게 된다. 이런 문제를 완화하기 위해, 본 논문에서는 정보의 흐름을 제어할 수 있는 게이트 선형 유닛과 추가의 파라미터 없이 수용영역을 넓혀 줄 수 있는 확장 합성곱 신경망을 적용하였다. 실험을 위해 사용된 데이터는 URBAN-SED와 자체 제작한 조류 울음소리 데이터이며, 제안하는 DCGLU 모델이 기존 베이스라인 논문들보다 더 좋을 성능을 보였다. 특히, DCGLU 모델이 자연 소리가 섞인 환경인 세 개의 Signal to Noise Ratio(SNR)(20 dB, 10 dB, 0 dB)에서 강인하다는 것을 확인하였다.

Keywords

References

  1. P. K. Atrey, N. C. Maddage, and M. S. Kankanhlli, "Audio based event detection for multimedia surveillance," Proc. IEEE ICASSP. 5. V-V (2006).
  2. J. Maxime, X. Alameda-Pineda, L. Girin, and R. Horaud, "Sound representation and classification benchmark for domestic robots," Proc. IEEE ICRA. 6285-6292 (2014).
  3. D. Stowell, M. Wood, Y. Stylianou, and H. Glotin, "Bird detection in audio: a survey and a challenge," Proc. IEEE 26th MLSP. 1-6 (2016).
  4. D. Stowell and M. D. Plumbley, "Audio-only bird classification using unsupervised feature learning," Proc. CLEF. 673-684 (2014).
  5. K. Ko, J. Park, D. K. Han, and H. Ko, "Channel and frequency attention module for diverse animal sound classification," IEICE Trans. on Information and Systems, E102-D, 2615-2618 (2019). https://doi.org/10.1587/transinf.2019EDL8128
  6. S. Park, M. Elhilali, D. K. Han, and H. Ko, "Amphibian sounds generating network based on adversarial learning," IEEE Signal Processing Letters, 27, 640-644 (2020). https://doi.org/10.1109/LSP.2020.2988199
  7. K. Ko, S. Park, and H. Ko, "Convolutional neural netework based amphibian sound classification using covariance and modulogram" (in Korean), J. Acoust. Soc. Kr. 37, 61-65 (2018).
  8. D. Stowell, D. Giannoulis, E. Benetos, M. Lagrange, and M. D. Plumbley, "Detection and classification of acoustic scenes and events," IEEE Trans. Multimedia, 17, 1733-1746 (2015). https://doi.org/10.1109/TMM.2015.2428998
  9. G. Parascandolo, H. Huttunen, and T. Virtanen, "Recurrent neural networks for polyphonic sound event detection in real life recordings," Proc. IEEE ICASSP. 6440-6444 (2016).
  10. A. Mesaros, T. Heittola, and T. Virtanen, "TUT database for acoustic scene classification and sound event detection," Proc. 24th EUSIPCO. 1128-1132 (2016).
  11. S.-Y. Chou, J.-S. R. Jang, and Y.-H. Yang, "Frame CNN: A weakly-supervised learning framework for frame-wise acoustic event detetion and classification," DACSE. Tech. Rep., 2017.
  12. A. Kumar and B. Raj, "Deep cnn framework for audio event recognition using weak labeled web data," arXiv: 1707.02530 (2017).
  13. Y. Xu, Q. Kong, W. Wang, and M. D. Plumbley, "Large-scale weakly supervised audio classification using gated convolutional neural network," Proc. IEEE ICASSP. 121-125 (2018).
  14. Q. Kong, Y. Xu, I. Sobieraj, W. Wang, and M. D. Plumbley, "Sound event detection and time-frequency segmentation from weak labelled data," IEEE/ACM Trans. on Audio, Speech, And Lang. Processing, 27, 777-787 (2019). https://doi.org/10.1109/TASLP.2019.2895254
  15. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv: 1409.1556 (2014).
  16. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," Proc. IEEE CVPR. 770-778 (2016).
  17. Y. N. Dauphin, A. Fan, M. Auli, and D. Grangier, "Language modeling with gated convolutional networks," Proc. PMLR. 70, 933-941 (2017).
  18. Y. Chen, Q. Guo, X. Liang, J. Wang, and Y. Qian, "Environmental sound classification with dilated convolutions," Applied Acoustics, 148, 123-132 (2019). https://doi.org/10.1016/j.apacoust.2018.12.019
  19. J. Salamon, D. MacConnell, M. Cartwright, P. Li, and J. P. Bello, "SCAPER:a library for soundscape synthesis and augmentation," Proc. IEEE WASPAA. 344-348 (2017).
  20. A. Kolesnikov and C. H. Lampert, "Seed, expand and constrain: Three principles for weakly-supervised image segmentation," Proc. ECCV. 695-711 (2016).
  21. Q. Kong, T. Iqbal, Y. Xu, W. Wang, and M. D. Plumbley, "DCASE 2018 challenge baseline with convolutional neural networks," DACSE. Tech. Rep., 2018.
  22. K. Miyazaki, T. Komatsu, T. Hayashi, S. Watanabe, T. Tuda, and K. Takeda, "Weakly-supervised sound event detection with self-attention," Proc. IEEE ICASSP. 66-70 (2020).
  23. Y. Li, M. Liu, K. Drossos, and T. Virtanen, "Sound event detection via dilated convolutional recurrent neural networks," Proc. IEEE ICASSP. 286-290 (2020).
  24. D. Kingma and J. Ba, "Adam: a method for stochastic optimization," arXiv:1412.6980 (2015).
  25. S. Ioffe and C. Szegedy, "Batch normalization: accelerating deep network training by reducing internal covariate shift," Proc. 32nd ICML. 448-456 (2015).
  26. J. A. Hanley and B. J. McNeil, "The meaning and use of the area under a receiver operating characteristic (ROC) curve," Radiology, 431, 29-36 (1982).
  27. R. Girshich, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," Proc. IEEE CVPR. 580-587 (2014).