Journal of Internet Computing and Services (인터넷정보학회논문지)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Robust Staff Line Height and Staff Line Space Estimation for the Preprocessing of Music Score Recognition

악보인식 전처리를 위한 강건한 오선 두께와 간격 추정 방법

  • Na, In-Seop (School of Electronics and Computer Engineering, Chonnam National Univ.) ;
  • Kim, Soo-Hyung (School of Electronics and Computer Engineering, Chonnam National Univ.) ;
  • Nquyen, Trung Quy (School of Electronics and Computer Engineering, Chonnam National Univ.)
  • Received : 2014.10.14
  • Accepted : 2014.11.01
  • Published : 2015.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a robust pre-processing module for camera-based Optical Music Score Recognition (OMR) on mobile device. The captured images likely suffer for recognition from many distortions such as illumination, blur, low resolution, etc. Especially, the complex background music sheets recognition are difficult. Through any symbol recognition system, the staff line height and staff line space are used many times and have a big impact on recognition module. A robust and accurate staff line height and staff line space are essential. Some staff line height and staff line space are proposed for binary image. But in case of complex background music sheet image, the binarization results from common binarization algorithm are not satisfactory. It can cause incorrect staff line height and staff line space estimation. We propose a robust staff line height and staff line space estimation by using run-length encoding technique on edge image. Proposed method is composed of two steps, first step, we conducted the staff line height and staff line space estimation based on edge image using by Sobel operator on image blocks. Each column of edge image is encoded by run-length encoding algorithm Second step, we detect the staff line using by Stable Path algorithm and removal the staff line using by adaptive Line Track Height algorithm which is to track the staff lines positions. The result has shown that robust and accurate estimation is possible even in complex background cases.

이 논문에서는 모바일 기기상에서 카메라기반 악보인식을 위한 오선 두께와 오선 간격을 추정하는 전처리 기술을 제안한다. 캡쳐된 영상은 조명이나, 흐려짐, 저해상도 등의 많은 왜곡으로 인해 인식에 어려움이 있다. 특히 복잡한 배경을 가지고 있는 악보 영상인식의 경우 더욱 그렇다. 악보 기호 인식에서 오선 두께와 오선 간격은 인식에 큰 영향을 끼친다. 이들 정보는 이진화에도 사용되는데, 복잡한 배경을 가지고 있는 경우 일반적인 이진 영상은 오선 두께와 간격을 추정하는데 만족스럽지 못하다. 따라서 우리는 에지영상에서 런-길이 엔코딩 기술을 이용해 오선 두께와 간격 추정하는 강건한 알고리즘을 제안한다. 제안된 방법은 2단계로 구성되어 있다. 첫 번째 단계는 소벨 연산자에 의해 영역별로 에지 영상을 기반으로 오선 두께와 간격을 추정한다. 각 에지 영상의 열은 런-길이 엔코딩 알고리즘에 의해 기술된다. 두 번째 단계는 안정한 경로 알고리즘을 이용한 오선 검출과 오선 위치를 추적하는 적응적 LTH알고리즘을 이용한 오선 제거이다. 실험결과 복잡한 영상의 경우에도 강건함과 높은 인식률을 보였다.

Keywords

References

  1. Cardoso JS, Rebelo,"A Robust staffline thickness and distance estimation in binary and gray-level music scores", Proceedings of The twentieth international conference on pattern recognition, pp. 1856-1859,2010. http://dx.doi.org/10.1109/icpr.2010.458
  2. Pinto T, Rebelo A, Giraldi G, Cardoso JS,"Music score binarization based on domain knowledge", Pattern recognition and image analysis. Lecture notes in computer science, 6669. Springer, Heidelberg, pp. 700-708, 2011. http://dx.doi.org/10.1007/978-3-642-21257-4_87
  3. Otsu, N."A threshold selection method from gray-level histograms", IEEE Transactions on Systems, Man and Cybernetics 9(1), pp. 62-66, 1979. http://dx.doi.org/10.1109/tsmc.1979.4310076
  4. J. dos Santos Cardoso, A. Capela, A. Rebelo, C. Guedes, and J. P. da Costa, "Staff detection with stable paths", IEEE Transactions on PAMI, 31, pp. 1134-1139, 2009. http://dx.doi.org/10.1109/tpami.2009.34
  5. Randriamahefa, R., Cocquerez, J. P., Fluhr, C., Ppin, F., and Philipp, S,"Printed music recognition", Proceedings of the 2nd International Conference on Document Analysis and Recognition (ICDAR'93), pp. 898-901, 1993. http://dx.doi.org/10.1109/icdar.1993.395592
  6. Fujinaga I,"Staff detection and removal", George S (ed) Visual perception of music notation: on-line and off-line recognition. Idea Group Inc., Hershey, pp 1-39, 2004. http://dx.doi.org/10.4018/978-1-59140-298-5.ch001
  7. Soo-Hyung Kim, Hwa-Jeong Son, Sung-Ryul Oh, Chil-Woo Lee, and Il-Seok Oh, "Staff-Line Detection and Removal Algorithm for a Camera-based Recognition of Music Score Images", Proc. 2nd Korea-Japan Joint Workshop on Pattern Recognition, pp. 131-138, 2007.
  8. A. Rebelo, A. Capela, J.F.P. da Costa, C. Guedes, E. Carrapatoso, and J.S. Cardoso, "A Shortest Path Approach for Staff Line Detection", Proc. Third Int'l Conf. Automated Production of Cross Media Content for Multichannel Distribution, pp. 79-85, 2007. http://dx.doi.org/10.1109/axmedis.2007.2
  9. Miyao H, Nakano Y.,"Note symbol extraction for printed piano scores using neural networks", IEICE Trans Inform Syst E79-D, pp.548-554, 1996.
  10. Bainbridge, D.,"An extensible optical music recognition system", Nineteenth Australasian Computer Science Conference, pp. 308-317, 1997.
  11. Bellini P, Bruno I and Nesi P.,"Optical music recognition: architecture and algorithms", Interactive multimedia music technologies. IGI Global, Hershey, pp 80-110, 2008. http://dx.doi.org/10.4018/978-1-59904-150-6.ch005
  12. Rossant F and Bloch I,"Robust and adaptive OMR system including fuzzy modeling, fusion of musical rules, and possible error detection", EURASIP J Appl Signal Process (1),2007. http://dx.doi.org/10.1155/2007/81541
  13. Prerau D.,"Computer pattern recognition of standard engraved music notation", 1970. "A critical survey of music image analysis", Blostein D, Baird H (eds) Structured document image analysis. Springer, Heidelberg, pp. 405-434,1992. http://dx.doi.org/10.1007/978-3-642-77281-8_19
  14. Roach JW and Tatem JE,"Using domain knowledge in low level visual processing to interpret handwritten music: an experiment", 1988. "A critical survey of music image analysis", In: Blostein D, Baird H (eds) Structured document image analysis. Springer, Heidelberg, pp 405-434, 1992. http://dx.doi.org/10.1016/0031-3203(88)90069-6
  15. Tardon LJ, Sammartino S, Barbancho I, Gomez V, Oliver A.,"Optical music recognition for scores written in white mensural notation", EURASIP J Image Video Process. 2009. http://dx.doi.org/10.1155/2009/843401
  16. C. Dalitz, M. Droettboom, B. Czerwinski, and I. Fujigana, "A Comparative Study of Staff Removal Algorithms", IEEE Trans. Pattern Analysis and Machine Intelligence,30(5), pp. 753-766,2008. http://dx.doi.org/10.1109/tpami.2007.70749
  17. P. Martin, C. Bellissant., "Low-Level Analysis of Music Drawing Images", First International Conference on Document Analysis and Recognition,France, pp. 417-425, 1991.
  18. D. Bainbridge, T.C. Bell.,"Dealing with superimposed Objects in Optical Music Recognition", Proceedings of the 6th International Conference on Image Processing and its Applications. pp. 756-760, 1997. http://dx.doi.org/10.1049/cp:19970997
  19. Tardon LJ, Sammartino S, Barbancho I, Gomez V, Oliver A.,"Optical music recognition for scores written in white mensural notation", EURASIP J Image Video Process, 2009. http://dx.doi.org/10.1155/2009/843401
  20. N.P. Carter, R.A. Bacon. "Automatic Recognition of Printed Music", Structured Document Image Analysis, Springer, pp. 454-65, 1992. http://dx.doi.org/10.1109/icpr.1996.547279
  21. Pietikainen, Matti, and Oleg Okun. "Edge-based method for text detection from complex document images", Document Analysis and Recognition, 2001. Proceedings. Sixth International Conference on. IEEE, 2001. http://dx.doi.org/10.1109/icdar.2001.953800
  22. A. Fornes, A. Dutta, A. Gordo, and J. Llados, "The ICDAR 2011 music scores competition: Staff removal and writer identification", ICDAR, pp. 1511-1515, 2011. http://dx.doi.org/10.1109/icdar.2011.300
  23. Carter NP, (1992) "Automatic recognition of printed music in the context of electronic publishing", "A critical survey of music image analysis", In: Blostein D, Baird H (eds) Structured document image analysis. Springer, Heidelberg, pp. 405-434, 1989. http://dx.doi.org/10.1007/978-3-642-77281-8_19
  24. http://en.wikipedia.org/wiki/MusicXML.