Smart Media Journal (스마트미디어저널)

THE KOREAN INSTITUTE OF SMART MEDIA (한국스마트미디어학회)
권호 표지
DOI QR코드

DOI QR Code

A Covariance-matching-based Model for Musical Symbol Recognition

  • Received : 2017.09.04
  • Accepted : 2017.11.30
  • Published : 2018.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

A musical sheet is read by optical music recognition (OMR) systems that automatically recognize and reconstruct the read data to convert them into a machine-readable format such as XML so that the music can be played. This process, however, is very challenging due to the large variety of musical styles, symbol notation, and other distortions. In this paper, we present a model for the recognition of musical symbols through the use of a mobile application, whereby a camera is used to capture the input image; therefore, additional difficulties arise due to variations of the illumination and distortions. For our proposed model, we first generate a line adjacency graph (LAG) to remove the staff lines and to perform primitive detection. After symbol segmentation using the primitive information, we use a covariance-matching method to estimate the similarity between every symbol and pre-defined templates. This method generates the three hypotheses with the highest scores for likelihood measurement. We also add a global consistency (time measurements) to verify the three hypotheses in accordance with the structure of the musical sheets; one of the three hypotheses is chosen through a final decision. The results of the experiment show that our proposed method leads to promising results.

Keywords

References

  1. A. Rebelo, G. Capela, and J. S. Cardoso, "Optical recognition of music symbols: A comparative study," Int. J. Doc. Anal. Recognit., vol. 13, no. 1, p. 19-31, Mar. 2010. https://doi.org/10.1007/s10032-009-0100-1
  2. D. Pruslin, "Automatic recognition of sheet music," in Structured Document Image Analysis, H. Baird, H. Bunke, and K. Yamamoto, Eds. Springer Berlin Heidelberg, 1966.
  3. N. P. Carter and R. A. Bacon, "Automatic recognition of printed music," in Structured Document Image Analysis, H. Baird, H. Bunke, and K. Yamamoto, Eds. Springer Berlin Heidelberg, p. 456-465, 1992.
  4. R. Randriamahefa, J.-P. Cocquerez, C. Fluhr, F. Pepin, and S. Philipp, "Printed music recognition," in Document Analysis and Recognition, Proceedings of the Second International Conference on, p. 898-901, 1993.
  5. F.Rossant and I.Bloch, "A fuzzy model for optical recognition of musical scores", in Fuzzy Sets and Systems, vol. 141, p. 165-201, 2003.
  6. F.Rossant and I.Bloch, "Robust and Adaptive OMR System Including Fuzzy Modeling, Fusion of Musical Rules, and Possible Error Detection", EURASIP Journal on Advances in Signal Processing 2007.
  7. K.T.Reed and J.R.Parker, "Automatic computer recognition of printed music". Proc. 13th Int. Conf. Pattern Recognit., p. 803-807 (1996).
  8. P. Bellini, B. Ivan, and N. Paolo, "Optical music recognition: Architecture and algorithms," in Interactive Multimedia Music Technologies, N. Kia and N. Paolo, Eds. IGI Global, 2008, ch. 5, p. 80-110.
  9. H.N.Bui, "Camera-based Printed Music Score Recognition Using Deformable Line Adjacency Graph", M.D thesis, Chonnam National University, Gwangju, South Korea.
  10. H.N.Bui, "Camera-based Printed Music Score Recognition Using Deformable Line Adjacency Graph", M.D thesis , Chonnam National University, Gwangju, South Korea.
  11. O.Tuzel, F.Porikli and P.Meer, "Pedestrian Detection via Classification on Riemannian Manifolds", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.30, No.10, 2008.
  12. H.N Bui, I.S. Na, G.S Lee, H.J. Yang, S.H. Kim, "Boosted Stable Path for Staff-line Detection Using Order Statistic Downscaling and Coarse-to-Fine Technique,"Proc. 22nd International Conference on Pattern Recognition, pp. 522-526, Stockholm, Sweden, Aug. 2014.
  13. J.S.Cardoso, A.Capela, A.Rebelo, C.Guedes, "A connected path approach for staff detection on a music score". In: Proceedings of the International Conference on Image Processing (ICIP 2008), p. 1005-1008 (2008).
  14. J.S.Cardoso, A.Capela, A.Rebelo, C.Guedes, J.P. da Costa, "Staff detection with stable paths". IEEE Trans. Pattern Anal. Mach. Intell. 31(6), p. 1134-1139 (2009). https://doi.org/10.1109/TPAMI.2009.34
  15. R. Kimmel, N. Kiryati, and A. M. Bruckstein, "Distance maps and weighted distance transform", Journal of Mathematical Imaging and Vision, Special Issue on Topology and Geometry in Computer Vision, 6:223-233,1996.
  16. R. Andrzej & L. Yongmin, "Learning pairwise image similarities for multi-classification using Kernel Regression Trees." Pattern Recognition, 45(4), 1396-1408, (2012). https://doi.org/10.1016/j.patcog.2011.09.028
  17. L. Chengjun, "Discriminant analysis and similarity measure." Pattern Recognition, 47(1), 359-367, (2014). https://doi.org/10.1016/j.patcog.2013.06.023
  18. K. Takumi, "Kernel-based transition probability toward similarity measure for semi-supervised learning." Pattern Recognition, 47(5), 1994-2010, (2014). https://doi.org/10.1016/j.patcog.2013.11.011
  19. W. Forstner and B. Moonen, "A metric for Covariance Matrices", Geodesy-The Challenge of the 3rd Millennium
  20. http://www.musitek.com/smartscore-piano.html
  21. B. Gatos, I. Pratikakis, S.J. Perantonis, "Adaptive degraded document image binarization.", Pattern Recognition, 39(3), 317-327, (2006). https://doi.org/10.1016/j.patcog.2005.09.010
  22. H.N Bui, I.S. Na, S.H. Kim, "Staff Line Removal Using Line Adjacency Graph and Staff Line Skeleton for Camera-Based Printed Music Scores.", 22nd International Conference on Pattern Recognition (ICPR), p. 2787-2789, 2014.
  23. L. Pugin, "Optical music recognition of early typographic prints using Hidden Markov models." In: Proceedings of the International Society for Music, information retrieval, p. 53-56, 2006.
  24. W. Homenda and M. Luckner, "Automatic knowledge acquisition: recognizing music notation with methods of centroids and classifications trees." In: Proceedings of the international joint conference on neural networks, p. 3382-3388, 2006.
  25. http://gamera.informatik.hsnr.de/addons/musicstaves/index.html.
  26. 도루녹, "Musical Symbols Recognition with Region Proposal Networks and Fast Regional Convolution Neural Network : R-CNNs 과 RPNs 통한 악보인식." 전남대학교: 전자컴퓨터공학과 박사학위 논문 2017. 2