Browse > Article
http://dx.doi.org/10.7471/ikeee.2020.24.1.254

Implementation of the System Converting Image into Music Signals based on Intentional Synesthesia  

Bae, Myung-Jin (Dept. of Convergence IT Engineering, Kyungnam University)
Kim, Sung-Ill (Dept. of Electronic Engineering, Kyungnam University)
Publication Information
Journal of IKEEE / v.24, no.1, 2020 , pp. 254-259 More about this Journal
Abstract
This paper is the implementation of the conversion system from image to music based on intentional synesthesia. The input image based on color, texture, and shape was converted into melodies, harmonies and rhythms of music, respectively. Depending on the histogram of colors, the melody can be selected and obtained probabilistically to form the melody. The texture in the image expressed harmony and minor key with 7 characteristics of GLCM, a statistical texture feature extraction method. Finally, the shape of the image was extracted from the edge image, and using Hough Transform, a frequency component analysis, the line components were detected to produce music by selecting the rhythm according to the distribution of angles.
Keywords
Visualization; Sonification; Sound; Music; Color; Texture; Shape; Melody; Harmony; Rhythm;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 J. Ward, "Synesthesia," Annual Review of Psychology, Vol.64, No.1, pp.49-75, 2013. DOI: 10.1146/annurev-psych-113011-143840   DOI
2 Suslick, S Kenneth. "Synesthesia in science and technology: more than making the unseen visible," Current opinion in chemical biology, Vol.16, No.5, pp.557-563, 2012. DOI: 10.1016/j.cbpa.2012.10.030   DOI
3 Smilek, D., Dixon, M. J. & Merikle, P. M. "Synaesthesia: discordant male monozygotic twins," Neurocase, Vol.11, No.5, pp.363-370, 2005. DOI: 10.1080/13554790500205413   DOI
4 Smilek, D. et al. "Synaesthesia: a case study of discordant monozygotic twins," Neurocase, Vol.8, No.4, pp.338-342, 2002. DOI: 10.1076/neur.8.3.338.16194   DOI
5 S. Kim, "Conversion of Image into Sound Based on HSI Histogram," The Journal of the Acoustical Society of Korea, Vol.30, No.3, pp.142-148, 2011. DOI: 10.7776/ASK.2011.30.3.142   DOI
6 M. Bae, S. Kim, "Implementation of the Visualization and Sonification System of Human Movements based on Intentional Synesthesia," Journal of Korean institute of intelligent systems, Vol.28, No.1, pp.83-90, 2018. DOI: 10.7471/ikeee.2018.22.2.362   DOI
7 D. Bor, N. Rothen, D. Schwartzman, S. Clayton, A. Seth, "Adults Can Be Trained to Acquire Synesthetic Experiences," Scientific Reports 4, No.7089, 2014. DOI: 10.1038/srep07089
8 Witthoft, Nathan, and Jonathan Winawer. "Learning, memory, and synesthesia," Psychological science, Vol.24, No.3, pp.258-265, 2013. DOI: 10.1177/0956797612452573   DOI
9 M. Bae, S. Kim, "The System of Converting Muscular Sense Into both Color and Sound Based on the Synesthetic Perception," Journal of Korean institute of intelligent systems, Vol.24 No.5, pp.462-469, 2014. DOI: 10.5391/JKIIS.2014.24.5.462   DOI