International Journal of Fuzzy Logic and Intelligent Systems

  • 제12권2호
  • /
  • Pages.101-107
  • /
  • 2012
  • /
  • 1598-2645(pISSN)
  • /
  • 2093-744X(eISSN)
한국지능시스템학회 (Korean Institute of Intelligent Systems)
권호 표지
DOI QR코드

DOI QR Code

A Basic Study on the Conversion of Sound into Color Image using both Pitch and Energy

  • Kim, Sung-Ill (Department of Electronic Engineering, Kyungnam University)
  • 투고 : 2012.02.16
  • 심사 : 2012.06.16
  • 발행 : 2012.06.25
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

This study describes a proposed method of converting an input sound signal into a color image by emulating human synesthetic skills which make it possible to associate an sound source with a specific color image. As a first step of sound-to-image conversion, features such as fundamental frequency(F0) and energy are extracted from an input sound source. Then, a musical scale and an octave can be calculated from F0 signals, so that scale, energy and octave can be converted into three elements of HSI model such hue, saturation and intensity, respectively. Finally, a color image with the BMP file format is created as an output of the process of the HSI-to-RGB conversion. We built a basic system on the basis of the proposed method using a standard C-programming. The simulation results revealed that output color images with the BMP file format created from input sound sources have diverse hues corresponding to the change of the F0 signals, where the hue elements have different intensities depending on octaves with the minimum frequency of 20Hz. Furthermore, output images also have various levels of chroma(or saturation) which is directly converted from the energy.

키워드

참고문헌

  1. O. Teng, "Synesthesia: Beyond the Five Senses," Executive intelligence review, vol. 38, no. 5, pp. 6-9, 2011.
  2. R. E. Cytowic, "Synesthesia: A Union of the Senses," The MIT Press, 2002.
  3. L. C. Robertson, N. Sagiv, "Synesthesia: Perspectives from Cognitive Neuroscience," Oxford University Press, 2004.
  4. G. H. Kim, J. G. Beak, Sound Color Harmonism(in Korean), Impress, 2003.
  5. G. H. Kim, Method and Apparatus for harmonizing Colors by Harmonics and converting Sound into Colors mutually(in Korean), Korean Intellectual Property, 10-99- 34242, 1999.
  6. G. H. Kim, The Sound-to-Color Conversion Table using a Low of Harmony(in Korean), Korean Intellectual Property, 10-2001-0087651, 2001.
  7. J. Ward, B. Huckstep, E. Tsakanikos, "Sound-Colour Synaesthesia: to What Extent Does it Use Cross-Modal Mechanisms Common to us All," Cortex; a journal devoted to the study of the nervous system and behavior, vol. 42, no. 2, pp. 264-280, 2006. https://doi.org/10.1016/S0010-9452(08)70352-6
  8. Thorisson, K. R., Donoghue, K., "Synthetic Synesthesia: Mixing Sound with Color," InterChi Adjunct Proceedings, pp. 65-66, 1993.
  9. Leonard N. Foner, "Artificial synesthesia via sonification: A wearable augmented sensory system," Mobile networks and applications : MONET, vol. 4, no. 1, pp. 75-81, 1999. https://doi.org/10.1023/A:1019178210975
  10. Peter B.L. Meijer, "An Experimental System for Auditory Image Representations," IEEE trans. on bio-medical engineering, vol. 39, no. 2, pp. 112-121, 1992. https://doi.org/10.1109/10.121642
  11. G. Loy, "Musimathics: The Mathematical Foundations of Music (Volume 1)," The MIT Press, 2006.
  12. G. Loy, J. Chowning, "Musimathics: The Mathematical Foundations of Music (Volume 2)," The MIT Press, 2007.
  13. Michael Freeman, "Mastering Color Digital Photography," Ilex Press, 2004.
  14. John R. Deller Jr., John H. L. Hansen, John G. Proakis, "Discrete-Time Processing of Speech Signals," Wiley- IEEE Press, 1999.
  15. Speech Signal Processing Toolkit (SPTK), http://sptk.sourceforge.net/

피인용 문헌

  1. Implementation of ARM based Embedded System for Muscular Sense into both Color and Sound Conversion vol.16, pp.8, 2016, https://doi.org/10.5392/JKCA.2016.16.08.427