• Journal of Korea Multimedia Society
• /
• v.19 no.3
• /
• pp.642-649
• /
• 2016

This paper presents a method for conversion of muscular sense into both visual and auditory senses based on synesthetic perception. Muscular sense can be defined by rotation angles, direction changes and motion degrees of human body. Synesthetic interconversion can be made by learning, so that it can be possible to create intentional synesthetic phenomena. In this paper, the muscular sense was converted into both color and sound signals which comprise the great majority of synesthetic phenomena. The measurement of muscular sense was performed by using the AHRS(attitude heading reference system). Roll, yaw and pitch signals of the AHRS were converted into three basic elements of color as well as sound, respectively. The proposed method was finally applied to a wearable device, Samsung gear S, successfully.

• Science of Emotion and Sensibility
• /
• v.16 no.2
• /
• pp.187-194
• /
• 2013

The final aim of the present study is to build a system of converting a color image into musical elements based on a synesthetic perception, emulating human synesthetic skills, which make it possible to associate a color image with a specific sound. This can be done on the basis of the similarities between physical frequency information of both light and sound. As a first step, an input true color image is converted into hue, saturation, and intensity domains based on a color model conversion theory. In the next step, musical elements including note, octave, loudness, and duration are extracted from each domain of the HSI color model. A fundamental frequency (F0) is then extracted from both hue and intensity histograms. The loudness and duration are extracted from both intensity and saturation histograms, respectively. In experiments, the proposed system on the conversion of a color image into musical elements was implemented using standard C and Microsoft Visual C++(ver. 6.0). Through the proposed system, the extracted musical elements were synthesized to finally generate a sound source in a WAV file format. The simulation results revealed that the musical elements, which were extracted from an input RGB color image, reflected in its output sound signals.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• v.12 no.2
• /
• pp.101-107
• /
• 2012

This study describes a proposed method of converting an input sound signal into a color image by emulating human synesthetic skills which make it possible to associate an sound source with a specific color image. As a first step of sound-to-image conversion, features such as fundamental frequency(F0) and energy are extracted from an input sound source. Then, a musical scale and an octave can be calculated from F0 signals, so that scale, energy and octave can be converted into three elements of HSI model such hue, saturation and intensity, respectively. Finally, a color image with the BMP file format is created as an output of the process of the HSI-to-RGB conversion. We built a basic system on the basis of the proposed method using a standard C-programming. The simulation results revealed that output color images with the BMP file format created from input sound sources have diverse hues corresponding to the change of the F0 signals, where the hue elements have different intensities depending on octaves with the minimum frequency of 20Hz. Furthermore, output images also have various levels of chroma(or saturation) which is directly converted from the energy.

• The Journal of the Korea Contents Association
• /
• v.15 no.12
• /
• pp.8-17
• /
• 2015

This paper presents a sound into color signal conversion using a colored-hearing synesthesia. The aim of the present paper is to implement a real-time conversion system which focuses on both hearing and sight which account for a great part of bodily senses. The proposed method of the real-time conversion of color into sound, in this paper, was simple and intuitive where scale, octave and velocity were extracted from MIDI input signals, which were converted into hue, intensity and saturation, respectively, as basic elements of HSI color model. In experiments, we implemented both the hardware system for delivering MIDI signals to PC and the VC++ based software system for monitoring both input and output signals, so we made certain that the conversion was correctly performed by the proposed method.

• The Journal of the Acoustical Society of Korea
• /
• v.30 no.3
• /
• pp.142-148
• /
• 2011

The final aim of the present study is to develop the intelligent robot, emulating human synesthetic skills which make it possible to associate a color image with a specific sound. This can be done on the basis of the mutual conversion between color image and sound. As a first step of the final goal, this study focused on a basic system using a conversion of color image into sound. This study describes a proposed method to convert color image into sound, based on the likelihood in the physical frequency information between light and sound. The method of converting color image into sound was implemented by using HSI histograms through RGB-to-HSI color model conversion, which was done by Microsoft Visual C++ (ver. 6.0). Two different color images were used on the simulation experiments, and the results revealed that the hue, saturation and intensity elements of each input color image were converted into fundamental frequency, harmonic and octave elements of a sound, respectively. Through the proposed system, the converted sound elements were then synthesized to automatically generate a sound source with wav file format, using Csound.

• Science of Emotion and Sensibility
• /
• v.15 no.2
• /
• pp.231-238
• /
• 2012

This study aims for building an application system of converting sound into color image based on synesthetic perception. As the major features of input sound, both scale and octave elements extracted from F0(fundamental frequency) were converted into both hue and intensity elements of HSI color model, respectively. In this paper, we used the fixed saturation value as 0.5. On the basis of color model conversion theory, the HSI color model was then converted into the RGB model, so that a color image of the BMP format was finally created. In experiments, the basic system was implemented on both software and hardware(TMS320C6713 DSP) platforms based on the proposed sound-color image conversion method. The results revealed that diverse color images with different hues and intensities were created depending on scales and octaves extracted from the F0 of input sound signals. The outputs on the hardware platform were also identical to those on the software platform.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.20 no.2
• /
• pp.251-256
• /
• 2010

This paper aims for developing the intelligent robot emulating human synesthetic skills which associate a color image with sound, so that we are able to build an application system based on the principle of mutual conversion between color image and sound. As the first step, in this study, we have tried to realize a basic system using the color image to sound conversion. This study describes a new conversion method to convert color image into sound, based on the likelihood in the physical frequency information between light and sound. In addition, we present the method of converting color image into sound using color model conversion as well as histograms in the converted color model. In the basis of the method proposed in this study, we built a basic system using Microsoft Visual C++(ver. 6.0). The simulation results revealed that the hue, saturation and intensity elements of a input color image were converted into F0, harmonic and octave elements of a sound, respectively. The converted sound elements were synthesized to generate a sound source with WAV file format using Csound toolkit.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.24 no.5
• /
• pp.462-469
• /
• 2014

As a basic study on both engineering applications and representation methods of synesthesia, this paper aims at building basic system which converts a muscular sense into both visual and auditory elements. As for the building method, data of the muscular sense can be acquired through roll and pitch signals which are calculated from both three-axis acceleration sensor and the two-axis gyro sensor. The roll and pitch signals are then converted into both visual and auditory information as outputs. The roll signals are converted into both intensity elements of the HSI color model and octaves as one of auditory elements. In addition, the pitch signals are converted into both hue elements of the HSI color model and scales as another one of auditory elements. Each of the extracted elements of the HSI color model is converted into each of the three elements of the RGB color model respectively, so that the real-time output color signals can be obtained. Octaves and scales are also converted and synthesized into MIDI signals, so that the real-time sound signals can be obtained as anther one of output signals. In experiments, the results revealed that normal color and sound output signals were successfully obtained from roll and pitch values that represent muscular senses or physical movements, depending on the conversion relationship based on the similarity between color and sound.