• The Journal of the Korea Contents Association
• /
• v.16 no.8
• /
• pp.427-434
• /
• 2016

This paper focuses on a real-time hardware processing by implementing the ARM Cortex-M4 based embedded system, using a conversion algorithm from a muscular sense to both visual and auditory elements, which recognizes rotations of a human body, directional changes and motion amounts out of human senses. As an input method of muscular sense, AHRS(Attitude Heading Reference System) was used to acquire roll, pitch and yaw values in real time. These three input values were converted into three elements of HSI color model such as intensity, hue and saturation, respectively. Final color signals were acquired by converting HSI into RGB color model. In addition, Three input values of muscular sense were converted into three elements of sound such as octave, scale and velocity, which were synthesized to give an output sound using MIDI(Musical Instrument Digital Interface). The analysis results of both output color and sound signals revealed that input signals of muscular sense were correctly converted into both color and sound in real time by the proposed conversion method.

• The Journal of the Korea Contents Association
• /
• v.15 no.12
• /
• pp.8-17
• /
• 2015

This paper presents a sound into color signal conversion using a colored-hearing synesthesia. The aim of the present paper is to implement a real-time conversion system which focuses on both hearing and sight which account for a great part of bodily senses. The proposed method of the real-time conversion of color into sound, in this paper, was simple and intuitive where scale, octave and velocity were extracted from MIDI input signals, which were converted into hue, intensity and saturation, respectively, as basic elements of HSI color model. In experiments, we implemented both the hardware system for delivering MIDI signals to PC and the VC++ based software system for monitoring both input and output signals, so we made certain that the conversion was correctly performed by the proposed method.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.20 no.2
• /
• pp.251-256
• /
• 2010

This paper aims for developing the intelligent robot emulating human synesthetic skills which associate a color image with sound, so that we are able to build an application system based on the principle of mutual conversion between color image and sound. As the first step, in this study, we have tried to realize a basic system using the color image to sound conversion. This study describes a new conversion method to convert color image into sound, based on the likelihood in the physical frequency information between light and sound. In addition, we present the method of converting color image into sound using color model conversion as well as histograms in the converted color model. In the basis of the method proposed in this study, we built a basic system using Microsoft Visual C++(ver. 6.0). The simulation results revealed that the hue, saturation and intensity elements of a input color image were converted into F0, harmonic and octave elements of a sound, respectively. The converted sound elements were synthesized to generate a sound source with WAV file format using Csound toolkit.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.24 no.5
• /
• pp.462-469
• /
• 2014

As a basic study on both engineering applications and representation methods of synesthesia, this paper aims at building basic system which converts a muscular sense into both visual and auditory elements. As for the building method, data of the muscular sense can be acquired through roll and pitch signals which are calculated from both three-axis acceleration sensor and the two-axis gyro sensor. The roll and pitch signals are then converted into both visual and auditory information as outputs. The roll signals are converted into both intensity elements of the HSI color model and octaves as one of auditory elements. In addition, the pitch signals are converted into both hue elements of the HSI color model and scales as another one of auditory elements. Each of the extracted elements of the HSI color model is converted into each of the three elements of the RGB color model respectively, so that the real-time output color signals can be obtained. Octaves and scales are also converted and synthesized into MIDI signals, so that the real-time sound signals can be obtained as anther one of output signals. In experiments, the results revealed that normal color and sound output signals were successfully obtained from roll and pitch values that represent muscular senses or physical movements, depending on the conversion relationship based on the similarity between color and sound.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2005.05a
• /
• pp.3-6
• /
• 2005

본 논문에서는 Deway order기법을 이용하여 관계형 데이터베이스 환경에서 효율적으로 XML 문서를 저장, 검색, 결과값을 반환하기 위한 XQuery 질의 처리기를 설계하고 구현한다. 우선 첫 번째로 dewey order기법을 이용하여 XML문서를 저장하기 위한 색인 모델을 관계형 데이터 베이스에 설계하고 XML문서를 저장한다. 두 번째로 이를 기반으로 XML 전문검색 언어인 XPath식을 SQL로 변환하는 전체적인 알고리즘을 제시한다. 세 번째로 위에서 변환된 SQL문의 질의 결과값을 처 음에 저장될 당시의 XML문서의 형태와 Text를 그대로 유지하면서 사용자에게 반환하는 알고리즘을 제시한다. 이 부분은 기존의 발표된 논문에서는 좀처럼 보기 드문 내용으로 XQuery에 포함되는 다양 한 형태의 Xpath식을 SQL문으로 변환할 수 있는 정확한 방법 뿐만 아니라 각각 한번의 질의로 얻고 하는 엘리먼트들과 어트리뷰트들을 찾아 XML문서 그대로 출력하는 방법에 초점을 두어 본 논문을 기 술한다. 마지막으로 이를 실제로 구현하고 Test한 결과를 바탕으로 Dewey order기법을 이용하여 XML 색인 모델을 설계 할 경우 SQL문으로 변환하여 질의를 처리하는 측면에서나 질의한 결과값을 XML문서 형태로 반환하는 측면에서나 이 기법을 사용하지 않는 Local order방식보다 성능이 훨씬 우수하다라는 결론을 제시한다.

• Journal of Korea Multimedia Society
• /
• v.19 no.3
• /
• pp.642-649
• /
• 2016

This paper presents a method for conversion of muscular sense into both visual and auditory senses based on synesthetic perception. Muscular sense can be defined by rotation angles, direction changes and motion degrees of human body. Synesthetic interconversion can be made by learning, so that it can be possible to create intentional synesthetic phenomena. In this paper, the muscular sense was converted into both color and sound signals which comprise the great majority of synesthetic phenomena. The measurement of muscular sense was performed by using the AHRS(attitude heading reference system). Roll, yaw and pitch signals of the AHRS were converted into three basic elements of color as well as sound, respectively. The proposed method was finally applied to a wearable device, Samsung gear S, successfully.

• Journal of the Korean Society for information Management
• /
• v.30 no.3
• /
• pp.23-48
• /
• 2013

This study compared and analyzed 15 CBMR (Content-based Music Retrieval) systems accessible on the web in terms of DB size and type, query type, access point, input and output type, and search functions, with reviewing features of music information and techniques used for transforming or transcribing of music sources, extracting and segmenting melodies, extracting and indexing features of music, and matching algorithms for CBMR systems. Application of text information retrieval techniques such as inverted indexing, N-gram indexing, Boolean search, truncation, keyword and phrase search, normalization, filtering, browsing, exact matching, similarity measure using edit distance, sorting, etc. to enhancing the CBMR; effort for increasing DB size and usability; and problems in extracting melodies, deleting stop notes in queries, and using solfege as pitch information were found as the results of analysis.

• 한국HCI학회:학술대회논문집
• /
• 2008.02b
• /
• pp.268-273
• /
• 2008

This work defines user intention according to intention range, and also proposes an interactive sound installation which reflects and varies above features. User intention consists of several decomposition concepts, which are elemental intentions, partial intentions, and a universal intention. And also, each concept is defined as inclusion/affiliation relationship with other concepts. For the representation of elemental intention, we implemented an musical interface, Color note, which represents the colors and notes according to response of participants. We also propose Harmonic Defragmentation (HD), which arranges the partial intentions with harmonic rule. Finally, the universal intention is inferred to the comprehensive direction of elemental intentions. We used Karhunen-Lo$\`{e}$ve(K-L) Transform for the inference. For verifying the validity of our proposed interface, the "Color Note," and the various techniques, we installed our work and surveyed various users for the evaluation of HD and statistical techniques. Also, we commissioned another survey to find out satisfaction measurement which was used for expressing universal intention.