• Journal of Korea Multimedia Society
• /
• v.19 no.3
• /
• pp.642-649
• /
• 2016

This paper presents a method for conversion of muscular sense into both visual and auditory senses based on synesthetic perception. Muscular sense can be defined by rotation angles, direction changes and motion degrees of human body. Synesthetic interconversion can be made by learning, so that it can be possible to create intentional synesthetic phenomena. In this paper, the muscular sense was converted into both color and sound signals which comprise the great majority of synesthetic phenomena. The measurement of muscular sense was performed by using the AHRS(attitude heading reference system). Roll, yaw and pitch signals of the AHRS were converted into three basic elements of color as well as sound, respectively. The proposed method was finally applied to a wearable device, Samsung gear S, successfully.

• Science of Emotion and Sensibility
• /
• v.16 no.2
• /
• pp.187-194
• /
• 2013

The final aim of the present study is to build a system of converting a color image into musical elements based on a synesthetic perception, emulating human synesthetic skills, which make it possible to associate a color image with a specific sound. This can be done on the basis of the similarities between physical frequency information of both light and sound. As a first step, an input true color image is converted into hue, saturation, and intensity domains based on a color model conversion theory. In the next step, musical elements including note, octave, loudness, and duration are extracted from each domain of the HSI color model. A fundamental frequency (F0) is then extracted from both hue and intensity histograms. The loudness and duration are extracted from both intensity and saturation histograms, respectively. In experiments, the proposed system on the conversion of a color image into musical elements was implemented using standard C and Microsoft Visual C++(ver. 6.0). Through the proposed system, the extracted musical elements were synthesized to finally generate a sound source in a WAV file format. The simulation results revealed that the musical elements, which were extracted from an input RGB color image, reflected in its output sound signals.

• Journal of the Korean Society of Clothing and Textiles
• /
• v.43 no.4
• /
• pp.474-490
• /
• 2019

Fashion films with screen limits have a way of changing communication methods through sensorial organs in order to go beyond limits. This study shows that such change is possible if fashion films are based on haptic factors. This study examines haptic factors of fashion films from the three perspectives of filming factors of different shot size, synesthetic, and cinematic screen methods. First, when the subject to be emphasized is enlarging, the observer comes to project themselves to the situation and incurs a haptic sensation. Second, when associating an experience by personal recollection or social customs when more than two senses are stimulated simultaneously, haptic sensations, triggered by multiple senses, takes place. Third, a blurred image shows haptic sensations through inducing observers to see into the meaning of a shot. As a result, the senses of the observer enlarge and enhance a communication ability through absorbing and accepting a fashion film. Furthermore, fashion films are effective in understanding the cultural forms of the age.

• Proceedings of the Korea Contents Association Conference
• /
• 2013.05a
• /
• pp.313-314
• /
• 2013

브랜딩의 목적은 브랜드 아이덴티티 구축의 개념에서 브랜드체험의 전달로 변화하며 오감과 감성의 활용에 주목하는 사회 환경에서 디자인도 감각디자인, 경험디자인 등으로 개념화되고 있다. 따라서 공감각적 표현의 전략적인 활용이 브랜드인지를 촉진한다는 가정을 하였다. 국내외 브랜드의 디자인에 나타나는 공감각적 표현을 분석해본 결과 시각과 청각, 촉각, 후각을 이용한 공감각적 표현이 가장 많이 나타났다.

• Korean Institute of Interior Design Journal
• /
• v.22 no.5
• /
• pp.11-21
• /
• 2013

The development of modern industry and the production of new materials of architecture have recognized as an important aspect. This study intended to discuss about the meaning of main elements of the architecture materials, its implications for properties, and, highlighted meaning of the properties of modern interior design. The purpose of this study is to analyze the case studies of Place and Physical materials of synaesthesia of modern architecture as well as modern architecture is not only concentrated in the visual aspect but also the larger ramifications of sensory. Synesthesia in architectural space can be called experiential existence that built in the space of human perception and based on the human emotion that targeted by physical architectural space. Accordingly, these case studies are to consider about how the materials being passed by any effect in a modern interior space presented, In order to find the essential meaning in modern interior space, this studies focused on Swiss architect Peter Zumthor's cases. Following analyze of Peter Zumthor's place of architectural space and materials, this study structured by the six kinds of synesthesia characters. First, time and remember to infer changes, Second, traditional and regional ingredients involve the substitution of one empirical space, Third, non-materiality through the neutralization, Fourth, auditory memory through the embodiment of everydayness, Fifth, trail through the sense of smell for the presence, Sixth, the relationship of natural and industrial products. Peter Zumthor's characteristics of these works are result of factors on the synesthetic relationship with the human response to architecture, the five senses, memories, experiences, and temperature. Also it is a synesthetic result of all human emotion to clarify of the architecture materials and non-material mediums by which light by a compound.

• Journal of Popular Narrative
• /
• v.27 no.1
• /
• pp.265-299
• /
• 2021

The purpose of this research paper is to facilitate the understanding of the synesthetic aesthetics in the film The Age of Innocence through the intertextuality among the narrative, paintings, and music in the film. In this paper, a two-dimensional intertextual analysis of the paintings in relation to the narrative is conducted on the paintings owned by Old New York, the paintings owned by Ellen, the portraits of unknown artists on the street outside of Parker House, and Rubens' painting at the Louvre. A three-dimensional intertextual analysis of performances in relation to the narrative is conducted on the stages and the box seats at the New York Academy of Music, in which Charles F. Gounod's Faust is performed, and the Wallack's Theatre, in which Dion Boucicault's The Shaughraun is performed. An intertextual analysis of music in relation to the narrative is also conducted on the diegetic and non-diegetic classical music of the film, including Beethoven's Piano Sonata No. 8 and Mendelssohn's String Quintet No. 2, as well as Elmer Bernstein's non-diegetic music of the film. The constituent event of The Age of Innocence represents the passion trapped in the reflection of love and desire that are not lasting, and the supplementary event embodies the narrow viewpoint and the inversion of values caused by the patriarchal authority of Old New York. The characters in the film live a double life, presenting an unaffected surface and concealing the problems behind it. The characters restrain their emotions at both the climax and the ending. The most powerful aspect of the film is the type and nature of oppressive life, which are more delicately described with the help of paintings and music, as there is a limit to describing them only by acting. In intertextual terms, paintings and music in The Age of Innocence continuously emphasize "feeling of emotions that cannot be expressed in language." With a synesthetic image, as if each part were imprinted on the previous part, the continuity "responds to continuous camera movements and montage effects." In The Age of Innocence, erotic dynamism brings dramatic excitement to the highest level, switching between the satisfaction of revealing desire and the disappointment of hiding desire due to its taboo status. This is possible because paintings and music related to the narrative have made aesthetic achievements that overcome the limitations of two-dimensional planes and limited frames. The significance of this study lies in that, since the identification in The Age of Innocence is based on the establishment of a synesthetic aesthetic through audio-visual representation of the film narrative, it helps us to rediscover the possibility of cinematic aesthetics.

• Journal of Korea Multimedia Society
• /
• v.23 no.4
• /
• pp.603-614
• /
• 2020

Analyzing various works of Nam June Paik who pioneered the world of video art, we can find a number of audiovisual art pieces - in a broad sense - with the characteristics of "intermedia" that are not clearly identified as one genre. Especially, his career as a musician becomes an important clue for us to discuss the unique originality found in Paik's works. Nevertheless, examples of research that specifically explore how Paik's musicality is reflected in his artistry and could be deeply understood is still rare. This article examines Paik's audiovisual art world, and reviews how his musicality is expressed through the characteristics of media and form, as well as contributed to the ultimate technical and aesthetic realizations of his pieces from the perspective of musical synesthesia. The fact that Paik expanded the concept of music through technology, and fused different media to visualize music, is an important achievement in the context of aesthetic, synesthetic music in the field of audiovisual art.

• Journal of the Korea Fashion and Costume Design Association
• /
• v.8 no.1
• /
• pp.111-125
• /
• 2006

This study measured and analyzed musical sense objectively, and visualized it to express visual effects as if watching music with the eye. For the visualization of music, we examined the concept of synesthesia and the correlation between hearing and seeing, and reviewed theoretical grounds and a number of cases necessary in reproducing musical sense visually based on synesthetic expression. Furthermore, we studied the visual elements and associated colors of rhythm to prove that visual conception of music through various approaches enable a transitional approach to aesthetic exploration and interpretation. The conclusions drawn from this research are as follows. First, classic music selected for visualization had musical characteristics highly correlated with dress design. Second, basic formative elements suggested as materials of visualization in this study were suitable in their form for expressing the rhythmic sense of music and, because they started from the most basic form, they were effective in extracting design elements. Third, when the result of the questionnaire survey, which was focused on the visualization of the rhythmic sense of the five pieces of classic music, was analyzed from the aspect of sensibility ergonomics, design elements of each piece of music were obtained in an objective and scientific way. Fourth, it was confirmed that common concepts could be derived from intangible elements such as forms and sounds observed in the rhythmic sense of music obtained from the result of the questionnaire survey. Fifth, works were made based on the results of this study and, according to the result, musical images can express sense through dress design and obtain visual effects as if watching music with the eye. Dress desist through the visualization of music in this study was an attempt to suggest that the language of music can be expressed in dress design, a visual formative language, based on synesthetic expression. Through this attempt, we confirmed the infinity of music as motives of dress design and suggested a method of aesthetic expression demanded in contemporary society that is pursuing aesthetic values.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.24 no.5
• /
• pp.462-469
• /
• 2014

As a basic study on both engineering applications and representation methods of synesthesia, this paper aims at building basic system which converts a muscular sense into both visual and auditory elements. As for the building method, data of the muscular sense can be acquired through roll and pitch signals which are calculated from both three-axis acceleration sensor and the two-axis gyro sensor. The roll and pitch signals are then converted into both visual and auditory information as outputs. The roll signals are converted into both intensity elements of the HSI color model and octaves as one of auditory elements. In addition, the pitch signals are converted into both hue elements of the HSI color model and scales as another one of auditory elements. Each of the extracted elements of the HSI color model is converted into each of the three elements of the RGB color model respectively, so that the real-time output color signals can be obtained. Octaves and scales are also converted and synthesized into MIDI signals, so that the real-time sound signals can be obtained as anther one of output signals. In experiments, the results revealed that normal color and sound output signals were successfully obtained from roll and pitch values that represent muscular senses or physical movements, depending on the conversion relationship based on the similarity between color and sound.

• (The) Research of the performance art and culture
• /
• no.40
• /
• pp.5-35
• /
• 2020

Affect is intensity and quality that are generated as the physical body senses the outside world. Of experienced affect, notions that are granted meaning and interpretation are emotions. Affect theory distinguishes emotion and affect and by focusing on affect, it provides methods with which to analyze physical body responses and changes and it presents new possibilities to performing arts research that uses the physical body as a medium. Nong-ak is art that concentrates mainly on the occurrence of affect rather than 'representation'. Nong-ak is a performance type in which sound, color, texture, and physical movement overlap and exist in a synesthetic way. Here, physical things such as instruments, props, costumes, and stage devices are gathered together with non-physical things such as rhythm, mood, and atmosphere around human bodies. The physical body is stimulated by these things, displays tendencies that suit performances, and becomes 'the body without an image' as it immerses itself into the performance, acting while displaying 'quasi-corporeality'. The body, which moves automatically as if without consciousness, appears more easily within groups. To transition individuals of everyday life to 'the body without an image', Nong-ak executes the group physical exercise of 'Jinpuri'. Such physical exercise builds up affect by increasing nonverbal communion and communication and brings out the creativity of individuals within mutual trust and a sense of belonging. Affect and emotion stirred up by Nong-ak act as confirmation and affirmation of the existence, vitality, and ability of one's self and groups. Such affirmation recalls Nong-ak as a meaningful and important value from group dimensions and perceives it as a performance form that should be preserved and passed on.