• International Journal of Computer Science & Network Security
• /
• v.23 no.8
• /
• pp.40-48
• /
• 2023

Speech Emotions recognition has become the active research theme in speech processing and in applications based on human-machine interaction. In this work, our system is a two-stage approach, namely feature extraction and classification engine. Firstly, two sets of feature are investigated which are: the first one is extracting only 13 Mel-frequency Cepstral Coefficient (MFCC) from emotional speech samples and the second one is applying features fusions between the three features: Zero Crossing Rate (ZCR), Teager Energy Operator (TEO), and Harmonic to Noise Rate (HNR) and MFCC features. Secondly, we use two types of classification techniques which are: the Support Vector Machines (SVM) and the k-Nearest Neighbor (k-NN) to show the performance between them. Besides that, we investigate the importance of the recent advances in machine learning including the deep kernel learning. A large set of experiments are conducted on Surrey Audio-Visual Expressed Emotion (SAVEE) dataset for seven emotions. The results of our experiments showed given good accuracy compared with the previous studies.

• Journal of KIISE:Computing Practices and Letters
• /
• v.9 no.4
• /
• pp.420-430
• /
• 2003

This paper presents an immersive multimodal Gumdo simulation game that allows a user to experience the whole body interaction with an intelligent cyber fencer. The proposed system consists of three modules: (i) a nondistracting multimodal interface with 3D vision and speech (ii) an intelligent cyber fencer and (iii) an immersive feedback by a big screen and sound. First, the multimodal Interface with 3D vision and speech allows a user to move around and to shout without distracting the user. Second, an intelligent cyber fencer provides the user with intelligent interactions by perception and reaction modules that are created by the analysis of real Gumdo game. Finally, an immersive audio-visual feedback by a big screen and sound effects helps a user experience an immersive interaction. The proposed system thus provides the user with an immersive Gumdo experience with the whole body movement. The suggested system can be applied to various applications such as education, exercise, art performance, etc.

• 제어로봇시스템학회:학술대회논문집
• /
• 2003.10a
• /
• pp.2676-2681
• /
• 2003

The Haptic Exploration Laboratory at The Johns Hopkins University is currently exploring many problems related to haptics (force and tactile information) in human-machine systems. We divide our work into two main areas: virtual environments and robot-assisted manipulation systems. Our interest in virtual environments focuses on reality-based modeling, in which measurements of the static and dynamic properties of actual objects are taken in order to produce realistic virtual environments. Thus, we must develop methods for acquiring data from real objects and populating pre-defined models. We also seek to create systems that can provide active manipulation assistance to the operator through haptic, visual, and audio cues. These systems may be teleoperated systems, which allow human users to operate in environments that would normally be inaccessible due to hazards, distance, or scale. Alternatively, cooperative manipulation systems allow a user and a robot to share a tool, allowing the user to guide or override the robot directly if necessary. Haptics in human-machine systems can have many applications, such as undersea and space operations, training for pilots and surgeons, and manufacturing. We focus much of our work on medical applications.

• Journal of The Institute of Information and Telecommunication Facilities Engineering
• /
• v.10 no.4
• /
• pp.143-147
• /
• 2011

The system using augmented reality can save the time and cost. It is verified in various fields under the possibility of a technology by solving unrealistic feeling in the virtual space. Therefore, augmented reality has a variety of the potential to be used. Generally, multimodal senses such as visual/auditory/tactile feed back are well known as a method for enhancing the immersion in case of interaction with virtual object. By adapting tangible object we can provide touch sensation to users. a 3D model of the same scale overlays the whole area of the tangible object; thus, the marker area is invisible. This contributes to enhancing immersive and natural images to users. Finally, multimodal feedback also creates better immersion. In this paper, sound feedback is considered. By further improving immersion learning augmented reality for children with the initial step learning content is presented. Augmented reality is in the intermediate stages between future world and real world as well as its adaptability is estimated more than virtual reality.

• Journal of Korea Multimedia Society
• /
• v.17 no.1
• /
• pp.104-112
• /
• 2014

With recent development of virtual reality technology, coupled with the growth of the marine industry, virtual underwater simulation systems are under development in various studies, for educational purposes and to simulate virtual reality experiences. Current literature indicates many underwater simulation systems to date have focused on the quality of visual stimulus delivered through three-dimensional graphics user interface, limiting the reality of the experience. In order to improve the quality of the reality delivered by such virtual simulations, it is crucial to develop multi-sensory technology rather than focus on the conventional audio-visual interaction, which limits experiencer from the sense of underwater immersion and existence within the simulation. This work proposes the immersive multi-sensory effector system, delivering the users with a more realistic underwater experience. The sense of reality perceived was evaluated, as the main factor of the virtual reality system.

• Cartoon and Animation Studies
• /
• s.35
• /
• pp.329-346
• /
• 2014

Art is the product from the combination of politics, economy, and social and cultural aspects. Recent development of digital media has affected on the expansion of visual expression in art. Digital media allow artists to use sound and physical interaction as well as image as an plastic element for making a work of art. Also, digital media help artists create an interactive, synaesthetic and visual perceptive environment by combining viewers' physical interaction with the reconstruction of image, sound, light, and among other plastic elements. This research was focused on the analysis of the relationship between images in art work and the viewer and data visualization using sound from the perspective of visual perception. This research also aimed to develop an interactive art by visualizing physical data with sound generating from outer stimulus or the viewer. Physical data generating from outer sound can be analyzed in various aspects. For example, Sound data can be analyzed and sampled within pitch, volume, frequency, and etc. This researcher implemented a new form of media art through the visual experiment of LED light triggered by sound frequency generating from viewers' voice or outer physical stimulus. Also, this researcher explored the possibility of various visual image expression generating from the viewer's reaction to illusionary characteristics of light(LED), which can be transformed within external physical data in real time. As the result, this researcher used a motif from Piet Mondrian's Broadway Boogie Woogie in order to implement a visual perceptive interactive work reacting with sound. Mondrian tried to approach at the essence of visual object by eliminating unnecessary representation elements and simplifying them in painting and making them into abstraction consisting of color, vertical and horizontal lines. This researcher utilized Modrian's simplified visual composition as a representation metaphor in oder to transform external sound stimulus into the element of light(LED), and implemented an environment inducing viewers' participation, which is a dynamic composition maximizing a synaesthetic expression, differing from Modrian's static composition.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.24 no.7B
• /
• pp.1211-1219
• /
• 1999

High quality real-time interactive multimedia services like VOD gained huge momentum by information technology industries during the middle of 1990, but failed successful commercial service deployment despite of the expectations. Among the number of reasons, lack of high bandwidth and QoS support from the network infrastructure have become main cause of the failure. On the other hand, the advance of computer industry-driven storage technologies has been reducing the price of in-home storage dramatically. This fact and the delivery of audio-visual material in a digital form to the home storage make very interesting multimedia service applications possible. It enables anytime content delivery (e.g., during off-peak time) without user interaction and anytime content play and, also, enhances utilization by reducing the loads applied to servers and networks using multicast network infrastructure. In this paper, we propose design and implementation of our anytime multimedia service system based on in-home storage, called IMPRESS-AMS.

• The Journal of the Korea Contents Association
• /
• v.18 no.10
• /
• pp.11-19
• /
• 2018

This paper is about multimodal storytelling experience applying Virtual Reality technology in museum. Specifically, this research argues virtual reality in both intuitive understanding of history also multimodal experience in the space. This research investigates cases regarding use of virtual reality in museum sector. As a research method, this paper conducts a literature review regarding multimodal experience and examples applying virtual reality related technologies in museum. Based on the literature review to investigate the concept necessary with its related cases. Based on the investigation, this paper suggests constructing elements for multimodal storytelling based on VR. Ultimately, this paper suggests the elements of building VR storytelling where dynamic audio-visual and interaction mode combines with historical resources for diverse audiences.

• Fashion & Textile Research Journal
• /
• v.25 no.5
• /
• pp.578-592
• /
• 2023

This study identified and evaluated by deriving and categorizing concepts related to the user experience of metaverse fashion shows using grounded theory, which is a qualitative research method. Based on experiential marketing theory, in-depth interviews were conducted for 14 days with 14 males and females in their 20s and 30s. The research results and contents are as follows: The causal condition was the purpose of using metaverse fashion shows, and the action/interaction strategy caused by such a case was found to be establishing a system for metaverse fashion shows and promoting a positive brand image. The results included content evaluation of satisfaction, normal, or dissatisfaction. The contextual condition was a change in the form of consumption that emphasized experience, while the interventional condition was psychological distance. Based on this, the core category was defined as "consumption patterns that emphasized the purpose of use and experience affects the metaverse fashion shows and psychological distance appeared as a user experience evaluation through the establishment of a system of metaverse fashion shows and the promotion of a positive brand image". User types were classified as active or passive. Active users have the autonomy to select content according to their individual preferences, and accordingly, their experience preference tends to change. In contrast, passive users' preference for the technical quality of content is relatively low, but they have a high concentration of content diversity and audio-visual interest elements.