• Journal of Information Processing Systems
• /
• v.13 no.1
• /
• pp.8-25
• /
• 2017

This survey paper explores the application of multimodal feedback in automated systems for motor learning. In this paper, we review the findings shown in recent studies in this field using rehabilitation and various motor training scenarios as context. We discuss popular feedback delivery and sensing mechanisms for motion capture and processing in terms of requirements, benefits, and limitations. The selection of modalities is presented via our having reviewed the best-practice approaches for each modality relative to motor task complexity with example implementations in recent work. We summarize the advantages and disadvantages of several approaches for integrating modalities in terms of fusion and frequency of feedback during motor tasks. Finally, we review the limitations of perceptual bandwidth and provide an evaluation of the information transfer for each modality.

• Proceedings of the Korean Society for Emotion and Sensibility Conference
• /
• 2011.11a
• /
• pp.68-69
• /
• 2011

• 한국HCI학회:학술대회논문집
• /
• 2006.02a
• /
• pp.579-586
• /
• 2006

모바일 환경에서 휴대폰을 사용하다 보면 무의식적으로 전화를 받지 못하는(Missing call) 상황이 빈번하게 일어난다. 휴대폰에서는 기본적으로 시각 청각 촉각의 다중양식 피드백(Multimodal feedback)을 제공하고 있음에도 불구하고, 이렇게 Missing call 이 발생하는 현상에는 여러 가지 다양한 원인이 존재할 것이다. 본 연구에서는 이러한 원인을 찾기 위해 모바일 환경 내 휴대폰 사용자 행동연구를 실시하여 일반적인 휴대패턴을 분석하고, Missing call과 관련하여 주로 설정하는 수신모드와 휴대패턴의 상호영향을 연구하고자 하였다. 본 연구결과, 모바일 환경에서 휴대폰 수신 탐지능력에는 인지심리학적(감각과 주의관련), 환경적, 행동학적 요인이 영향을 미칠 수 있는 것으로 나타났다. 또한 모바일 환경에서 주로 사용하는 수신모드인 진동모드 설정시, 휴대폰 사용자가 속한 환경에 다른 요인들이 복합적으로 존재할 경우 휴대폰 수신에 대한 탐지만족도가 감소함을 확인할 수 있었다.

• The Journal of the Korea Contents Association
• /
• v.10 no.8
• /
• pp.1-10
• /
• 2010

For virtual education, the multimodal learning environment with haptic feedback, termed 'immersive textbook', is necessary to enhance the learning effectiveness. However, the learning contents for immersive textbook are not widely available due to the constraints in creation and playback environments. To address this problem, we propose a framework for producing and displaying the multimodal contents for immersive textbook. Our framework provides an XML-based meta-language to produce the multimodal learning contents in the form of intuitive script. Thus it can help the user, without any prior knowledge of multimodal interactions, produce his or her own learning contents. The contents are then interpreted by script engine and delivered to the user by visual and haptic rendering loops. Also we implemented a prototype based on the aforementioned proposals and performed user evaluation to verify the validity of our framework.

• 한국HCI학회:학술대회논문집
• /
• 2007.02b
• /
• pp.118-123
• /
• 2007

본 논문에서는 증강현실 기반 제품 디자인 시, 논문 [1]에서 제안된 몰입감 향상 기법들에 대한 사용성 평가를 통해, 사용자에게 다차원 피드백(multimodal feedback)을 제공하는 것이 몰입감 향상에 어떠한 영향을 미치는가에 대해 알아본다. 우선 기존의 데스크톱 기반의 디자인 환경과 다차원 피드백을 가능하게 하는 증강현실 기반 디자인 환경과의 비교 후, 시각 피드백에 대한 사용성 평가로써 증강된 가상객체에 의해 가려진 손을 다시 보이게 함으로 얻게 되는 실재감 향상 정도와 사용자의 입력에 따른 컨텐츠의 시각적 맵핑의 적절성을 평가한다. 또한 청각 피드백에 대한 평가 기준으로써 시나리오 상의 상황에 따른 배경음/효과음 맵핑의 몰입감과 적절성을 평가한다. 마지막으로 촉각 피드백에 대한 평가로써, 물리적인 감각형 객체에 가상객체를 정합함으로써 얻게 되는 촉감과 시나리오 상의 이벤트에 따라 제시되는 진동-촉각 피드백에 의한 몰입감 향상 정도를 평가한다. 제시된 사용성 평가의 결과는 기존의 증강현실 기반 디자인 시스템의 몰입감을 향상 시키기 위한 구현에 있어, 고려될 수 있는 사항으로써 의미가 있다.

• Journal of KIISE:Computing Practices and Letters
• /
• v.9 no.4
• /
• pp.420-430
• /
• 2003

This paper presents an immersive multimodal Gumdo simulation game that allows a user to experience the whole body interaction with an intelligent cyber fencer. The proposed system consists of three modules: (i) a nondistracting multimodal interface with 3D vision and speech (ii) an intelligent cyber fencer and (iii) an immersive feedback by a big screen and sound. First, the multimodal Interface with 3D vision and speech allows a user to move around and to shout without distracting the user. Second, an intelligent cyber fencer provides the user with intelligent interactions by perception and reaction modules that are created by the analysis of real Gumdo game. Finally, an immersive audio-visual feedback by a big screen and sound effects helps a user experience an immersive interaction. The proposed system thus provides the user with an immersive Gumdo experience with the whole body movement. The suggested system can be applied to various applications such as education, exercise, art performance, etc.

• The Journal of the Korea Contents Association
• /
• v.18 no.8
• /
• pp.92-101
• /
• 2018

As the interactive AI speaker becomes popular, voice recognition is regarded as an important vehicle-driver interaction method in case of autonomous driving situation. The purpose of this study is to confirm whether multimodal interaction in which feedback is transmitted by auditory and visual mode of AI characters on screen is more effective in user experience optimization than auditory mode only. We performed the interaction tasks for the music selection and adjustment through the AI speaker while driving to the experiment participant and measured the information and system quality, presence, the perceived usefulness and ease of use, and the continuance intention. As a result of analysis, the multimodal effect of visual characters was not shown in most user experience factors, and the effect was not shown in the intention of continuous use. Rather, it was found that auditory single mode was more effective than multimodal in information quality factor. In the semi-autonomous driving stage, which requires driver 's cognitive effort, multimodal interaction is not effective in optimizing user experience as compared to single mode interaction.

• Journal of The Institute of Information and Telecommunication Facilities Engineering
• /
• v.10 no.4
• /
• pp.143-147
• /
• 2011

The system using augmented reality can save the time and cost. It is verified in various fields under the possibility of a technology by solving unrealistic feeling in the virtual space. Therefore, augmented reality has a variety of the potential to be used. Generally, multimodal senses such as visual/auditory/tactile feed back are well known as a method for enhancing the immersion in case of interaction with virtual object. By adapting tangible object we can provide touch sensation to users. a 3D model of the same scale overlays the whole area of the tangible object; thus, the marker area is invisible. This contributes to enhancing immersive and natural images to users. Finally, multimodal feedback also creates better immersion. In this paper, sound feedback is considered. By further improving immersion learning augmented reality for children with the initial step learning content is presented. Augmented reality is in the intermediate stages between future world and real world as well as its adaptability is estimated more than virtual reality.

• The Journal of Korean Institute for Practical Engineering Education
• /
• v.5 no.1
• /
• pp.40-44
• /
• 2013

This paper suggests a multimodal interface control module which allows a student to naturally interact with educational contents in virtual environment. The suggested module recognizes a user's motion when he/she interacts with virtual environment and then conveys the user's motion to the virtual environment via wireless communication. Futhermore, a haptic actuator is incorporated into the proposed module in order to create haptic information. Due to the proposed module, a user can haptically sense the virtual object as if the virtual object is exists in real world.

• Journal of the Korean Society of Physical Medicine
• /
• v.16 no.4
• /
• pp.125-137
• /
• 2021

PURPOSE: The growing number of people exposed to a static sitting posture has resulted in an increase in people with a poor posture out of the optimally aligned posture because of the low awareness of a correct sitting posture. Learning the correct sitting posture by applying sensory feedback is essential because a poor posture has negative consequences for the spine. Therefore, this study examined the effects of the sensory feedback types on learning correct sitting posture. METHODS: Thirty-six healthy adult males were assigned to a visual feedback group, a tactile feedback group, and a visuotactile feedback group to learn the correct sitting posture by applying sensory feedback. The spine angle, muscle activity, and muscle thickness were measured in the sitting position using retro-reflexive markers, electromyography, and ultrasound immediately after, five minutes, and 10 minutes after intervention. RESULTS: The intervention time was significantly shorter in the visuotactile feedback group than the visual feedback group (p < .05). Compared to the pre-intervention, the repositioning error angles of the thoracic and lumbar vertebrae of all groups were reduced significantly immediately after intervention and after five minutes. After 10 minutes, there was a significant difference in the thoracic and lumbar repositioning error angles of the tactile feedback group and the visuotactile feedback group (p < .05). No significant difference was noted at any time compared to the pre-intervention in all groups (p > .05). CONCLUSION: The use of tactile and visuotactile feedback in intervention to correct the sitting posture is proposed.