• MALSORI
• /
• no.67
• /
• pp.103-120
• /
• 2008

Basic components for multimodal interface, such as speech recognition, speech synthesis, gesture recognition, and multimodal fusion, have their own technological limitations. For example, the accuracy of speech recognition decreases for large vocabulary and in noisy environments. In spite of those technological limitations, there are lots of applications in which speech-oriented multimodal user interfaces are very helpful to users. However, in order to expand application areas for speech-oriented multimodal interfaces, we have to develop the interfaces focused on usability. In this paper, we introduce usability and user-centered design methodology in general. There has been much work for evaluating spoken dialogue systems. We give a summary for PARADISE (PARAdigm for Dialogue System Evaluation) and PROMISE (PROcedure for Multimodal Interactive System Evaluation) that are the generalized evaluation frameworks for voice and multimodal user interfaces. Then, we present usability components for speech-oriented multimodal user interfaces and usability testing guidelines that can be used in a user-centered multimodal interface design process.

• Proceedings of the IEEK Conference
• /
• 2000.11c
• /
• pp.121-124
• /
• 2000

In this paper, we propose a pen gesture recognition system for user interface in multimedia terminal which requires fast processing time and high recognition rate. It is realtime and interaction system between graphic and text module. Text editing in recognition system is performed by pen gesture in graphic module or direct editing in text module, and has all 14 editing functions. The pen gesture recognition is performed by searching classification features that extracted from input strokes at pen gesture model. The pen gesture model has been constructed by classification features, ie, cross number, direction change, direction code number, position relation, distance ratio information about defined 15 types. The proposed recognition system has obtained 98% correct recognition rate and 30msec average processing time in a recognition experiment.

• 한국HCI학회:학술대회논문집
• /
• 2006.02a
• /
• pp.666-671
• /
• 2006

'Multi modal 인터페이스'란 인간과 기계의 통신을 위해 음성, 키보드, 펜을 이용, 인터페이스를 하는 방법을 말한다. 최근 들어 많은 휴대용 단말기가 보급 되고, 단말기가 소형화, 지능화 되어가고, 단말기의 어플리케이션도 다양해짐에 따라 사용자가 보다 편리하고 쉽게 사용할 수 있는 입력 방법에 기대치가 높아가고 있다. 현재 휴대용 단말기에 가능한 입력장치는 단지 단말기의 버튼이나 터치 패드(PDA 경우)이다. 하지만 장애인의 경우 버튼이나 터치 패드를 사용하기 어렵고, 휴대용 단말기로 게임을 하는데 있어서도, 어려움이 많으며 새로운 게임이나 어플리케이션 개발에도 많은 장애요인이 되고 있다. 이런 문제점들은 극복하기 위하여, 본 논문에서는 휴대용 단말기의 새로운 Multi Modal 인터페이스를 제시 하였다. PDA(Personal Digital Assistants)를 이용하여 더 낳은 재미와 실감을 줄 수 있는 Multi Modal 인터페이스를 개발하였다. 센서를 이용하여 휴대용 단말기를 손목으로 제어를 가능하게 함으로서, 사용자에게 편리하고 색다른 입력 장치를 제공 하였다. 향후 음성 인식 기능이 추가 된다면, 인간과 인간 사이의 통신은 음성과 제스처를 이용하듯이 기계에서는 전통적으로 키보드 나 버튼을 사용하지 않고 인간처럼 음성과 제스처를 통해 통신할 수 있을 것이다. 또한 여기에 진동자를 이용하여 촉감을 부여함으로써, 그 동안 멀티 모달 인터페이스에 소외된 시각 장애인, 노약자들에게도 정보를 제공할 수 있다. 실제로 사람은 시각이나 청각보다 촉각에 훨씬 빠르게 반응한다. 이 시스템을 게임을 하는 사용자한테 적용한다면, 능동적으로 게임참여 함으로서 좀더 실감나는 재미를 제공할 수 있다. 특수한 상황에서는 은밀한 정보를 제공할 수 있으며, 앞으로 개발될 모바일 응용 서비스에 사용될 수 있다.

• Proceedings of the KSPS conference
• /
• 2007.05a
• /
• pp.41-44
• /
• 2007

In this paper, we introduce a system in which car navigation scenario is plugged multimodal interface based on multimodal middleware. In map-based system, the combination of speech and pen input/output modalities can offer users better expressive power. To be able to achieve multimodal task in car environments, we have chosen SCXML(State Chart XML), a multimodal authoring language of W3C standard, to control modality components as XHTML, VoiceXML and GPS. In Network Manager, GPS signals from navigation software are converted to EMMA meta language, sent to MultiModal Interaction Runtime Framework(MMI). Not only does MMI handles GPS signals and a user's multimodal I/Os but also it combines them with information of device, user preference and reasoned RDF to give the user intelligent or personalized services. The self-simulation test has shown that middleware accomplish a navigational multimodal task over multiple users in car environments.

• Journal of Korea Entertainment Industry Association
• /
• v.8 no.2
• /
• pp.283-297
• /
• 2014

The future of human life is expected to be innovative by increasing social, economic, political and personal, including all areas of life across the multi-disciplinary skills. Particularly, in the field of robotics and next-generation games with robots, by multidisciplinary contributions and interaction, convergence between technology is expected to accelerate more and more. The purpose of this study is that by new interface model beyond the technical limitations of the "human-robot interface technology," until now and time and spatial constraints and through fusion of various modalities which existing human-robot interface technologies can't have, the research of more reliable and easy free "human-robot interface technology". This is the research of robot game system which develop and utilizing real time synchronization engine linking between biped humanoid robot and the behavior of the position value of mobile device screen's 3D content (contents), robot (virtual robots), the wireless protocol for sending and receiving (Protocol) mutual information and development of a teaching program of "Direct Teaching & Play" by the study for effective teaching.

• Journal of Internet Computing and Services
• /
• v.15 no.3
• /
• pp.11-19
• /
• 2014

The natural user interface/experience (NUI/NUX) is used for the natural motion interface without using device or tool such as mice, keyboards, pens and markers. Up to now, typical motion recognition methods used markers to receive coordinate input values of each marker as relative data and to store each coordinate value into the database. But, to recognize accurate motion, more markers are needed and much time is taken in attaching makers and processing the data. Also, as NUI/NUX framework being developed except for the most important intuition, problems for use arise and are forced for users to learn many NUI/NUX framework usages. To compensate for this problem in this paper, we didn't use markers and implemented for anyone to handle it. Also, we designed multi-modal NUI/NUX framework controlling voice, body motion, and facial expression simultaneously, and proposed a new algorithm of mouse operation by recognizing intuitive hand gesture and mapping it on the monitor. We implement it for user to handle the "hand mouse" operation easily and intuitively.

• Journal of Korea Multimedia Society
• /
• v.6 no.2
• /
• pp.216-223
• /
• 2003

In this paper, we present SCCRS(Speech and Character Combined Recognition System) for speaker /writer independent. on-line multimodal interfaces. In general, it has been known that the CHMM(Continuous Hidden Markov Mode] ) is very useful method for speech recognition and on-line character recognition, respectively. In the proposed method, the same CHMM is applied to both speech and character recognition, so as to construct a combined system. For such a purpose, 115 CHMM having 3 states and 9 transitions are constructed using MLE(Maximum Likelihood Estimation) algorithm. Different features are extracted for speech and character recognition: MFCC(Mel Frequency Cepstrum Coefficient) Is used for speech in the preprocessing, while position parameter is utilized for cursive character At recognition step, the proposed SCCRS employs OPDP (One Pass Dynamic Programming), so as to be a practical combined recognition system. Experimental results show that the recognition rates for voice phoneme, voice word, cursive character grapheme, and cursive character word are 51.65%, 88.6%, 85.3%, and 85.6%, respectively, when not using any language models. It demonstrates the efficiency of the proposed system.

• Journal of KIISE:Software and Applications
• /
• v.35 no.4
• /
• pp.245-254
• /
• 2008

This paper proposes a unified multi-modal input framework to interface the recognition engines such as IBM ViaVoice and Microsoft handwriting-recognition system with general window applications, particularly, for pen-input displays. As soon as user pushes a hardware button attached to the pin-input display with one hand, the current window of focus such as a internet search window and a word processor is overlaid with a transparent window covering the whole desktop; upon which user inputs handwriting with the other hand, without losing the focus of attention on working context. As well as freeform handwriting on this transparent input overlay as a sketch pad, the user can dictate some words and draw diagrams to communicate with the system.

• The Journal of the Convergence on Culture Technology
• /
• v.9 no.1
• /
• pp.787-797
• /
• 2023

The use of service robots, particularly guide robots, is becoming increasingly prevalent in public institutions. However, there has been limited research into the interactions between users and guide robots. To explore the customer experience with the guidance robot, we selected 'QI', which has been meeting customers for the longest time, and collected all reviews since the service was launched in public institutions. By using text mining techniques, we identified the main keywords and user experience factors and examined factors that hinder user experience. As a result, the guide robot's functionality, appearance, interaction methods, and role as a cultural commentator and helper were key factors that influenced the user experience. After identifying hindrance factors, we suggested solutions such as improved interaction design, multimodal interface service design, and content development. This study contributes to the understanding of user experience with guide robots and provides practical suggestions for improvement.

• Journal of Korea Multimedia Society
• /
• v.15 no.11
• /
• pp.1384-1390
• /
• 2012

It is well known that the exercising and stimulating of fingers have an important bearing on the brain. We take note of the fact and develope a game device for improving one's health and brain ability in respect of the education and training. Especially, we develope the device focused on the balanced exercising of five finger for improving brain function. The game device is possible to used in two-ways, namely online and off-line mode. In online mode, the device is connected with other visual devices such as smart phone and smart TV and communicated with Bluetooth and it is used as a MMI(multi-modal interface) device. Whereas, in off-line mode the game device works independently and it makes possible to enjoy auditorial and tactual games without video images for promotion of brain activity and emotional cultivation. For certification of the device, we implement two games(a fishing game for off-line mode and a shooting game for online mode) for people of all age, especially good for the elderly. It is usable as a game device to offering the elderly a great help for preventing impairment of the cognitive functions.