Browse > Article
http://dx.doi.org/10.5909/JBE.2015.20.1.140

W3C based Interoperable Multimodal Communicator  

Park, Daemin (Multimedia Engineering, Hanbat National University)
Gwon, Daehyeok (Multimedia Engineering, Hanbat National University)
Choi, Jinhuyck (Multimedia Engineering, Hanbat National University)
Lee, Injae (Broadcasting & Telecommunications Media Research Department, ETRI)
Choi, Haechul (Multimedia Engineering, Hanbat National University)
Publication Information
Journal of Broadcast Engineering / v.20, no.1, 2015 , pp. 140-152 More about this Journal
Abstract
HCI(Human Computer Interaction) enables the interaction between people and computers by using a human-familiar interface called as Modality. Recently, to provide an optimal interface according to various devices and service environment, an advanced HCI method using multiple modalities is intensively studied. However, the multimodal interface has difficulties that modalities have different data formats and are hard to be cooperated efficiently. To solve this problem, a multimodal communicator is introduced, which is based on EMMA(Extensible Multimodal Annotation Markup language) and MMI(Multimodal Interaction Framework) of W3C(World Wide Web Consortium) standards. This standard based framework consisting of modality component, interaction manager, and presentation component makes multiple modalities interoperable and provides a wide expansion capability for other modalities. Experimental results show that the multimodal communicator is facilitated by using multiple modalities of eye tracking and gesture recognition for a map browsing scenario.
Keywords
Human Computer Interface; Multimodal; W3C; Eye tracking;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 M. Araki and K. Tachibana, “Multimodal dialog description language for Rapid system development,” Proc. 7th SIGdial workshop on discourse and dialogue, Sydney, 2006.
2 S. Scherer, S. Marsella, G. Stratou, Y. Xu, F. Morbini, A. Egan, A. Rizzo, and L-P. Morency, “Perception markup language: towards a standardized representation of perceived nonverbal behaviors,” Lecture Notes in Computer Science. Springer Berlin Heidelberg, pp 455–463, 2012.
3 S. Kopp, B. Krenn, S. Marsella, A. Marshall, C. Pelachaud, H. Pirker, K-R. Thorisson, and H. Vilhjalmsson, “Towards a common framework for multimodal generation: The behavior markup language,” Proc. International Conference on Intelligent Virtual Agent (IVAA), Marina del Rey, California, 2006.
4 D. Heylen, S. Kopp, S. Marsella, C. Pelachaud, and H. Vilhjalmsson, “The next step towards a functional markup language,” Proc. Intelligent International Conference on Virtual Agents (IVAA), Tokyo, 2008.
5 R. Vertegaal, “Attentive User Interfaces,” Communications of the ACM, Vol. 46, No. 3, pp. 33, 2003.   DOI
6 Johnston, M, P. Ehlen, D. Gibbon, and Z. Liu, “The Multimodal Presentation Dashboard,” Proc. the NAACL-HLT 2007 Workshop: Bridging the Gap: Academic and Industrial Research in Dialog Technologies, 2008
7 B. Michael, J. Michael, K. Sunil, P. Stephen , and W. Keith, “W3C Multimodal Interaction Framework,” W3C NOTE 06 May 2003.
8 B. Paolo, C. Daniel . Burnett, C. Jerry, Deborah A. Dahl, Gerry McCobb, and R. Dave, “EMMA: Extensible MultiModal Annotation markup language,” W3C Recommendation 10 February 2009.
9 J. Lim, P. Park, “Design of dataglove based multimodal interface for 3D object manipulation in virtual environment,” Journal of Human-Computer Interaction Vol. 2, No. 2, 2006.
10 G. Han, A. Baek, and H. Choi, “MPEG-U based Advanced User Interaction Interface System Using Hand Posture Recognition,” Journal of Broadcast Engineering Vol. 19, No. 1, January 2014.   DOI   ScienceOn
11 S. Glashan, D-C. Burnett, J. Carter, P. Danielsen, J. Ferrans, A. Hunt, B. Lucas, B. Porter, K. Rehor, and S. Tryphonas, “Voice Extensible Markup Language (VoiceXML 2.0),” World Wide Web Consortium(W3C), http://w3.org/TR/voicexml20/, November 9 2012.
12 J. Barnett, R. Akolkar, R-J. Auburn, M. Bodell, DC. Burnett , J. Carter, S. McGlashan, T. Lager, M. Helbing, R. Hosn, T-V. Raman, K. Reifenrath, and N. Rosenthal , “State chart XML (SCXML): state machine notation for control abstraction,” World Wide Web Consortium (W3C), http://www.w3.org/TR/scxml/. December 6 2012.