Browse > Article
http://dx.doi.org/10.7315/CADCAM.2015.337

Dynamic Association and Natural Interaction for Multi-Displays Using Smart Devices  

Kim, Minseok (Dept. of Industrial Engineering, Chonnam Nat'l Univ.)
Lee, Jae Yeol (Dept. of Industrial Engineering, Chonnam Nat'l Univ.)
Publication Information
Korean Journal of Computational Design and Engineering / v.20, no.4, 2015 , pp. 337-347 More about this Journal
Abstract
This paper presents a dynamic association and natural interaction method for multi-displays composed of smart devices. Users can intuitively associate relations among smart devices by shake gestures, flexibly modify the layout of the display by tilt gestures, and naturally interact with the multi-display by multi-touch interactions. First of all, users shake their smart devices to create and bind a group for a multi-display with a matrix configuration in an ad-hoc and collaborative situation. After the creation of the group, if needed, their display layout can be flexibly changed by tilt gestures that move the tilted device to the nearest vacant cell in the matrix configuration. During the tilt gestures, the system automatically modifies the relation, view, and configuration of the multi-display. Finally, users can interact with the multi-display through multi-touch interactions just as they interact with a single large display. Furthermore, depending on the context or role, synchronous or asynchronous mode is available to them for providing a split view or another UI. We will show the effectiveness and advantages of the proposed approach by demonstrating implementation results and evaluating the method by the usability study.
Keywords
Dynamic association; Gesture-based interaction; Multi-display; Natural interaction; Smart devices;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Chong, M.K., Kawsar, F. and Gellersen, H., 2011, Spatial Co-location for Device Association: the Connected Object Way, Proc. the 2011 Interaction Workshop on Networking and Object Memories for the Internet of Things, pp.21-26.
2 Conotter, V., Grassel, G. and De Natale, F.D., 2014, Multi-device Interaction for Content Sharing, Proc. the 1st International Workshop on Emerging Multimedia Applications and Services for Smart Cities, pp.29-34.
3 Dippon, A., Wiedermann, N. and Klinker, G., 2012, Seamless Integration of Mobile Devices into Interactive Surface Environments, Proc. the 2012 ACM International Conference on Interactive Tabletops and Surfaces, pp.331-334.
4 Fails, J.A., Druin, A. and Guha, M.L., 2010, Mobile Collaboration: Collaboratively Reading and Creating Children's Stories on Mobile Devices. Proc. the 9th International Conference on Interaction Design and Children, pp.20-29.
5 Girouard, A., Tarun, A. and Vertegaal, R., 2012, DisplayStacks: Interaction Techniques for Stacks of Flexible Thin-film Displays. Proc. the SIGCHI Conference on Human Factors in Computing Systems, pp.2431-2440.
6 Hinckley, K., 2003, Synchronous Gestures for Multiple Persons and Computers, Proc. UIST '03, pp. 149-158.
7 Hinckley, K., Ramos, G., Guimbretiere, F., Baudisch, P. and Smith, M., 2004, Stitching: Pen Gestures That Span Multiple Displays, Proc. the Working Conference on Advanced Visual Interfaces, pp.23-31.
8 Holmquist, L.E., Mattern, F., Schiele, B., Alahuhta, P., Beigl, M. and Gellersen, H.W., 2001, Smart-its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts, Proc. Ubicomp '01, pp.116-122.
9 Huang, D.Y., Lin, C.P., Hung, Y.P., Chang, T.W., Yu, N.H., Tsai, M.L. and Chen, M.Y., 2012, MagMobile: Enhancing Social Interactions with Rapid View-stitching Games of Mobile Devices, Proc. the 11th International Conference on Mobile and Ubiquitous Multimedia, pp.61.
10 Kim, M. and Lee, J.Y., 2015, Interactive Lens Through Smartphones for Supporting Level-ofdetailed Views in a Public Display, Journal of Computational Design and Engineering, 2(2), pp.73-78.   DOI
11 Li, M. and Kobbelt, L., 2012, Dynamic Tiling Display: Building an Interactive Display Surface Using Multiple Mobile Devices. Proc. the 11th International Conference on Mobile and Ubiquitous Multimedia, pp.24.
12 Lucero, A., Keränen, J. and Korhonen, H., 2010, Collaborative Use of Mobile Phones for Brainstorming, Proc. MobileHCI '10, pp.337-340.
13 Lucero, A., Holopainen, J. and Jokela, T., 2011, Pass-them-around: Collaborative Use of Mobile Phones for Photo Sharing, Proc. CHI '11, pp.1787-1796.
14 Lucero, A., Jones, M., Jokela, T. and Robinson, S., 2013, Mobile Collocated Interactions: Taking an Offline Break Together, Interactions, 20(2), pp.26-32.   DOI
15 Lyons, K., Pering, T., Rosario, B., Sud, S. and Want, R., 2009, Multi-display Composition: Supporting Display Sharing for Collocated Mobile Devices, Proc. the 12th IFIP TC 13 International Conference on Human-Computer Interaction, pp.758-771.
16 Marquardt, N., Hinckley, K. and Greenberg, S., 2012, Cross-device Interaction via Micro-mobility and f-formations Proc. the 25th Annual ACM symposium on User Interface Software and Technology, pp.13-22.
17 Merrill, D., Kalanithi, J. and Maes, P., 2007, Siftables: Towards Sensor Network User Interfaces, Proc. TEI '07, pp.75-78.
18 Nielsen, H.S., Olsen, M.P., Skov, M.B. and Kjeldskov, J., 2014, JuxtaPinch: Exploring Multidevice Interaction in Collocated Photo Sharing. Proc. MobileHCI '14, pp.183-192.
19 Schmitz, A., Li, M., Schonefeld, V. and Kobbelt, L., 2010, Ad-hoc Multi-displays for Mobile Interactive Applications, Proc. 31st Annual Conference of the European Association for Computer Graphics, 29(2), pp.8.
20 Rädle, R., Jetter, H.C., Marquardt, N., Reiterer, H. and Rogers, Y., 2014, HuddleLamp: Spatiallyaware Mobile Displays for Ad-hoc Around-thetable Collaboration, Proc. the 9th ACM International Conference on Interactive Tabletops and Surfaces, pp.45-54, 2014.
21 Schwarz, J., Klionsky, D., Harrison, C., Dietz, P. and Wilson, A., 2012, Phone as a Pixel: Enabling Ad-hoc, Large-scale Displays Using Mobile Devices. Proc. CHI '12, pp.2235-2238.
22 Sohn, M.G., Lee, S.T. and Lee, J.Y., 2014, Research on Cognitive Effects and Responsiveness of Smartphone-based Augmented Reality Navigation, Trans. of the Society of CAD/CAM Engineers, 19(3), pp.272-280.   DOI
23 Tilt Tray Sorter, http://www.bastiansolutions.com/automation/conveyor-systems/conveyor-sortation-systems/sortation-conveyors/tilt-tray-sorters