Korean Journal of Computational Design and Engineering (한국CDE학회논문집)

Society for Computational Design and Engineering (한국CDE학회)
권호 표지
DOI QR코드

DOI QR Code

Dynamic Association and Natural Interaction for Multi-Displays Using Smart Devices

다수의 스마트 디바이스를 활용한 멀티 디스플레이 동적 생성 및 인터랙션

  • Kim, Minseok (Dept. of Industrial Engineering, Chonnam Nat'l Univ.) ;
  • Lee, Jae Yeol (Dept. of Industrial Engineering, Chonnam Nat'l Univ.)
  • 김민석 (전남대학교 산업공학과) ;
  • 이재열 (전남대학교 산업공학과)
  • Received : 2015.04.16
  • Accepted : 2015.06.22
  • Published : 2015.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a dynamic association and natural interaction method for multi-displays composed of smart devices. Users can intuitively associate relations among smart devices by shake gestures, flexibly modify the layout of the display by tilt gestures, and naturally interact with the multi-display by multi-touch interactions. First of all, users shake their smart devices to create and bind a group for a multi-display with a matrix configuration in an ad-hoc and collaborative situation. After the creation of the group, if needed, their display layout can be flexibly changed by tilt gestures that move the tilted device to the nearest vacant cell in the matrix configuration. During the tilt gestures, the system automatically modifies the relation, view, and configuration of the multi-display. Finally, users can interact with the multi-display through multi-touch interactions just as they interact with a single large display. Furthermore, depending on the context or role, synchronous or asynchronous mode is available to them for providing a split view or another UI. We will show the effectiveness and advantages of the proposed approach by demonstrating implementation results and evaluating the method by the usability study.

Keywords

References

  1. Chong, M.K., Kawsar, F. and Gellersen, H., 2011, Spatial Co-location for Device Association: the Connected Object Way, Proc. the 2011 Interaction Workshop on Networking and Object Memories for the Internet of Things, pp.21-26.
  2. Conotter, V., Grassel, G. and De Natale, F.D., 2014, Multi-device Interaction for Content Sharing, Proc. the 1st International Workshop on Emerging Multimedia Applications and Services for Smart Cities, pp.29-34.
  3. Dippon, A., Wiedermann, N. and Klinker, G., 2012, Seamless Integration of Mobile Devices into Interactive Surface Environments, Proc. the 2012 ACM International Conference on Interactive Tabletops and Surfaces, pp.331-334.
  4. Fails, J.A., Druin, A. and Guha, M.L., 2010, Mobile Collaboration: Collaboratively Reading and Creating Children's Stories on Mobile Devices. Proc. the 9th International Conference on Interaction Design and Children, pp.20-29.
  5. Girouard, A., Tarun, A. and Vertegaal, R., 2012, DisplayStacks: Interaction Techniques for Stacks of Flexible Thin-film Displays. Proc. the SIGCHI Conference on Human Factors in Computing Systems, pp.2431-2440.
  6. Hinckley, K., 2003, Synchronous Gestures for Multiple Persons and Computers, Proc. UIST '03, pp. 149-158.
  7. Hinckley, K., Ramos, G., Guimbretiere, F., Baudisch, P. and Smith, M., 2004, Stitching: Pen Gestures That Span Multiple Displays, Proc. the Working Conference on Advanced Visual Interfaces, pp.23-31.
  8. Holmquist, L.E., Mattern, F., Schiele, B., Alahuhta, P., Beigl, M. and Gellersen, H.W., 2001, Smart-its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts, Proc. Ubicomp '01, pp.116-122.
  9. Huang, D.Y., Lin, C.P., Hung, Y.P., Chang, T.W., Yu, N.H., Tsai, M.L. and Chen, M.Y., 2012, MagMobile: Enhancing Social Interactions with Rapid View-stitching Games of Mobile Devices, Proc. the 11th International Conference on Mobile and Ubiquitous Multimedia, pp.61.
  10. Kim, M. and Lee, J.Y., 2015, Interactive Lens Through Smartphones for Supporting Level-ofdetailed Views in a Public Display, Journal of Computational Design and Engineering, 2(2), pp.73-78. https://doi.org/10.1016/j.jcde.2014.12.001
  11. Li, M. and Kobbelt, L., 2012, Dynamic Tiling Display: Building an Interactive Display Surface Using Multiple Mobile Devices. Proc. the 11th International Conference on Mobile and Ubiquitous Multimedia, pp.24.
  12. Lucero, A., Keränen, J. and Korhonen, H., 2010, Collaborative Use of Mobile Phones for Brainstorming, Proc. MobileHCI '10, pp.337-340.
  13. Lucero, A., Holopainen, J. and Jokela, T., 2011, Pass-them-around: Collaborative Use of Mobile Phones for Photo Sharing, Proc. CHI '11, pp.1787-1796.
  14. Lucero, A., Jones, M., Jokela, T. and Robinson, S., 2013, Mobile Collocated Interactions: Taking an Offline Break Together, Interactions, 20(2), pp.26-32. https://doi.org/10.1145/2427076.2427083
  15. Lyons, K., Pering, T., Rosario, B., Sud, S. and Want, R., 2009, Multi-display Composition: Supporting Display Sharing for Collocated Mobile Devices, Proc. the 12th IFIP TC 13 International Conference on Human-Computer Interaction, pp.758-771.
  16. Marquardt, N., Hinckley, K. and Greenberg, S., 2012, Cross-device Interaction via Micro-mobility and f-formations Proc. the 25th Annual ACM symposium on User Interface Software and Technology, pp.13-22.
  17. Merrill, D., Kalanithi, J. and Maes, P., 2007, Siftables: Towards Sensor Network User Interfaces, Proc. TEI '07, pp.75-78.
  18. Nielsen, H.S., Olsen, M.P., Skov, M.B. and Kjeldskov, J., 2014, JuxtaPinch: Exploring Multidevice Interaction in Collocated Photo Sharing. Proc. MobileHCI '14, pp.183-192.
  19. Rädle, R., Jetter, H.C., Marquardt, N., Reiterer, H. and Rogers, Y., 2014, HuddleLamp: Spatiallyaware Mobile Displays for Ad-hoc Around-thetable Collaboration, Proc. the 9th ACM International Conference on Interactive Tabletops and Surfaces, pp.45-54, 2014.
  20. Schmitz, A., Li, M., Schonefeld, V. and Kobbelt, L., 2010, Ad-hoc Multi-displays for Mobile Interactive Applications, Proc. 31st Annual Conference of the European Association for Computer Graphics, 29(2), pp.8.
  21. Schwarz, J., Klionsky, D., Harrison, C., Dietz, P. and Wilson, A., 2012, Phone as a Pixel: Enabling Ad-hoc, Large-scale Displays Using Mobile Devices. Proc. CHI '12, pp.2235-2238.
  22. Sohn, M.G., Lee, S.T. and Lee, J.Y., 2014, Research on Cognitive Effects and Responsiveness of Smartphone-based Augmented Reality Navigation, Trans. of the Society of CAD/CAM Engineers, 19(3), pp.272-280. https://doi.org/10.7315/CADCAM.2014.272
  23. Tilt Tray Sorter, http://www.bastiansolutions.com/automation/conveyor-systems/conveyor-sortation-systems/sortation-conveyors/tilt-tray-sorters