Browse > Article

The Effects of Interface Modality on Cognitive Load and Task Performance in Media Multitasking Environment  

Lee, Dana (연세대학교 일반대학원 인지과학협동과정)
Han, Kwang-Hee (연세대학교 심리학과, 인지과학협동과정)
Publication Information
Journal of the HCI Society of Korea / v.14, no.2, 2019 , pp. 31-39 More about this Journal
Abstract
This research examined the changes that fast-growing voice-based devices would bring in the media multitasking environment. Based on the theoretical background that information processing efficiency improves when performing multiple tasks requiring different resource structures at the same time, we conducted an experiment where participants searched for information with voice-based or screen-based devices while performing an additional visual task. Results showed that both task performance environment and interface modality had significant main effects on cognitive load. The overall cognitive load level was higher in the voice interface group, but the difference in cognitive load between the two groups decreased in a multitasking environment where the additional visual resources was required. The visual task performance was significantly higher when using the voice interface than the screen interface. Our findings suggest that voice interfaces offered advantages in the cognitive load and task performance by distributing two tasks to the auditory and visual channels. The results of this study imply that voice-based devices have the potential to facilitate efficient information processing in the screen-centric environment where visual resources collide. We provided theoretical evidence of resource distribution using multiple resource theory and tried to identify the advantages of the voice interface more specifically based on the threaded cognition model.
Keywords
Voice Interface; Media Multitasking; Cognitive Load; Sensory Modality; Multiple Resource Theory;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Wang, Z., Irwin, M., Cooper, C. and Srivastava, J. Multidimensions of media multitasking and adaptive media selection. Human Communication Research. 41(1). International Communication Association. pp. 102-127. 2015.   DOI
2 Canalys. Smart speaker installed base to hit 100 million by end of 2018. https://www.canalys.com/newsroom/smart-speaker-installed-base-to-hit-100-million-by-end-of-2018 January 14. 2019.
3 Lang, A. The limited capacity model of mediated message processing. Journal of communication. 50(1). International Communication Association. pp. 46-70. 2000.   DOI
4 Mayer, R. E. and Moreno, R. Nine ways to reduce cognitive load in multimedia learning. Educational psychologist. 38(1). Taylor & Francis. pp. 43-52. 2003.   DOI
5 Wickens, C. D. Multiple resources and performance prediction. Theoretical issues in Ergonomics Science. 3(2). Taylor & Francis. pp. 159-177. 2002.   DOI
6 Salvucci, D. D. and Taatgen, N. A. Threaded cognition: An integrated theory of concurrent multitasking. Psychological review. 115(1). American Psychological Association. pp. 101-130. 2008.   DOI
7 Salvucci, D. D., Taatgen, N. A. and Borst, J. P. Toward a unified theory of the multitasking continuum: From concurrent performance to task switching, interruption, and resumption. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM. pp. 1819-1828. 2009.
8 Armstrong, G. B. and Chung, L. Background television and reading memory in context: Assessing TV interference and facilitative context effects on encoding versus retrieval processes. Communication Research. 27(3). SAGE journals. pp. 327-352. 2000.
9 Bowman, L. L., Levine, L. E., Waite, B. M. and Gendron, M. Can students really multitask? An experimental study of instant messaging while reading. Computers & Education. 54(4). Elsevier. pp. 927-931. 2010.   DOI
10 Bolls, P. D. and Muehling, D. D. The effects of dual-task processing on consumers' responses to high-and low-imagery radio advertisements. Journal of Advertising. 36(4). Taylor & Francis. pp. 35-47. 2007.   DOI
11 Pool, M. M., Van Der Voort, T. H., Beentjes, J. W. and Koolstra, C. M. Background television as an inhibitor of performance on easy and difficult homework assignments. Communication Research. 27(3). SAGE journals. pp. 293-326. 2000.
12 Jeong, S. H. and Hwang, Y. Multitasking and persuasion: The role of structural interference. Media Psychology. 18(4). Taylor & Francis. pp. 451-474. 2015.   DOI
13 Hwang, Y. and Jeong, S. H. Multitasking and task performance: Roles of task hierarchy, sensory interference, and behavioral response. Computers in Human Behavior. 81. Elsevier. pp. 161-167. 2018.   DOI
14 Maulsby, D., Greenberg, S. and Mander, R. Prototyping an intelligent agent through Wizard of Oz. In Proceedings of the INTERACT '93 and CHI'93 conference on Human factors in computing systems. ACM. pp. 277-284. 1993.
15 Lauricella, A. R., Blackwell, C. K. and Wartella, E. The "New" technology environment: The role of content and context on learning and development from mobile media. In Media exposure during infancy and early childhood. Springer. pp. 1-23. 2017.
16 Wang, Z., David, P., Srivastava, J., Powers, S., Brady, C., D'Angelo, J. and Moreland, J. Behavioral performance and visual attention in communication multitasking: A comparison between instant messaging and online voice chat. Computers in Human Behavior. 28(3). Elsevier. pp. 968-975. 2012.   DOI
17 Myers, C., Furqan, A., Nebolsky, J., Caro, K. and Zhu, J. Patterns for How Users Overcome Obstacles in Voice User Interfaces. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM. pp. 6. 2018.
18 D Hart, S. G. and Staveland, L. E. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology. 52. Elsevier. pp. 139-183. 1988.   DOI
19 NAVER Corp. 네이버 - NAVER. https://itunes.apple.com/us/app/id393499958?mt=8 2019.1.14.
20 Paas, F., Tuovinen, J. E., Tabbers, H. and VanGerven, P. W. Cognitive load measurement as a means to advance cognitive load theory. Educational psychologist. 38(1). Taylor & Francis. pp. 63-71. 2003.   DOI
21 Yeh, Y. Y. and Wickens, C. D. Dissociation of performance and subjective measures of workload. Human Factors. 30(1). SAGE journals. pp. 111-120. 1988.
22 Sweller, J. Cognitive load theory, learning difficulty, and instructional design. Learning and instruction. 4(4). Elsevier. pp. 295-312. 1994.   DOI
23 Facebook business. Mobile and TV: Between the Screens. https://www.facebook.com/business/news/insights/mobile-and-tv-between-the-screens#Multiscreening-has-gone-mainstream January 14. 2019.
24 Limerick, H., Moore, J. W. and Coyle, D. Empirical evidence for a diminished sense of agency in speech interfaces. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM. pp. 3967-3970. 2015.
25 Corbett, E. and Weber, A. What can I say?: addressing user experience challenges of a mobile voice user interface for accessibility. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services. ACM. pp. 72-82. 2016.
26 Getty, D., Biondi, F., Morgan, S. D., Cooper, J. M. and Strayer, D. L. The Effects of Voice System Design Components on Driver Workload. Transportation Research Record. 0361198118777382. SAGE journals. pp. 1-7. 2018.   DOI
27 Van Cauwenberge, A., Schaap, G. and Van Roy, R. "TV no longer commands our full attention": Effects of second-screen viewing and task relevance on cognitive load and learning from news. Computers in Human Behavior. 38. Elsevier. pp. 100-109. 2014.   DOI
28 NewsMediaWorks. What we learned fromthe 2018 Deloitte Media Consumer Survey. https://newsmediaworks.com.au/what-we-learnedfrom-the-2018-deloitte-media-consumer-survey January 14. 2019.