Browse > Article
http://dx.doi.org/10.5762/KAIS.2013.14.2.793

Web Usability Testing by Using Scanpath Similarity Analysis  

Kim, Youngjun (Department of Psychology, Ajou University)
Kim, Youngjin (Department of Psychology, Ajou University)
Publication Information
Journal of the Korea Academia-Industrial cooperation Society / v.14, no.2, 2013 , pp. 793-803 More about this Journal
Abstract
This study was performed to determine the usefulness of scanpath similarity analysis as one of new web usability testing. The 5 websites of public institutions were used and 15 students participated. First of all, eye movements were tracked and visual appeal ratings were measured as participants freely viewed each website for 3 seconds. Subsequently in continuously tracking the eye movements we asked the participants to perform 17 missions. Finally, in interview the participants rated on satisfaction, awareness, and mission difficulty. Results of this study showed that scanpath similarity had a significant relationship with both the visual appeal ratings(first impression) and the satisfaction. In other words, higher the visual appeal ratings were related to higher scanpath similarity. This result showed that measurement such as scanpath similarity of eye movements could become an objective index for usability testing instead of subjective evaluation such as the satisfaction. We discussed possibility that the usability testing by using the scanpath similarity with both fixation and duration on eye movements will find more appropriately inference on observers' experiences in websites.
Keywords
Eye-tracking; First Impression; ROI(Region of Interest); Scanpath Similarity Analysis; Web Usability Testing;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Y. Puzis. "Accessible web automation interface: A user study". Proc. of the 14th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS). 2012. DOI: http://dx.doi.org/10.1145/2384916.2384999   DOI
2 D. A. Norman, The Design of Everyday Things, New York: Doubleday. 1998.
3 J. Nielsen, Usability Engineering, Boston, MA: Academic Press. 1993.
4 B. Keevil, Measuring the usability index of your Web site, Proc. of the 16th annual international conference on Computer documentation(ACM), pp. 271-277. 1998.
5 A. Fernandez, S. Abrahao, and E. Insfran, "Empirical validation of a usability inspection method for model-driven Web development", Journal of Systems and Software, 86, 1, pp. 161-186, 2013. DOI: http://dx.doi.org/10.1016/j.jss.2012.07.043   DOI   ScienceOn
6 A. P. Afonso, J. R. Lima, and M. P. Cota. "A heuristic evaluation of usability of Web interfaces". Proc. of Iberian Conference on Information Systems and Technologies(CISTI), 2012.
7 J. M. Spool, Web Site Usability : A Designer's Guide. San Francisco, EUA: Morgan Kaufmann. 1999.
8 T. E. Moon and H. N. Moon, "A study on the evaluation and improvement methods of web accessibility and usability of Korea government department websites.", Daehan Journal of Business, 22, 3, pp. 1511-1535, 2009.
9 J. G. Min and K. S. Lee, "Quality Evaluation for the Usability of Multimedia Web Sites", Journal of the Korea Society of Computer and Information, 11, 5, pp. 139-148, 2006.   과학기술학회마을
10 M. S. Lee and K. S. Lee, "Quality Evaluation for Usability of Internet Shopping Mall", Journal of the Korea Society of Computer and Information, 12, 6, pp. 59-68, 2007.   과학기술학회마을
11 H. W. Kwak, J. U. Kwahk, S. J. Kim, and J. M. Lee, "Usability testing of the domestic web site design: questionnaire and heuristic evaluation", Koean Journal of Cognitive Science, 11, 1, pp. 33-45, 2000.   과학기술학회마을
12 B. J. Kim and K. P. Lee, "Development of Integrated Analysis Model and Tool for Web Usability Test-with Emphasis on Eye-Tracking, Mouse-Tracking, and Retrospective Think Aloud.", Journal of Korean Society of Design Science, 20, 5, pp. 39-50, 2007.
13 K. Choi, J. Shu, and Y. Kim, "An Investigation of Banner Ad Effects Using Eye Tracking: An Explorative Observation", The Korean Journal of Experimental Psychology, 16, 4, pp. 421-434, 2004.
14 D. Rosaci and G. M. L. Sarne, "Recommending multimedia web services in a multi-device environment", Information Systems, 38, 2, pp. 198-212, 2013. DOI: http://dx.doi.org/10.1016/j.is.2012.08.002   DOI   ScienceOn
15 A. L. Yarbus, Eye movements and vision, New York: Plenum Press. 1967.
16 S. Josephson and M. E. Holmes, "Attention to repeated images on the World-Wide Web: Another look at scanpath theory", Behavior Research Methods Instruments & Computers, 34, 4, pp. 539-548, 2002. DOI: http://dx.doi.org/10.3758/BF03195483   DOI   ScienceOn
17 C. M. Privitera and L. W. Stark, "Algorithms for defining visual regions-of-interest: Comparison with eye fixations", Ieee Transactions on Pattern Analysis and Machine Intelligence, 22, 9, pp. 970-982, 2000. DOI: http://dx.doi.org/10.1109/34.877520   DOI   ScienceOn
18 J. M. Findlay and V. Brown, "Eye scanning of multi-element displays: I. Scanpath planning", Vision Research, 46, 1-2, pp. 179-195, 2006.   DOI   ScienceOn
19 R. Pieters, E. Rosbergen, and M. Wedel, "Visual attention to repeated print advertising: A test of scanpath theory", Journal of Marketing Research, 36, 4, pp. 424-438, 1999. DOI: http://dx.doi.org/10.2307/3151998   DOI   ScienceOn
20 D. Noton and L. Stark, "Scanpaths in saccadic eye movements while viewing and recognizing patterns", Vision Research, 11, 9, pp. 929-932, 1971. DOI: http://dx.doi.org/10.1016/0042-6989(71)90213-6   DOI   ScienceOn
21 J. H. Goldberg and J. I. Helfman. "Visual scanpath representation". Proc. of Eye Tracking Research and Applications Symposium (ETRA), 2010.
22 C. Holland, O. Komogortsev, and D. Tamir. "Identifying usability issues via algorithmic detection of excessive visual search". Proc. of Conference on Human Factors in Computing Systems.2012.
23 L. W. Stark and Y. S. Choi, Experimental metaphysics: The scanpath as an epistemological mechanism. pp. 3-69. 1996.
24 Y. S. Choi, A. D. Mosley, and L. W. Stark, "STRING EDITING ANALYSIS OF HUMAN VISUAL-SEARCH", Optometry and Vision Science, 72, 7, pp. 439-451, 1995. DOI: http://dx.doi.org/10.1097/00006324-199507000-00003   DOI
25 A. T. Duchowski, J. Driver, S. Jolaoso, W. Tan, B. N. Ramey, and A. Robbins, Scanpath comparison revisited, Proc. of the 2010 Symposium on Eye-Tracking Research & Applications(ACM), pp. 219-226. 2010.
26 G. Lindgaard and C. Dudek, User Satisfaction, Aesthetics and Usability: Beyond Reductionism, Proc. of the IFIP 17th World Computer Congress-TC13 Stream on Usability: Gaining a Competitive Edge, pp. 231-246. 2002.
27 G. Lindgaard, G. Fernandes, C. Dudek, and J. Brown, "Attention web designers: you have 50 milliseconds to make a good first impression!", Behaviour & Information Technology, 25, 2, pp. 115-126, 2006. DOI: http://dx.doi.org/10.1080/01449290500330448   DOI   ScienceOn
28 A. Campbell and S. Pisterman, "A Fitting Approach to Interactive Service Design: THE IMPORTANCE OF EMOTIONAL NEEDS", Design Management Journal (Former Series), 7, 4, pp. 10-14, 1996.
29 W. Schneider, A. Eschman, and A. Zuccolotto, E-Prime User's Guide, Pittsburgh: Psychology Software Tools, Inc. 2002.
30 A. VoBkuhler. OGAMA Description (for Version 2.5). Available from: http://www.ogama.net/sites/default/files/pdf/OGAMA-De scriptionV25.pdf,(accessed Dec, 27, 2012).
31 A. VoBkuhler, V. Nordmeier, L. Kuchinke, and A. Jacobs, "OGAMA (Open Gaze and Mouse Analyzer): Open-source software designed to analyze eye and mouse movements in slideshow study designs", Behavior Research Methods, 40, 4, pp. 1150-1162, 2008. DOI: http://dx.doi.org/10.3758/BRM.40.4.1150   DOI   ScienceOn
32 J. H. An, "A Study on the User's Evaluation Factors of a Broadcasting Company Web Site", Korean journal of journalism & communication studies, 47, 1, pp. 191-215, 2003.
33 J. Nielsen. Thinking Aloud: The #1 Usability Tool. Alertbox: Current Issues in Web Usability 2012, Available from: http://www.useit.com/alertbox/thinking-aloud-tests.html, (accessed Nov, 12, 2012).
34 J. E. Kwahk and H. W. Kwak, "Usability of Internet Search Interfaces: Establishing Design Guidelines based on Expert Review and User Testing Methods", The Korean journal of experimental and cognitive and biological psychology, 14, 4, pp. 463-483, 2002.
35 J. Brooke, SUS: A quick and dirty usability scale, in Usability evaluation in industry, Edited by P. W. Jordan, B. Weerdmeester, A. Thomas, and I. L. McLelland, Taylor and Francis. 1996.