DOI QR코드

DOI QR Code

A Content Analysis for Website Usefulness Evaluation: Utilizing Text Mining Technique

  • Received : 2015.04.30
  • Accepted : 2015.07.27
  • Published : 2015.08.31

Abstract

With the increasing influence of online media, company websites have become important communication channels between companies and customers. Companies use their websites as a marketing tool for a variety of purposes, including enhancing their image and selling products or services. Many researchers have examined the criteria, methods, and tools for website evaluation, but most have focused on usability. Prior content analyses have focused not on text content but on website components, an approach likely to produce subjective evaluations. This study attempts to objectively evaluate company websites by utilizing text mining. We analyze the usefulness of company websites by presenting visualized outputs from a business perspective, allowing practitioners to easily understand the results of the website evaluation and use them in decision making. To demonstrate our method empirically, we selected a company with a number of affiliates in Korea and analyzed the text content of their websites to assess their usefulness using natural language processing and graphics packages in R. Practitioners can easily employ our objective evaluation method, and researchers can use it to gain a new perspective on website evaluation.

Keywords

References

  1. S. Y. Chen, and R. D. Macredie, "The assessment of usability of electronic shopping: A heuristic evaluation," International Journal of Information Management, Vol. 25, No. 6, pp. 516-532, 2005. http://dx.doi.org/10.1016/j.ijinfomgt.2005.08.008
  2. S. S. Robbins, and A. C. Stylianou, "Global corporate web sites: an empirical investigation of content and design", Information & Management, Vol. 40, No.3, pp. 205-212, 2003. http://dx.doi.org/10.1016/s0378-7206(02)00002-2
  3. T. W. Gu, S. J. Hong, and K. M. Lee, "Design and Implementation of Personal Preference Module on Web Service." Journal of Internet Computing and Services (JICS) Vol. 10, No. 4, pp. 161-176. 2009. http://www.dbpia.co.kr/Article/1070615
  4. A. M. Aladwani, and P. C. Palvia, "Developing and validating an instrument for measuring user-perceived web quality", Information & Management, Vol. 39, No. 6, pp. 467-476, 2002. http://dx.doi.org/10.1016/s0378-7206(01)00113-6
  5. W. C. Chiou, C. C. Lin, and C. Perng, "A strategic framework for website evaluation based on a review of the literature from 1995-2006", Information & Management, Vol. 47, No.5-6, pp. 282-290, 2010. http://dx.doi.org/10.1016/j.im.2010.06.002
  6. W. Wang, and Y. Zhou, "E-business Websites Evaluation Based on Opinion Mining", In Proc. of International Conference on Electronic Commerce and Business Intelligence, pp. 87-90, 2009. http://dx.doi.org/10.1109/ecbi.2009.93
  7. P. Zhang, and G. M. Von Dran, "Satisfiers and Dissatisfiers : A Two-Factor Model for website design and evaluation", Journal of the American Society for Information Science, Vol. 51, No. 14, pp. 1253-1268, 2000. http://dx.doi.org/10.1002/1097-4571(2000)9999:9999<::aid-asi1039>3.0.co;2-o
  8. A. Fernandez, E. Insfran, and S. Abrahao, "Usability evaluation methods for the web: A systematic mapping study", Information and Software Technology, Vol. 53, No. 8, pp. 789-817, 2011. http://dx.doi.org/10.1016/j.infsof.2011.02.007
  9. L. Huang, and M. Li, "Research on C2C E-Commerce Website Usability Evaluation System", In Proc. of IEEE 11th International Conference, Computer-Aided Industrial Design & Conceptual Design (CAIDCD), pp. 1371-1374, 2010. http://dx.doi.org/10.1109/caidcd.2010.5681954
  10. H. Lee, Y. Chae, and B. H. Hong, "Comparative Analysis of Web-based Service Quality of Leading Companies in Korea, U.S., Japan and China", Journal of Internet Computing and Services (JICS), Vol. 6, No. 127-139, http://www.dbpia.co.kr/Article/6674032005.
  11. S. Baloglu, and Y. A. Pekcan, "The website design and Internet site marketing practices of upscale and luxury hotels in Turkey", Tourism Management, Vol. 27, No. 1, pp. 171-176, 2006. http://dx.doi.org/10.1016/j.tourman.2004.07.003
  12. W. Tan, D. Liu, and R. Bishu, "Web evaluation: Heuristic evaluation vs. user testing", International Journal of Industrial Ergonomics, Vol. 39, No. 4, pp. 621-627, 2009. http://dx.doi.org/10.1016/j.ergon.2008.02.012
  13. ISO FDIS 9241-210, Ergonomics of human system interaction-Part 210: Human-centered design for interactive systems (formerly known as 13407). International Organization for Standardization (ISO), 2008.
  14. J. Kim, "Introduction to HCI", 2012 http://agbook.co.kr/book/1311/
  15. M. Petre, S. Minocha, and D. Roberts, "Usability beyond the website: An empirically-grounded e-commerce evaluation instrument for the total customer experience", Behaviour & Information Technology, Vol. 25, No. 2, pp.189-203, 2006. http://dx.doi.org/10.1080/01449290500331198
  16. A. Sivaji, and S. S. Tzuaan, "Website user experience (UX) testing tool development using Open Source Software (OSS)", In Proc. of Southeast Asian Network of Ergonomics Societies Conference (SEANES), pp. 1-6, 2012. http://dx.doi.org/10.1109/seanes.2012.6299576
  17. A.P.O.S. Vermeeren, E. L. Law, and V. Roto, M. Obrist, J. Hoonhout, K. Vaananen-Vainio-Mattila, "User Experience Evaluation Methods : Current State and Development Needs", In Proc. of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries, pp. 521-530, 2010. http://dx.doi.org/10.1145/1868914.1868973
  18. F. Stefano, S. Borsci, and G. Stamerra, "Web usability evaluation with screen reader users: implementation of the partial concurrent thinking aloud technique", Cognitive Processing, Vol. 11, No. 3, pp. 263-72, 2010. http://dx.doi.org/10.1007/s10339-009-0347-y
  19. H. Petrie, and O. Kheir, "The relationship between accessibility and usability of websites", In Proc. of the SIGCHI conference on Human factors in computing systems, pp. 397-406, 2007. http://dx.doi.org/10.1145/1240624.1240688
  20. Y. Kim and S. R. Jeong, "A Web Accessability Compliance Framework for Website Development: A Case of W Bank Internet Banking Project." Journal of Internet Computing and Services (JICS), Vol. 14, No. 5, pp. 87-99, 2013. http://dx.doi.org/10.7472/jksii.2013.14.5.87
  21. ISO/IEC 9241-11: Ergonomic Requirements for Office work with Visual Display Terminals (VDTs)-Part 11: Guidance on Usability, International Organization for Standardization (ISO), 1998.
  22. J. S. Dumas, and J. Redish, "A practical guide to usability testing", 1999. http://www.jedbrubaker.com/wp-content/uploads/2013/03/Dumas-99.pdf
  23. B. Shneiderman, "Universal usability", Communications of the ACM, Vol. 43, No. 5, pp. 84-91, 2000. http://dx.doi.org/10.1145/332833.332843
  24. T. K. Chiew, and S. S. Salim, "Webuse: website usability evaluation tool", Malaysian Journal of Computer Science, Vol. 16, No. 1, pp. 47-57, 2003. http://majlis.fsktm.um.edu.my/document.aspx?FileName=199.pdf&origin=publicationDetail
  25. S. Cebi, "A quality evaluation model for the design quality of online shopping websites", Electronic Commerce Research and Applications, Vol. 12, No. 2, pp. 124-135, 2003. http://dx.doi.org/10.1016/j.elerap.2012.12.001
  26. E. H. Chi, "Improving Web Usability Through Visualization", Internet Computing, IEEE, Vol. 6, No. 2, pp. 64-71, 2002. http://dx.doi.org/10.1109/4236.991445
  27. S. Elling, L. Lentz, M. de Jong, and H. van den Bergh, "Measuring the quality of governmental websites in a controlled versus an online setting with the 'Website Evaluation Questionnaire'", Government Information Quarterly, Vol. 29, No. 3, pp. 383-393, 2012. http://dx.doi.org/10.1016/j.giq.2011.11.004
  28. V. Moustakis, C. Litos, A. Dalivigas, and L. Tsironis, "WEBSITE QUALITY ASSESSMENT CRITERIA", In Proc. of the Ninth International Conference on Information Quality, pp. 59-73, 2004. https://www.researchgate.net/profile/Vassilis_Moustakis/publication/220918684_Website_Quality_Assessment_Criteria/links/0c96051af8db13013c000000.pdf
  29. Perdue, R. R, "Internet Site Evaluations : The Influence of Behavioral Experience, Existing Images, and Selected Website Characteristics", Journal of Travel & Tourism, Vol. 11 No. 2-3, pp. 21-38, 2002. http://dx.doi.org/10.1300/j073v11n02_02
  30. L. Manzari, and J. Trinidad-Christensen, "User-Centered Design of a Web Site for Library and Information Science Students: Heuristic Evaluation and Usability Testing", Information Technology and Libraries, Vol. 25, No. 3, pp.163-169, 2013. http://dx.doi.org/10.6017/ital.v25i3.3348
  31. X. Dreze, and F. Zufryden, "Testing Web Site Design and Promotional Content", Journal of Advertising Research, Vol. 37, No. 2, pp. 77-91, 1998. http://www.xdreze.org/Publications/jar7.pdf
  32. N. Avouris, N. Tselios, C. Fidas, and E. Papachristos, "Website Evaluation : A Usability-Based Perspective", Advances in Informatics, pp. 217-231, 2003. http://dx.doi.org/10.1007/3-540-38076-0_15
  33. R. Govers, and F. M. Go, "PROJECTED DESTINATION IMAGE ONLINE : WEBSITE CONTENT ANALYSIS OF PICTURES AND TEXT", Information Technology and Tourism, Vol. 7, No.2, pp. 73-89, 2004. http://dx.doi.org/10.3727/1098305054517327
  34. S. J. Simon, "The impact of culture and gender on web sites: an empirical study", ACM SIGMIS Database, Vol. 32, No. 1, pp. 18-37, 2000. http://dx.doi.org/10.1145/506740.506744
  35. D. Cyr, M. Head, and H. Larios, "Colour appeal in website design within and across cultures: A multi-method evaluation", International Journal of Human-Computer Studies, Vol. 68, No. 1-2, pp. 1-21, 2010. http://dx.doi.org/10.1016/j.ijhcs.2009.08.005
  36. M. T. Thielsch, and G. Hirschfeld, "Spatial frequencies in aesthetic website evaluations--explaining how ultra-rapid evaluations are formed", Ergonomics, Vol. 55, No. 7, pp. 731-42, 2012. http://dx.doi.org/10.1080/00140139.2012.665496
  37. M. Y. Ivory, and M. A. Hearst, "Improving Web Site Design", Internet Computing, IEEE, Vol. 6, No. 2, pp. 56-63, 2002. http://dx.doi.org/10.1109/4236.991444
  38. Y. Kim, and S. R. Jeong, "Opinion Mining Methodology for Social Media Analytics," KSII Transactions on Internet and Information Systems (SCIE), (9:1) pp. 391-406. http://dx.doi.org/10.3837/tiis.2015.01.024
  39. A. Sutcliffe, "Assessing the Reliability of Heuristic Evaluation for Website Attractiveness and Usability", In Proc. of the 35th Hawaii International Conference on System Sciences, pp. 1-10, 2002. http://dx.doi.org/10.1109/hicss.2002.994098
  40. R. Agarwal, and V. Venkatesh, "Assessing a Firm's Web Presence : A Heuristic Evaluation Procedure for the Measurement of Usability" Information Systems Research, Vol. 13, No. 2, pp. 168-186, 2002. http://dx.doi.org/10.1287/isre.13.2.168.84
  41. S. Hong, and J. Kim, "Architectural criteria for website evaluation-conceptual framework and empirical validation", Behaviour & Information Technology, Vol. 23, No. 5, pp. 337-357, 2004. http://dx.doi.org/10.1080/01449290410001712753
  42. T. Mavridis, and A. L. Symeonidis, "Semantic analysis of web documents for the generation of optimal content", Engineering Applications of Artificial Intelligence, Vol.35, pp. 114-130, 2014. http://dx.doi.org/10.1016/j.engappai.2014.06.008
  43. Y. Kim, D. Y. Kwon, and S. R. Jeong, "Visualizing the Results of Opinion Mining from Social Media Contents: Case Study of a Noodle Company", Journal of Intelligence and Information Systems, Vol 20, No.4, pp.89-105, 2014. http://dx.doi.org/10.13088/jiis.2014.20.4.89

Cited by

  1. Compatibility of Firm Positioning Strategy and Website Content: Highest R&D Spending Firms in Turkey vol.16, pp.3, 2015, https://doi.org/10.21547/jss.311944