Browse > Article
http://dx.doi.org/10.14352/jkaie.2017.21.6.675

Development and validation of heuristics for usability improvements on web-based diagnostic assessment management system for primary-middle schoolers  

Cha, Hyunjin (ON Education Design Research Institute)
Hwang, YunJa (Center for Innovative Engineering Education, Dankook University)
Publication Information
Journal of The Korean Association of Information Education / v.21, no.6, 2017 , pp. 675-690 More about this Journal
Abstract
Preparing the 4th industrial revolution age, a diagnostic assessment management system which can collect nationwide data has been implemented and utilized for the development of the evidence-based educational policy. This study aims to develop heuristics to help to improve usability of such systems, which are utilized by primary and middle school community including teachers and students in the nationwide. To achieve the objective, previous studies related to the development of heuristics in the educational contexts as well as basic heuristics from Nielsen were reviewed, and usability problems of the web-based diagnostic-supplement system being currently utilized, as a sample system, were analyzed. Then, the first version of heuristics was developed. The developed heuristics were validated and revised through 2 rounds of the delphi methods with 15 experts. This study has an implication with preparing foundations on improving usability of such diagnostic assessment management system utilized by school community at the nationwide in educational contexts.
Keywords
Heuristics; usability; educational diagnostic assessment; usability evaluation tools; web-based assessment system;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C., and Ananthanarayanan, V. (2017). NMC Horizon Report: 2017 Higher Education Edition. Austin, Texas: The New Media Consortium.
2 Ahn, M.L, Noh, S.J., Kim, S.M. (2009). Research paper of Web accessibility.
3 Albion, P. R. (1999). Heuristic evaluation of educational multimedia: From theory to practice. In 16th Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education, ASCILITE, 9-15. Brisbane, Austrlia.
4 Alsumait, A. a., & Al-Osaimi, A. (2010). Usability heuristics evaluation for child e-learning applications. Journal of Software, 5(6).
5 Baker, K., Greenberg, S., Gutwin, C. (2002). Empirical Development of a Heuristic Evaluation Methodology for Shared Workspace Groupware, CSCW'02, November 16-20, 2002, New Orleans, Louisiana, USA.
6 Blecken, A., Marx, W., (2010). Usability Evaluation of a Learning Management System, Proceedings of the 43rd Hawaii International Conference on System Science.
7 Bonham, S.W., Titus, A., Beichner, R.J., Martin, L. Education Research using Web-based Assessment Systems, Journal of Research on Computing in Education, 2000.
8 Cha, H.J., Ahn, M.L. (2009). Cyber Home Learning system diagnosis-prescription system usability evaluation and improvements on learning experience, HCI Conference 2009, 876-883.
9 David, H. R.,, Meyer, A., Strangman, N., & Rappolt, G., Teaching every student in the digital age: Universal Design for Learning, The Association for supervision and curriculum design(ASCD), from http://www.ascd.org/publications/books/101042.aspx
10 Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American psychologist, 49(8), 725-747.   DOI
11 Fluck, A., Pullen, D. and Harper, C. (2009). Case study of a computer based examination system, Australasian Journal of Educational Technology, Vol. 25. No. 4. pp. 509-523
12 Grant, J. S., & Davis, L. L. (1997). Selection and use of content experts for instrument development. Research in Nursing & Health, 20(3), 269-274.   DOI
13 Jeon, S.J., Kim, H.S., (2016). A study on improvement of web-based diagnosis-supplement system for basic academic skills, Journal of The Korean Association of Information Education,, 20(5), 487-498.   DOI
14 Jeong, Y.S., (2008). Digital textbook usability principle development, Journal of Computer Science Education, 11(30), 13-20.
15 Korea Education & Research Information Service (2017). The educational use of IT-Convergent technology in 4th industrial revolution era. KERIS Research Report RR 2016-7.
16 Mtebe, J. S. (2015). Heuristics for Evaluating usability of learning management systems in Africa, IIMC International Information Management Corporation
17 Korea Education & Research Information Service (KERIS). (2016). Whitepaper 2016,.
18 Kwon, K.S., Park, J.S., Goo, C.D.(2014). Development of Agendas by using educational big data: Focused on analysis of SNS, KERIS, Research report KR 2014-10.
19 Lim, C.L,, Song, H.D., Lee, Y.K,, Lee. Y.T. (2009) A study on the developing and applying digital textbook platform usability evaluation tools, Educational Technology Research, 25(4), 125-155.
20 Mtebe, J. S., Kissaka, M. M. (2015), Heuristics for Evaluating Usability of Learning Management Systems in Africa, In IST-Africa Conference 2015, 1-13, IEEE
21 Nielsen, J. (1993). Usability Engineering, Academic Press Limited, London.
22 Nielsen, J. (1995). 10 Usability Heuristics for User Interface Design, Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/
23 Nokelainen, P. (2006). An empirical assessment of pedagogical usability criteria for digital learning material with elementary school students. Educational Technology & Society, 9(2). 178-197.
24 Park, C.H., Kim, G.T., Lee, B.H., Kang, T.H., Jeong, G.H., (2014). A study on analysis of online basic academic skills improvement system status and development of policy plans, KERIS, Research report KR 2014-6
25 P21(Paternership for 21st century learning), Framework for 21st century learning, 2017, from http://www.p21.org/our-work/p21-framework
26 Sim, G., Read, J.C., Cockton, G. (2009). Evidence based Design of Heuristics for Computer Assisted Assessment, IFIP Conference on Human-Computer Interaction, pp. 204-216.
27 Preece, J., Rogers, Y., Sharp, H., (2015). Interaction Design: beyound human-computer interaction, 4th ed. John Wiley & Sons Ltd. West Sussex:UK.
28 Seo, Y.S. (2007). Development of usability evaluation tools for web-based learning site. PhD thesis.
29 Silius, K. & Tervakari, A.-M. (2003). The Usefulness of Web-based Learning Environments. The Evaluation tool into the Portal of Finnish Virtual University. in International Conference on Network Universities and E-Learning. Proceedings of mENU. Valencia, Spain, V. Penarrocha.
30 Sim, G., and Read, J. C., (2015). Using computer-assisted assessment heuristics for usability evaluations, British journal of educational technology, 47(4).
31 Song, H.D., & Park, H.J. (2009). Exploring factors that affect the usability of digital textbook based on affordance perspectives, Korean society for educational technology, 25(3), 135-155.
32 Squires, D. and J. Preece, (1999). Predicting quality in educational software: Evaluating for learning, usability and the synergy between them. Interacting with Computers, Vol. 11. pp. 467-483.   DOI
33 W3C. Web Content Accessibility Guidelines, 2017 (WCAG) 2.0, from https://www.w3.org/TR/WCAG20/
34 Zhenming, Y., Liang, Z., Guohua, Z. (2003). A Novel Web-based Online Examinations System for Computer Science Education, 33rd ASEE/IEEE Frontiers in Education Conference, November 5-8.