Browse > Article
http://dx.doi.org/10.3837/tiis.2019.06.002

Applying and Evaluating Visualization Design Guidelines for a MOOC Dashboard to Facilitate Self-Regulated Learning Based on Learning Analytics  

Cha, Hyun-Jin (School of General Education, Dankook University)
Park, Taejung (College of Liberal Arts and Interdisciplinary Studies, Kyonggi University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.13, no.6, 2019 , pp. 2799-2823 More about this Journal
Abstract
With the help of learning analytics, MOOCs have wider potential to succeed in learning through promoting self-regulated learning (SRL). The current study aims to apply and validate visualization design guidelines for a MOOC dashboard to enhance such SRL capabilities based on learning analytics. To achieve the research objective, a MOOC dashboard prototype, LM-Dashboard, was designed and developed, reflecting the visualization design guidelines to promote SRL. Then, both expert and learner participants evaluated LM-Dashboard through iterations to validate the visualization design guidelines and perceived SRL effectiveness. The results of expert and learner evaluations indicated that most of the visualization design guidelines on LM-Dashboard were valid and some perceived SRL aspects such as monitoring a student's learning progress and assessing their achievements with time management were beneficial. However, some features on LM-Dashboard should be improved to enhance SRL aspects related to achieving their learning goals with persistence. The findings suggest that it is necessary to offer appropriate feedback or tips as well as to visualize learner behaviors and activities in an intuitive and efficient way for the successful cycle of SRL. Consequently, this study contributes to establishing a basis for the visual design of a MOOC dashboard for optimizing each learner's SRL.
Keywords
MOOCs; Self-Regulated Learning; Learning Analytics; Design Guidelines;
Citations & Related Records
연도 인용수 순위
  • Reference
1 L. Ali, M. Hatala, D. Gasevic, and J. Jovanovic, "A qualitative evaluation of evolution of a learning analytics tool," Computers & Education, vol.58, no.1, pp.470-489, 2012.   DOI
2 SAM DashBoard S/W. https://www.samlearning.com
3 FutureLearn dashboard. https://www.futurelearn.com
4 F. Grunewald, C. Meinel, M. Totschnig, and C. Willems, "Designing MOOCs for the support of multiple learning styles," in Proc. of European Conference on Technology Enhanced Learning, pp. 371-382, Springer, Berlin, Heidelberg, September, 2013.
5 B. Rienties, A. Boroowa, S. Cross, C. Kubiak, K. Mayles, and S. Murphy, "Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK," Journal of Interactive Media in Education, vol.2016, no.1, pp.2, 2016.   DOI
6 S. B. Shum, and R. Ferguson, "Social learning analytics : five approaches," in Proc. of the 2nd International Conference on Learning Analytics and Knowledge, pp. 23-33, 2012.
7 S. Knight, and K. Littleton, "Discourse-centric learning analytics: mapping the terrain," Journal of Learning Analytics, vol.2, no.1, pp.185-209, 2015.   DOI
8 K. Verbert, S. Govaerts, E. Duval, J. L. Santos, F. Van Assche, G. Parra, and J. Klerkx, "Learning dashboards: an overview and future research opportunities," Personal and Ubiquitous Computing, vol.18, no.6, pp.1499-1514, 2014.   DOI
9 Y. Ishikawa, and M. Hasegawa, "T-scroll: Visualizing trends in a time-series of documents for interactive user exploration," in Proc. of International Conference on Theory and Practice of Digital Libraries, pp. 235-246, Springer, Berlin, Heidelberg, September, 2007.
10 J. Nielsen, "Why you only need to test with 5 users," Nielsen Norman Group: World leaders in research-based user experience, 2000.
11 S. Few, "Information dashboard design: The effective visual communication of data. Sebastopol," CA: O'Reilly Media, Inc., 2006.
12 E. Panadero, "A review of self-regulated learning: six models and four directions for research," Frontiers in psychology, vol.8, p. 422, 2017.   DOI
13 A. R Artino, and K. D. Jones, "Exploring the complex relations between achievement emotions and self-regulated behaviors in online learning," The Internet and Higher Education, vol.15, no.3, pp. 170-175, 2012.   DOI
14 A. Ben-Eliyahu, and L. Linnenbrink-Garcia, "Extending self-regulated learning to include self-regulated emotion strategies," Motivation and Emotion, vol.37, no.3, pp.558-573, 2013.   DOI
15 J. Knox, "From MOOCs to Learning Analytics: Scratching the surface of the 'visual'," eLearn, November, 2014.
16 L. Lupu, E. C. Corbu, and E. Edelhauser, "Dashboards and Radar Charts, Performance Analytics Instruments in Higher Education," in Proc. of International Conference on Current Economic Trends in Emerging and Developing Countries (TIMTED-2017), Timisoara, May, 2017.
17 D. Henriksen, C. Richardson, and R. Mehta, "Design thinking: A creative approach to educational problems of practice," Thinking Skills and Creativity, vol.26, pp.140-153, 2017.   DOI
18 S. Patton, "Admissions professionals ask: Are graduate schools ready for MOOCs?," The Chronicle of Higher Education, APRIL 26, 2013.
19 D. Gasevic, S. Dawson, and G. Siemens, "Let's not forget: Learning analytics are about learning," TechTrends, vol.59, no.1, pp.64-71, 2015.
20 J. L. Santos, S. Govaerts, K. Verbert, and E. Duval, "Goal-oriented visualizations of activity tracking: a case study with engineering students," in Proc. of the 2nd International Conference on Learning Analytics and Knowledge, pp.143-152, Vancouver, Canada, April 29 - May 02, 2012.
21 D. Davis, I. Jivet, R. F. Kizilcec, G. Chen, C. Hauff, and G. J. Houben, "Follow the successful crowd: raising MOOC completion rates through social comparison at scale," in Proc. of the Seventh International Learning Analytics & Knowledge Conference, pp.454-463, ACM, March, 2017.
22 D. F. Onah, J. E. Sinclair, and R. Boyatt, "Exploring the use of MOOC discussion forums," in Proc. of London International Conference on Education, pp.1-4, November, 2014.
23 D. M. Anderson, and S. Staub, "Postgraduate digital badges in higher education: Transforming advanced programs using authentic online instruction and assessment to meet the demands of a global marketplace," Procedia-Social and Behavioral Sciences, vol.195, pp.18-23, 2015.   DOI
24 P. Fain, "Badging From Within," Changing Student Pathways Washington DC: Inside Higher Ed. 2014.
25 D. K. Mah, "Learning analytics and digital badges: potential impact on student retention in higher education," Technology, Knowledge and Learning, vol.21, no.3, pp. 285-305, 2016.   DOI
26 A. Littlejohn, and C. Milligan, "Designing MOOCs for professional learners: Tools and patterns to encourage self-regulated learning," eLearning, vol.42, no.4, pp.1-10, 2015.
27 S. Halawa, D. Greene, and J. Mitchell, "Dropout prediction in MOOCs using learner activity features," Experiences and best practices in and around MOOCs, vol.7, pp.3-12, 2014.
28 K. Jordan, "MOOC completion rates: the data," 2013.
29 R. Rivard, "Measuring the MOOC dropout rate," Inside Higher Ed, vol.8, 2013.
30 T. Park, H. Cha, and G. Lee, "A study on design guidelines of learning analytics to facilitate self-regulated learning in MOOCs," Educational Technology International, vol.17, no.1, pp.1-34.
31 R. F. Kizilcec, and E. Schneider, "Motivation as a lens to understand online learners: Toward data-driven design with the OLEI scale," ACM Transactions on Computer-Human Interaction (TOCHI), vol.22, no.2, p.6, 2015.
32 R. A. Kuiper, and D. J. Pesut, "Promoting cognitive and metacognitive reflective reasoning skills in nursing practice: self-regulated learning theory," Journal of Advanced Nursing, vol.45, no.4, PP.381-391, 2004.   DOI
33 B. J. Zimmerman, and M. M. Pons, "Development of a structured interview for assessing student use of self-regulated learning strategies," American educational research journal, vol.23, no.4, pp. 614-628, 1986.   DOI
34 B. J. Zimmerman, and M. Campillo, "Motivating self-regulated problem solvers," The psychology of problem solving, pp.233-262, 2003.
35 M. C. English, and A. Kitsantas, "Supporting student self-regulated learning in problem-and project-based learning," Interdisciplinary journal of problem-based learning, vol.7, no.2, p.6, 2013.
36 X. Lin, and J. D. Lehman, "Supporting learning of variable control in a computer-based biology environment: Effects of prompting college students to reflect on their own thinking," Journal of research in science teaching, vol.36, no.7, pp.837-858, 1999.   DOI
37 R. Azevedo, D. C. Moos, J. A. Greene, F. I. Winters, and J. G. Cromley, "Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia?," Educational Technology Research and Development, vol.56, no.1, pp.45-72, 2008.   DOI
38 M. Bannert, and C. Mengelkamp, "Scaffolding hypermedia learning through metacognitive prompts," International handbook of metacognition and learning technologies, pp. 171-186, Springer, New York, NY, 2013.
39 J. Broadbent, and W. L. Poon, "Self-regulated learning strategies & academic achievement in online higher education learning environments: A systematic review," The Internet and Higher Education, vol.27, pp.1-13, 2015.   DOI
40 S. Zheng, M. B. Rosson, P. C. Shih, and J. M. Carroll, "Understanding student motivation, behaviors and perceptions in MOOCs," in Proc. of the 18th ACM conference on computer supported cooperative work & social computing, pp.1882-1895, February, 2015.
41 I. de Waard, M. S. Gallagher, R. Zelezny-Green, L. Czerniewicz, S. Downes, A. Kukulska-Hulme, and J. Willems, "Challenges for conceptualising EU MOOC for vulnerable learner groups," in Proc. of the European MOOC Stakeholder Summit 2014, pp.33-42, 2014.
42 B. J. Zimmerman, "Becoming a self-regulated learner: An overview," Theory into practice, vol.41, no.2, pp.64-70, 2002.   DOI
43 H. Qu, and Q. Chen, "Visual analytics for MOOC data," IEEE computer graphics and applications, vol.35, no.6, pp.69-75, 2015.   DOI
44 I. Frolov, and S. Johansson, "An adaptable usability checklist for MOOCs: A usability evaluation instrument for Massive Open Online Course," Master Thesis, Department of Informatics, HCI, UMEA, 2013.
45 S. Downes, "The quality of Massive Open Online Courses," International Handbook of E-learning, vol.1, pp.65-77, 2013.
46 A. Creelman, U. Ehlers, and E. Ossiannilsson, "Perspectives on MOOC quality-An account of the EFQUEL MOOC Quality Project," INNOQUAL-International Journal for Innovation and Quality in Learning, vol.2, no.3, pp.78-87, 2014.
47 A. Ho, I. Chuang, J. Reich, C. A. Coleman, J. Whitehill, C. G. Northcutt, J. J. Williams, J. D. Hansen, G. Lopez, and R. Petersen, "HarvardX and MITx: Two years of Open Online Courses Fall 2012-Summer 2014," March, 2015.
48 M. Boekaerts, "Self-regulated learning: where we are today," International Journal of Educational Research, vol.31, pp.445-457, 1999.   DOI
49 P. R. Pintrich, "The role of goal orientation in self-regulated learning," Handbook of self-regulation, pp.451-502, 2000.
50 M. Taub, R. Azevedo, F. Bouchet, and B. Khosravifar, "Can the use of cognitive and metacognitive self-regulated learning strategies be predicted by learners' levels of prior knowledge in hypermedia-learning environments?," Computers in Human Behavior, vol.39, pp.356-367, 2014.   DOI
51 R. F. Kizilcec, M. Perez-Sanagustin, and J. J. Maldonado, "Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses," Computers & Education, vol.104, pp. 18-33, 2017.   DOI
52 T. Anderson (Ed.), "The theory and practice of online learning," Athabasca University Press, 2008.
53 A. Croll, and S. Power, "Complete web monitoring: watching your visitors, performance, communities, and competitors," O'Reilly Media, Inc., 2009.
54 T. Hey, and A. E. Trefethen, "Cyberinfrastructure for e-Science," Science, vol.308, no. 5723, pp.817-821, 2005.   DOI
55 C. Romero, and S. Ventura, "Educational data mining: A survey from 1995 to 2005," Expert systems with applications, vol.33, no.1, pp.135-146, 2007.   DOI
56 J. Schaffer, B. Huynh, J. O'Donovan, T. Hollerer, Y. Xia, and S. Lin, "An analysis of student behavior in two massive open online courses," in Proc. of 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pp. 380-385, August, 2016.
57 G. Chen, D. Davis, J. Lin, C. Hauff, and G. J. Houben, "Beyond the MOOC platform: gaining insights about learners from the social web," in Proc. of the 8th ACM Conference on Web Science, pp. 15-24, ACM, May, 2016.
58 E. Duval, "Attention please!: learning analytics for visualization and recommendation," in Proc. of the 1st international conference on learning analytics and knowledge, pp. 9-17, February, 2011.