Browse > Article
http://dx.doi.org/10.6109/jkiice.2021.25.11.1696

Development on Korean Visualization Literacy Assessment Test(K-VLAT) and Research Trend Analysis  

Kim, Ha-Neul (Department of IT Convergence, Dong-eui University)
Kim, Sung-Hee (Department of Industrial ICT Technology, Dong-eui University)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.25, no.11, 2021 , pp. 1696-1707 More about this Journal
Abstract
With the recent growth of information technology, various literacy such as digital literacy, data literacy, AI literacy is being studied. In this paper, we focus on data visualization literacy as visualization is an essential part of big data analysis and is used in several mobile apps. Visualization Literacy Assessment Test(VLAT) was developed in 2016 and we introduce how the test was developed and modified to a Korean version, K-VLAT. K-VLAT is consisted of 12 visualizations and 53 questions through a website. Additionally, to understand the research trend in visualization literacy we analyzed 81 papers that had cited the VLAT publication. We categorized the research into 4 categories with 11 sub-categories. The area of studies visualization literacy related to was understanding the relation with cognition, expanding the literacy measures, relation with education, utilization for developing user-centric dashboards or using the test to show effectiveness of visualizations. At last, we discuss about different ways to utilize K-VLAT for future research.
Keywords
Data visualization; Visualization literacy; Assessment test; Education;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. J. Kim, "A Study on the Educational Method of Data Visualization Literacy for Elementary School Students," Korea Institute of Design Research Society, vol. 6, no. 1, pp. 294-305, 2021.   DOI
2 E. M. Peck, S. E. Ayuso, and O. El-Etr, "Data is personal: Attitudes and perceptions of data visualization in rural pennsylvani," in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, no. 244, pp. 1-12, 2019.
3 S. Lee, B. C. Kwon, J. Yang, B. C. Lee, and S. H. Kim. (2019). The correlation between users' cognitive characteristics and visualization literacy. Applied Sciences [Internet]. 9(3), pp. 488. Available: https://doi.org/10.3390/app9030488.   DOI
4 P. C. Parsons, "Conceptual metaphor theory as a foundation for communicative visualization design," in IEEE VIS Workshop on Visualization for Communication, 2018.
5 Z. Liu, R. J. Crouser, and A. Ottley, "Survey on individual differences in visualization," in Computer Graphics Forum, vol. 39, no. 3, pp. 693-712, Jun. 2020.
6 K. Wu, S. Tanis, and D. Szafir, "Designing communicative visualization for people with intellectual developmental disabilities," 2019.
7 S. Campbell, "Feeling numbers the rhetoric of pathos in visualization," Doctoral dissertation, Northeastern University, 2018.
8 F. Gutierrez, X. Ochoa, K. Seipp, T. Broos, and K. Verbert, "Benefits and Trade-Offs of Different Model Representations in Decision Support Systems for Non-expert Users," in IFIP Conference on Human- Computer Interaction, pp. 576-597, Sep. 2019.
9 A. M. B. Rodrigues, G. D. J. Barbosa, H. C. V. Lopes, and S. D. J. Barbosa, "What questions reveal about novices' attempts to make sense of data visualizations: Patterns and misconceptions," Computers & Graphics, vol. 94, pp. 32-42, 2021.   DOI
10 S. Campbell and D. Offenhuber, "Feeling numbers: The emotional impact of proximity techniques in visualization," Information Design Journal, vol. 25, no. 1, pp. 71-86, 2019.   DOI
11 D. Long and B. Magerko, "What is AI Literacy? Competencies and Design Considerations," CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1-16, Apr. 2020.
12 Y. Eshet, "Digital Literacy: A Conceptual Framework for Survival Skills in the Digital Era," Journal of Educational Multimedia and Hypermedia, vol. 13, no. 1, pp. 93-106, 2004.
13 J. Choi, C. Oh, B. Suh, and N. W. Kim, "Toward a Unified Framework for Visualization Design Guidelines," Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1-7, May. 2021.
14 N. Yusop, M. M. Derus, and N. A. Bakar, "Technical Skills in Quantity Surveying and Relevant Practices: Discipline Standards," International Journal of AcademicResearch in Business and Social Sciences, pp. 1863-1873, 2018.
15 B. Copeland, C. Griffin, C. E. Bianco, N. Kononenko, and W. J. Craft, "Data Visualization in Games," Major Qualifying Projects, pp. 7, 2018.
16 S. L'Yi, Y. Chang, D. Shin, and J. Seo, "Toward understanding representation methods in visualization recommendations through scatterplot construction tasks," in Computer Graphics Forum, vol. 38, no. 3, pp. 201-211, Jun. 2019.
17 A. Locoro, W. P. Fisher, and L. Mari, "Visual Information Literacy: Definition, Construct Modeling and Assessment," IEEE access, vol. 9, pp. 71053-71071, 2021.   DOI
18 O. Ola, "The Design of Interactive Visualizations and Analytics for Public Health Datam," Electronic Thesis and Dissertation Repository, 2017.
19 K. Wu, E. Petersen, T. Ahmad, D. Burlinson, S. Tanis, and D. A. Szafir, "Understanding data accessibility for people with intellectual and developmental disabilities," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, no. 606, pp. 1-16, May. 2021.
20 K. Borner, A. Maltese, R. N. Balliet, and J. Heimlich, "Investigating aspects of data visualization literacy using 20 information visualizations and 273 science museum visitors," Information Visualization, vol. 15, no. 3, pp. 1-16, Aug. 2015.
21 Forbes. (2016) [Internet]. Available: https://www.forbes.com/sites/brentdykes/2016/03/31/data-storytelling-the-essential-data-science-skill-everyone-needs/?sh=6d4b7b5052ad.
22 W. Lee, "Overcome the information gap by data literacy in the Big data Era," KISO Jounral, vol. 21, 2015.
23 J. S. Lee, "A Study for Cultivating Data Literacy in the Social Studies Education - Focusing on the Analysis of the Data Visualization Contents of the Social Studies Textbook in the 2015 Curriculum," Teacher Education Research, vol. 58, no. 4, pp. 501-512, 2019.   DOI
24 D. Rees, R. S. Laramee, P. Brookes, T. D'Cruze, G. A. Smith, and A. Miah, "AgentVis: Visual Analysis of Agent Behavior withHierarchical Glyphs," IEEE transactions on visualization and computer graphics, Sep. 2020.
25 P. Tsagaroulis, "Data Visualization, Numeracy and Graph Literacy: Seeing and Thinking of Data Presented as Tables or Charts," Doctoral dissertation, The Chicago School of Professional Psychology, 2020.
26 S. Sultana, S. I. Ahmed, and J. M. Rzeszotarski, "Seeing in Context: Traditional Visual Communication Practices in Rural Bangladesh," Proceedings of the ACM on Human-Computer Interaction, vol. 4, no. 214, pp. 1-31, 2021.
27 H. A. He, J. Walny, S. Thoma, W. J. Willett, and S. Carpendale, "Discussing Open Energy Data and Data Visualizations with Canadians," 2019.
28 H. Nguyen, "Visualizing Hypothetical Outcomes in Quantitative Ethnography," in Second International Conference on Quantitative Ethnography: Conference Proceedings Supplement, p. 11, 2021.
29 M. Hind, "Explaining explainable AI," XRDS: Crossroads, The ACM Magazine for Students, pp. 16-19, 2019.
30 D. Zhang, A. Sarvghad, and G. Miklau, "Investigating Visual Analysis of Differentially Private Data," IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 2, pp. 1786-1796, 2020.
31 M. Keck, D. Kammer, A. Ferreira, A. Giachetti, and R. Groh, "VIDEM 2020: Workshop on Visual Interface Design Methods," in Proceedings of the International Conference on Advanced Visual Interfaces, pp. 1-2, Sep. 2020.
32 A. M. MacEachren, "(re) Considering Bertin in the age of big data and visual analytics," Cartography and Geographic Information Science, vol. 46, no. 2, pp. 101-118, 2019.   DOI
33 F. Chevalier, N. H. Riche, B., Alper, C. Plaisant, J. Boy, and N. Elmqvist, "Observations and reflections on visualization literacy in elementary school," IEEE computer graphics and applications, vol. 38, no. 3, pp. 21-29, 2018.   DOI
34 R. Schnurer, C. Dind, S. Schalcher, P. Tschudi, and L. Hurni, "Augmenting Printed School Atlases with Thematic 3D Maps," Multimodal Technologies and Interaction, 2020.
35 D. Donohoe, and E. Costello, "Data Visualisation Literacy in Higher Education: An Exploratory Study of Understanding of a Learning Dashboard Tool," International Journal of Emerging Technologies in Learning, vol. 15, pp. 115-126, 2020.   DOI
36 K. Kelecsenyi, "Non-Standard Data Visualization Methods in Undergraduate Statistics Education," GRADUS, pp. 54-63, 2019.
37 J. Y. Hong and Y. S. Ki, "Development of AI Data Science Education Program to Foster Data Literacy of Elementary School Student," Journal of The Korean Association of Information Education, vol. 24, no. 6, pp. 633-641, Dec. 2020.   DOI
38 A. Esselman, "A Study of Status Quo Bias in Data Visualizations Used for Decision Making," Wilmington University, Delaware, 2020.
39 X. Shu, A. Wu, J. Tang, B. Bach, Y. Wu, and H. Qu, "What makes a Data-GIF understandable?," IEEE Transactions on Visualization and Computer, vol. 27, no. 2, pp. 1492-1502, 2020.
40 J. Steinhardt, "The role of numeric and statistical content on risk perception in infographics about road safety," Journal of Risk Research, pp. 613-625, 2020.
41 S. R. Bartholomew, T. McGraw, D. Fauber, J. Charlesworth, and J. Weitlauf, "Reducing water waste through data-driven irrigation practices," Technology and Engineering Teacher, vol. 79, no. 4, pp. 21-25, 2019.
42 B. Alper, N. H. Riche, F. Chevalier, J. Boy, and M. Sezgin, "Visualization literacy at elementary school," in Proceedings of the 2017 CHI conference on human factors in computing systems, pp. 5485-5497, May. 2017.
43 K. Borner, A. Bueckle, and M. Ginda, "Data visualization literacy: Definitions, conceptual frameworks, exercises, and assessments," Proceedings of the National Academy of Sciences, pp. 1857-1864, 2019.
44 A. Veldhuis, R. H. Liang, and T. Bekker, "CoDa: Collaborative data interpretation through an interactive tangible scatterplot," in Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction, pp. 323-336, Feb. 2020.
45 S. E. Krejci, S. Ramroop-Butts, H. N. Torres, and R. D. Isokpehi, "Visual literacy intervention for improving undergraduate student critical thinking of global sustainability issues," Sustainability, vol. 12, pp. 10209, 2020.   DOI
46 A. Vazquez-Ingelmo, A. Garcia-Holgado, F. J. Garcia-Penalvo, and R. Theron, "Dashboard meta-model for knowledge management in technological ecosystem: a case study in healthcare," in Multidisciplinary Digital Publishing Institute Proceedings, vol. 31, no. 1, pp. 44, 2019.
47 A. Vazquez-Ingelmo, A. Garcia-Holgado, F. J. Garcia-Penalvo, and R. Theron, "A meta-model integration for supporting knowledge discovery in specific domains: a case study in healthcare," Sensors, pp. 4072, 2020.
48 A. Vazquez-Ingelmo, F. J. Garcia-Penalvo, R. Theron, and M. A. Conde, "Representing data visualization goals and tasks through meta-modeling to tailor information dashboards," Applied Sciences, pp. 2306, 2020.
49 A. Vazquez-Ingelmo, F. J. Garcia-Penalvo, R. Theron, and A. Garcia-Holgado, "Specifying information dashboards' interactive features through meta-model instantiation," in Proceedings of the LASI-SPAIN 2020. Learning Analytics Summer Institute Spain 2020: Learning Analytics, 2020.
50 O. Ola, and K. Sedig, "Health literacy for the general public: Making a case for non-trivial visualizations," in Informatics, Multidisciplinary Digital Publishing Institute, vol. 4, no. 4, pp. 33, Dec. 2017.
51 Y. S. Gugale, "Exploring ecosystems in indiana's education and workforce development using a data visualization dashboard," Doctoral dissertation, Purdue University Graduate School, 2020.
52 N. W. Kim, B. Bach, H. Im, S. Schriber, M. Gross, and H. Pfister, "Visualizing nonlinear narratives with story curves," IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 1, pp. 595-604, 2017.   DOI
53 H. K. Kong, Z. Liu, and K. Karahalios, "Frames and slants in titles of visualizations on controversial topics," in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, no. 438, pp. 1-12, Apr. 2018.
54 A. Denisova, R. Laramee, and E. Firat, "Treemap Literacy: A Classroom-Based Investigation," in Eurographics Proceedings, May. 2020.
55 N. Yusop, M. M. Derus, and I. Samsuddin, "Predictors of Spatial Visualization Ability (SVA) in construction and the quantification course achievements: a conceptual analysis," Malaysian construction Research Journal, vol. 10, no. 2, pp. 63, 2020.
56 C. Stoiber, F. Grassinger, M. Pohl, H. Stitz, M. Streit, and W. Aigner, "Visualization onboarding: Learning how to read and use visualizations," IEEE Workshop on Visualization for Communication, IEEE, Vancouver, BC, Canada, 2019.
57 E. Adar and E. Lee, "Communicative Visualizations as a Learning Problem," IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 2, pp. 946-956, 2020.
58 B. Saket, A. Endert, and C. Demiralp, "Task-based effectiveness of basic visualizations," IEEE transactions on visualization and computer graphics, vol. 25, no. 7, pp. 2505-2512, 2018.   DOI
59 A. Vazquez Ingelmo, F. J. Garcia Penalvo, R. Theron Sanchez, and M. A. C. Gonzalez, "Extending a dashboard meta-model to account for users' characteristics and goals for enhancing personalization," in Proceedings of LASI-SPAIN, no. 2415, pp. 35-42, 2019.
60 A. Vazquez-Ingelmo, F. J. Garcia-Penalvo, and R. Theron, "Capturing high-level requirements of information dashboards' components through meta-modeling," in Proceedings of the Seventh International Conference on Technological Ecosystems for Enhancing Multiculturality, pp. 815-821, Oct. 2019.
61 C. Hague and S. Payton, "Digital Literacy across the curriculum," Curriculum and Leadership Journal, vol. 9, no. 10, Apr. 2011.
62 P. Ruchikachorn, P. Duangklad, and T. Ruangdej, "A Case Study of Data Visualization and Storytelling Workshop for Middle School Students," in Proceedings of PacificVis 2020 Posters, Tianjin, China, 2020.
63 C. Lee, T. Yang, G. D. Inchoco, G. M. Jones, and A. Satyanarayan, "Viral Visualizations: How Coronavirus Skeptics Use Orthodox Data Practices to Promote Unorthodox Science Online," in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1-18, May. 2021.
64 M. Saddiqa, B. Larsen, R. Magnussen, L. L. Rasmussen, and J. M. Pedersen, "Open data visualization in Danish schools: a case study," International Conference in Central Europe on Computer Graphics, 2019.
65 M. Li, "Design Judgments in Information Visualization Design," Doctoral dissertation, Purdue University Graduate School, 2021.
66 S. Lee, "Designing Information Visualization to Support Visual Comparison for Novices," Doctoral dissertation, Seoul University Graduate School, 2020.
67 A. Vazquez-Ingelmo, F. J. Garcia-Penalvo, R. Theron, D. A. Filva, and D. F. Escudero, "Connecting domain-specific features to source code: Towards the automatization of dashboard generation," Cluster Computing, pp. 1803-1816, 2020.
68 H. K. Kong, W. Zhu, Z. Liu, and K. Karahalios, "Understanding visual cues in visualizations accompanied by audio narrations," in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, no. 50, pp. 1-13, May. 2019.
69 S. K. Lee, S. H. Kim, and B. C. Kwon, "VLAT: Development of a Visualization Literacy Assessment Test," IEEE Transactions Visualization and Computer Graphics, vol. 23, pp. 551-560, Jan. 2017.   DOI
70 P. Gilster, Digital Literacy, John Wiley & Sons, Inc., 1997.
71 E. B. Mandinach and E. S. Gummer, "A Systemic View of Implementing Data Literacy in Educator Preparation," Educational Researcher, vol. 42, no. 1, pp. 30-37, Jan. 2013.   DOI
72 M. A. Livingston and D. Brock, "Position: Visual Sentences: Definitions and Applications," 2020.
73 S. J. Park, "AI education perception of pre-service teachers according to AI learning experience, Interest in AI education, and Majo," Journal of The Korean Associationof Information Education, vol. 25, no. 1, pp. 103-111, Feb. 2021.   DOI
74 T. Munzner, "Visualization Analysis and Design," CRC Press, 2014.
75 S. Lee, "Investigation of Visualization Literacy: A Visualization Sensemaking Model, a Visualization Literacy Assessment Test, and the Effects of Cognitive Characteristics," Doctoral dissertation, Purdue University, 2017.
76 A. Burns, C. Xiong, S. Franconeri, A. Cairo, and N. Mahyar, "How to evaluate data visualizations across different levels of understandingn" in 2020 IEEE Workshop on Evaluation and Beyond-Methodological Approaches to Visualization, pp. 19-28, Oct. 2020.
77 K. Bergram and B. Ochan, "Data-driven Biased Decision-making? Exploring the landscape between dashboards, visualization literacy and decision bias," Department of Informatics, 2018.
78 M. Le Goc, C. Perin, S. Follmer, J. D. Fekete, and P. Dragicevic, "Dynamic composite data physicalization using wheeled micro-robots," IEEE transactions on visualization and computer graphics, vol. 25, no. 1, pp. 737-747, 2018.   DOI
79 A. Abdul, C. von der Weth, M. Kankanhalli, and B. Y. Lim, "COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations," in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 1-14, Apr. 2020.
80 A. Ulmer, M. Schufrin, H. Lucke-Tieke, C. D. Kannanayikkal, and J. Kohlhammer, "Towards Visual Cyber Security Analytics for the Masses," EuroVis Workshop on Visual Analytics, pp. 55-59, Jun. 2018.
81 R. Schertler, S. Kriglstein, and G. Wallner, "User guided movement analysis in games using semantic trajectories," in Proceedings of the Annual Symposium on Computer-Human Interaction in Play, pp. 613-623, Oct. 2019.
82 J. Boy, R. Rensink, E. Bertini, and J. D. Fekete, "A principled way of assessing visualization literacy," IEEE Transactions on Visualization and Computer Graphics, vol. 20, no. 12, pp. 1963-1972, Nov. 2014.   DOI
83 O. Lock, T. Bednarz, and C. Pettit, "The visual analytics of big, open public transport data-a framework and pipeline for monitoring system performance in Greater Sydney," Big Earth Data, pp. 134-159, 2021.
84 A. M. B. Rodrigues, G. D. J. Barbosa, R. de Araujo Lima, D. J. F. Braga, H. C. V. Lopes, and S. D. J. Barbosa, "Revisiting Visualization Task Taxonomies: Specifying Functions for the Data Transformations Stage," in International Conference on Human-Computer Interaction, pp. 655-671, Jul. 2020.
85 J. F. Kassel, "On intelligible multimodal visual analysis," Doctoral dissertation, Hannover: Institutionelles Repositorium der Leibniz Universitat Hannover, 2020.
86 M. A. Livingston, D. Brock, J. W. Decker, D. J. Perzanowski, C. Van Dolson, J. Mathews, and A. S. ulushi, "A query generation technique for measuring comprehension of statistical graphics," in International Conference on Applied Human Factors and Ergonomics, vol. 963, pp. 3-14, Jul. 2019.
87 H. Mansoor and L. Harrison, "Data visualization literacy and visualization biases: Cases for merging parallel threads," in Cognitive Biases in Visualizations, pp. 87-96, Jan. 2018.
88 B. Saket, A. Endert, and C. Demiralp, "Data and task based effectiveness of basic visualizations," CoRR, 2017.
89 A. M. Rodrigues, G. D. Barbosa, H. Lopes, and S. D. Barbosa, "Comparing the effectiveness of visualizations of different data distributions," in 2019 32nd SIBGRAPI Conference on Graphics, Patterns and Images, IEEE, pp. 84-91, Oct. 2019.
90 R.Schnurer, M. Ritzi, A. Coltekin, and R. Sieber, "An empirical evaluation of three-dimensional pie charts with individually extruded sectors in a geovisualization context," Information Visualization, vol. 19, no. 3, pp. 183-206, 2020.   DOI
91 B. J. Lee, S. h. Yoo, Y. E. Kim, H. J. Song, J. C. Lee, J. H. Hwang, B. H. Kim, and J. W. Seo, "Interactive Visualization Tool for Effective Evidence Mapping with Medical Research Papers," KIISE Transactions on Computing Practices, vol. 24, no. 12, pp. 662-669, 2018.   DOI
92 A. Denisova, R. Laramee, and E. Firat, "Treemap Literacy: A Classroom-Based Investigation," in Eurographics Proceedings, May. 2020.
93 M. A. Livingston, D. Brock, J. W. Decker, D. J. Perzanowski, C. Van Dolson, J. Matthews, and A. S. Lulushi, "Report on a Query Generation Technique for Measuring Comprehension of Statistical Graphics," Naval Research Lab Washington DC Washington United States, 2019.