Journal of the Korean earth science society (한국지구과학회지)

The Korean Earth Science Society (한국지구과학회)
권호 표지
DOI QR코드

DOI QR Code

The Validation of the Systems Thinking Assessment Tool for Measuring the Higher-order Thinking Ability of Vietnamese High School Students

  • Hyonyong Lee (Department of Earth Science Education, Kyungpook National University) ;
  • Nguyen Thi Thuy (Thuan Hoa High School) ;
  • Hyundong Lee (Department of Science Education, Daegu National University of Education) ;
  • Jaedon Jeon (Science Education Research Institute, Kyungpook National University) ;
  • Byung-Yeol Park (Department of Curriculum and Instruction, Neag School of Education, University of Connecticut)
  • Received : 2023.07.24
  • Accepted : 2023.08.31
  • Published : 2023.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study aimed to verify the validity of a measurement tool for Vietnamese high school students' systems thinking abilities. Two quantitative assessment tools, the Systems Thinking Measuring Instrument (Lee et al., 2013) and the Systems Thinking Scale (Dolansky et al., 2020), were used to measure students' systems thinking after translation into Vietnamese. As a result, it was revealed that Cronbach-α for each tool (i.e., STMI and STS) was .917 and .950, respectively, indicating high reliability for both. To validate the construct validity of the translated questionnaire, exploratory factor analysis was performed using SPSS 26.0, and confirmatory factor analysis was performed using AMOS 21.0. For concurrent validity, correlation analysis using structural equation modeling was performed to validate the translated questionnaire. Exploratory factor analysis revealed that 10 items from the STMI and 12 items from the STS loaded on the intended factors and appropriate factor loading values were obtained. For confirmatory factor analysis, a structural equation model organized with 10 items from the STMI and 12 items from the STS was used. The result of this showed that the convergent validity values of the model were all appropriate, and the model fit indices were analyzed to be χ2/df of 1.892, CFI of .928, TLI of .919, SRMR of .047, and RMSEA of .063, indicating that the model consisting of the 22 items of the two questionnaires was appropriate. Analysis of the concurrent validity of the two tools indicated a high correlation coefficient (.903) and high correlation (.571-.846) among the subfactors. In conclusion, both the STMI and STS are valid quantitative measures of systems thinking, and it can be inferred that the systems thinking of Vietnamese high-school students can be quantitatively measured using the 22 items identified in our analysis. Using the tool validated in this study with other tools (e.g., qualitative assessment) can help accurately measure Vietnamese high school students' systems thinking abilities. Furthermore, these tools can be used to collect evidence and support effective education in ODA projects and volunteer programs.

Keywords

References

  1. Anderson, V. and Johnson, L., 1997, Systems thinking basics. Pegasus Communications. 
  2. Arnold, R. D. and Wade, J. P., 2015, A definition of systems thinking: A systems approach. Procedia Computer Science, 44, 669-678.  https://doi.org/10.1016/j.procs.2015.03.050
  3. Ben-zvi-Assaraf, O. and Orion, N., 2005a, A study of junior high students' perceptions of the water cycle. Journal of Geoscience Education, 53(4), 366-373.  https://doi.org/10.5408/1089-9995-53.4.366
  4. Ben-zvi-Assaraf, O. and Orion, N., 2005b, Development of system thinking skills in the context of Earth system education. Journal of Research in Science Teaching, 42(5), 518-560.  https://doi.org/10.1002/tea.20061
  5. Ben-Zvi-Assaraf, O. and Orion, N., 2009, A design based research of an Earth systems based environmental curriculum. Eurasia Journal of Mathematics, Science & Technology Education, 5(1), 47-62. 
  6. Cabrera, D., Colosi, L., and Lobdell, C., 2008, Systems thinking. Evaluation and Program Planning, 31(3), 299-310.  https://doi.org/10.1016/j.evalprogplan.2007.12.001
  7. Chae, J. E., 2015, Vietnam education and ODA. The Southeast Asian Review, 25(4), 77-124.  https://doi.org/10.21652/kaseas.25.4.201511.77
  8. Cho, O. and Hwang, K., 2016, The effects of simulation-based education on nursing students' presence in education, systems thinking and proactivity in problem solving. Journal of Korean Academic Society of Home Health Care Nursing, 23(2), 147-154. 
  9. Choi, D. and Kim, K., 2015, The development and validation of self-regulation scale for elementary school students. Korean Journal of Elementary Education, 26(4), 59-81.  https://doi.org/10.20972/KJEE.26.4.201512.59
  10. Chung, O., Im, J., Chung, S., Kim, R., and Yoon, J., 2012, The development and validation of a self-directed learning inventory for elementary school students. The Korean Journal of Human Development, 19(4), 227-245. 
  11. Chung, S. and Choi, N., 2018, The validity and norms of reading motivation scale for elementary school students. The Korean Journal Child Education, 27(1), 215-240.  https://doi.org/10.17643/KJCE.2018.27.1.12
  12. Dolansky, M. A., Moore, S. M., Palmieri, P. A., and Singh, M. K., 2020, Development and validation of the systems thinking scale. Journal of General Internal Medicine, 35, 2314-2320.  https://doi.org/10.1007/s11606-020-05830-1
  13. Duc, N. M., Linh, N. Q., and Yuenyong, C., 2019, Implement of STEM education in Vietnamese high school: unit of acid-base reagent from purple cabbage. Journal of Physics: Conference Series, 1340, 012029, 1-9. 
  14. Han, D. and Jo, E., 2015, A study on the exploration of relationship between environmental literacy and systems thinking for sustainability education in social studies. Social Studies Education, 54, 65-83. 
  15. Hung, W., 2008, Enhancing systems thinking skills with modelling. British Journal of Educational Technology, 39(6), 1099-1120.  https://doi.org/10.1111/j.1467-8535.2007.00791.x
  16. Im, Y. and Lee, H., 2014, Development and analysis of effects of writing educational program for improving system thinking ability. Journal of Learner-Centered Curriculum and Instruction, 14, 407-427. 
  17. Jeon, J. and Lee, H., 2015, The development and application of STEAM education program based on systems thinking for high school students. Journal of the Korean Association for Science Education, 35(6), 1007-1018.  https://doi.org/10.14697/jkase.2015.35.6.1007
  18. Jeon, J. D., Lee, H. D., and Lee, H. N., 2022, Development and application of instrument for level scale of the systems thinking ability about carbon cycle. Journal of the Korean Association for Science Education, 42(4), 397-415. 
  19. Kim, H. and Mun, S., 2017, The effect of a system thinking-based steam-type esd program on environmental literacy and system thinking ability of elementary students. Journal of Korean Journal of Environmental Education, 30(1), 85-102.  https://doi.org/10.17965/kjee.2017.30.1.85
  20. Kim, H., Jeong, S., Jeong, S., and Mun, S., 2017, Development of a systems thinking-based steam education program for elementary school students and the effect of its application. School Science Journal, 11(3), 288-301.  https://doi.org/10.15737/SSJ.11.3.201710.288
  21. Lee, H., 2020, An analysis of systems thinking levels of the water cycle in Earth system of primary school preservice teachers. Teacher Education Research, 59(1), 21-38.  https://doi.org/10.15812/TER.59.1.202003.21
  22. Lee, H. and Lee, H., 2017, Analysis and effects of high school students' systems thinking using iceberg (IB) Model. Journal of the Korean Association for Science Education, 37(4), 611-624. 
  23. Lee, H. and Lee, H., 2016, Effects of systems thinking on high school students' science self-efficacy. The Journal of The Korean Earth Science Society, 37(3), 133-145.  https://doi.org/10.5467/JKESS.2016.37.3.133
  24. Lee, H. and Lee, H., 2013, Revalidation of measuring instrument systems thinking and comparison of systems thinking between science and general high school students. Journal of the Korean Association for Science Education, 33(4), 1237-1247.  https://doi.org/10.14697/jkase.2013.33.6.1237
  25. Lee, H., Jeon, J., and Lee, H., 2019, Verification the systems thinking factor structure and comparison of systems thinking based on preferred subjects about elementary school students'. Journal of the Korean Association for Science Education, 39(2), 161-171. 
  26. Lee, H., Jeon, J., and Lee, H., 2018, Development of framework and rubric for measuring students' level of systems thinking. Journal of the Korean Association for Science Education, 38(3), 355-367. 
  27. Lee, H., Kwon, H., Park, K., and Lee, H., 2013, An instrument development and validation for measuring high school students' systems thinking. Journal of the Korean Association for Science Education, 33(3), 995-1006.  https://doi.org/10.14697/jkase.2013.33.5.995
  28. Lee, H., Kwon, Y., Oh, H., and Lee, H., 2011, Development and application of the educational program to increase high school students' systems thinking skills: Focus on global warming. Journal of the Korean Earth Science Society, 32, 784-797.  https://doi.org/10.5467/JKESS.2011.32.7.784
  29. Lee, H. N., Lee, H. D., Kwon, D. K., and Jeon, J. D., 2022, Effect analysis on attitudes towards STEM and career perception according to the development and application of STEM thinking classroom program to improve Math and Science competency of secondary school students. Teacher Education Research, 61(2), 231-246.  https://doi.org/10.15812/TER.61.2.202206.231
  30. Lee, M., 2016, Development and validation of academic resiliency scale for elementary school students (ARSESS). Korean Education Inquiry, 34(4), 21-38.  https://doi.org/10.22327/kei.2016.34.4.021
  31. Linh, N. Q., Duc, N. M., and Yuenyong, C., 2019, Developing critical thinking of students through STEM educational orientation program in Vietnam. Journal of Physics: Conference Series, 1340, 012025, 1-11.  https://doi.org/10.1088/1742-6596/1340/1/012025
  32. Ministry of Education. (2020). Master plan of convergence education (2020-2024), Ministry of Education. 
  33. NGSS Lead States. (2013). Next generation science standards: For states, by states. National Academies Press. 
  34. Nguyen, T. P. L., Nguyen, T. H., and Tran, T. K., 2020, STEM education in secondary schools: Teachers' perspective towards sustainable development. Sustainability, 12(21), 8865, 1-16.  https://doi.org/10.3390/su12218865
  35. O'Connor, J. and McDermott, I., 1997, The art of systems thinking: Essential skills for creativity and problem solving. Thorsons Publishers. 
  36. Park, B. and Lee, H., 2014, Development and application of systems thinking-based STEAM Education Program to improve secondary science gifted and talented students' systems thinking skill. Journal of Gifted/Talented Education, 24(3), 421-444.  https://doi.org/10.9722/JGTE.2014.24.3.421
  37. Park, K., Jeong, H., Jeon, J., and Lee, H., 2018, Development and application of systems thinking-based STEAM education program to improve pre-service science teachers' systems thinking skill. Teacher Education Research, 57(1), 108-128.  https://doi.org/10.15812/ter.57.1.201803.108
  38. Park, K., Lee, H., Lee, H., and Jeon, J., 2019, Analysis of systems thinking level of pre-service teachers about carbon cycle in Earth systems using rubrics of evaluating systems thinking. Journal of the Korean Association for Science Education, 39(5), 599-611. 
  39. Park, Y. and Han, K., 2017, Validation of Korean version of CPAC (cognitive processes associated with creativity) for elementary school students. Journal of Elementary Education, 30(1), 97-119. 
  40. Richmond, B. and Peterson, S., 2001, An introduction to systems thinking. Isee Systems, Inc.
  41. Sanders, M., 2009, STEM, STEM education, STEM mania. Technology Teacher, 68(4), 20-26. 
  42. Senge, P. M., 2012, Schools that learn: A fifth discipline fieldbook for educators, parents, and everyone who cares about education. Doubleday. 
  43. Seong, T., 2011, Statistical package for the social science. Haksisa. 
  44. Song, J., 2011, SPSS/AMOS Statistical analysis. 21 Century Press. 
  45. Tharayil, S., Borrego, M., Prince, M., Nguyen, K. A., Shekhar, P., Finelli, C. J., and Waters, C., 2018, Strategies to mitigate student resistance to active learning. International Journal of STEM Education, 5, 1-16.  https://doi.org/10.1186/s40594-017-0097-9
  46. Tinh, P. T., Duc, N. M., Yuenyong, C., Kieu, N. T., and Nguyen, T. T., 2021, Development of STEM education learning unit in context of Vietnam Tan Cuong Tea village. Journal of Physics: Conference Series,1835, 012060, 1-10.  https://doi.org/10.1088/1742-6596/1835/1/012060
  47. Vuong, Q. H., Pham, T. H., Tran, T., Vuong, T. T., Nguyen, K. L. P., La, V. P., and Ho, M. T., 2020, STEM education and outcomes in Vietnam: Views from the social gap and gender issues. http://dx.doi.org/10.2139/ssrn.3543346