Journal of the Korean earth science society (한국지구과학회지)

The Korean Earth Science Society (한국지구과학회)
권호 표지
DOI QR코드

DOI QR Code

Re-validation of the Revised Systems Thinking Measuring Instrument for Vietnamese High School Students and Comparison of Latent Means between Korean and Vietnamese High School Students

베트남 고등학생을 대상으로 한 개정 시스템 사고 검사 도구 재타당화 및 한국과 베트남 고등학생의 잠재 평균 비교

  • Hyonyong Lee (Department of Earth Science Education, Kyungpook National University) ;
  • Nguyen Thi Thuy (Thuan Hoa High School) ;
  • Byung-Yeol Park (Institute of Integrated Science Education, Dankook University) ;
  • Jaedon Jeon (Science Education Research Institute, Kyungpook National University) ;
  • Hyundong Lee (Department of Science Education, Daegu National University of Education)
  • 이효녕 (경북대학교 지구과학교육과) ;
  • ;
  • 박병열 (단국대학교부설 통합과학교육연구소) ;
  • 전재돈 (경북대학교 과학교육연구소) ;
  • 이현동 (대구교육대학교 과학교육과)
  • Received : 2024.04.04
  • Accepted : 2024.04.23
  • Published : 2024.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

The purposes of this study were: (1) to revalidate the revised Systems Thinking Measuring Instrument (Re_STMI) reported by Lee et al. (2024) among Vietnamese high school students and (2) to investigate the differences in systems thinking abilities between Korean and Vietnamese high school students. To achieve this, data from 234 Vietnamese high school students who responded to translated Re_STMI consisting of 20 items and an Scale consisting of 20 items were used. Validity analysis was conducted through item response analysis (Item Reliability, Item Map, Infit and Outfit MNSQ, DIF between male and female) and exploratory factor analysis (principal axis factor analysis using Promax). Furthermore, structural equation modeling was employed with data from 475 Korean high school students to verify the latent mean analysis. The results were as follows: First, in the item response analysis of the 20 translated Re_STMI items in Vietnamese, the Item Reliability was .97, and the Infit MNSQ ranged from .67 to 1.38. The results from the Item Map and DIF analysis align with previous findings. In the exploratory factor analysis, all items were loaded onto intended sub-factors, with sub-factor reliabilities ranging from .662 to .833 and total reliability at .876. Confirmatory factor analysis for latent mean analysis between Korean and Vietnamese students yielded acceptable model fit indices (χ2/df: 2.830, CFI: .931, TLI: .918, SRMR: .043, RMSEA: .051). Lastly, the latent mean analysis between Korean and Vietnamese students revealed a small effect size in systems analysis, mental models, team learning, and shared vision factors, whereas a medium effect size was observed in personal mastery factors, with Vietnamese high school students showing significantly higher results in systems thinking. This study confirmed the reliability and validity of the Re_STMI items. Furthermore, international comparative studies on systems thinking using Re_STMI translated into Vietnamese, English, and other languages are warranted in the context of students' systems thinking analysis.

이 연구의 목적은 1) 베트남 고등학생들을 대상으로 Lee et al. (2024)에서 보고한 개정 시스템 사고 검사 도구의 타당도를 재검증하고, 2) 연구에 참여한 우리나라 고등학생과 베트남 고등학생 간 시스템 사고 능력에 대한 차이를 알아보는 것이다. 이를 위하여 베트남 고등학생 234명이 베트남어로 번역된 개정 시스템 사고 검사 도구 20문항과 STS 척도 20문항에 응답한 자료를 활용하였다. 타당도 분석은 문항 반응 분석(Item Reliability, Item Map, Infit and Outfit MNSQ, 남녀 집단의 DIF)과 탐색적 요인 분석(프로맥스를 활용한 주축 요인 분석)을 통해 검증하고, 나아가 우리나라 고등학생 475명의 데이터를 함께 활용하여 구조 방정식 모형을 이용한 잠재평균비교를 통해 검증하였다. 연구 결과는 다음과 같다. 첫째, 베트남어로 번역된 Re_ STMI 20문항의 문항 반응 분석 결과 Item Reliability는 .97, Infit MNSQ는 .67-1.38으로 나타났으며 Item Map과 DIF 분석에서도 선행 연구에서 보고된 결과와 일치하는 결과가 도출되었다. 탐색적 요인 분석에서는 모든 문항들이 의도한 하위 요인에 적재되었으며, 요인별 신뢰도는 .662-.833, 전체 신뢰도는 .876으로 분석되었다. 우리나라 고등학생과 잠재평균비교를 위한 확인적 요인 분석에서 도출된 모형 적합도 수치는 모두 수용 가능한 값으로 분석되었다(χ2/df: 2.830, CFI: .931, TLI: .918, SRMR: .043, RMSEA: .051). 마지막으로 연구에 참여한 우리나라 고등학생과 베트남 고등학생 간 잠재평균비교에서는 시스템 분석, 정신 모델, 팀 학습, 공유비전 요인에서 작은 효과 크기가, 개인 숙련 요인에선 중간 이상의 효과 크기를 보이며, 베트남 고등학생들이 시스템 사고 능력에서 유의미하게 높은 결과를 보여주었다. 이를 통해 개정 시스템 사고 검사 도구 문항은 안정적인 신뢰도와 타당도를 가지고 있음을 확인할 수 있었다. 앞으로 학생들의 시스템 사고 연구와 관련하여 베트남어 및 영어 등으로 번역한 문항을 활용하여 시스템 사고의 국제 비교 연구도 진행할 필요성이 있을 것이다.

Keywords

References

  1. Aguilar-Cisneros, J. R., Valerdi, R., and Sullivan, B. P., 2022, Students' Systems Thinking Competency Level Detection through Software Cost Estimation Concept Modeling. Programming & Computer Software, 48(8), 499-512.
  2. Batt, A. M., Williams, B., Brydges, M., Leyenaar, M., and Tavares, W., 2021, New ways of seeing: supplementing existing competency framework development guidelines with systems thinking. Advances in Health Sciences Education, 26(4), 1355-1371.
  3. Ben-zvi-Assaraf, O., and Orion, N., 2005a, Development of System Thinking Skills in the Context of Earth System Education. Journal of Research in Science Teaching, 42(5), 518-560.
  4. Ben-zvi-Assaraf, O., and Orion, N., 2005b, A Study of Junior High Students' Perceptions of the water cycle. Journal of Geoscience Education, 53(4), 366-373.
  5. Ben-Zvi-Assaraf, O., and Orion, N., 2009, A Design Based Research of an Earth Systems Based Environmental Curriculum. Eurasia Journal of Mathematics, Science & Technology Education, 5(1), 47-62.
  6. Boone, W., and Scantlebury, K., 2006, The Role of Rasch Analysis When Conducting Science Education Research Utilizing Multiple-Choice Tests. Science Education, 90, 253-269.
  7. Boone, W., Staver, J. and Yale, M., 2014, Rasch Analysis in the Human Sciences. Springer: Switzerland.
  8. Boone, W., Townsend, J., and Staver, J., 2011, Using Rasch Theory to Guide the Practice of Survey Development and Survey Data Analysis in Science Education and to Inform Science Reform Efforts: An Exemplar Utilizing STEBI Self-Efficacy Data. Science Education, 95, 258-280.
  9. Cho, K., and Kim, H., 2017, Effect of Systems Thinking Based STEAM Education Program on Climate Change Topics. Journal of Digital Contents Society, 17(7), 113-123.
  10. Cho, O., and Hwang, K., 2016, The effects of simulation-based education on nursing students' presence in education, systems thinking and proactivity in problem solving. Journal of Korean Academic Society of Home Health Care Nursing, 23(2), 147-154.
  11. Cohen, J., 2013, Statistical power analysis for the behavioral sciences. Academic press.
  12. Davis A., and Stroink, M., 2016, The Relationship between Systems Thinking and the New Ecological Paradigm. Systems Research and Behavioral Science, 33, 575-586.
  13. Dolansky, M. A., Moore, S. M., Palmieri, P. A., and Singh, M. K., 2020, Development and validation of the systems thinking scale. Journal of General Internal Medicine, 35, 2314-2320.
  14. Ha, M., 2016, Examining the Validity of History-of-Science-Based Evolution Concept Assessment and Exploring Conceptual Progressions by Contexts. Journal of the Korean Association for Science Education, 36(3), 509-517.
  15. Han, D., and Jo, E., 2015, A study on the exploration of relationship between environmental literacy and systems thinking for sustainability education in social studies. Social Studies Education, 54, 65-83.
  16. Hung, W., 2008, Enhancing systems thinking skills with modelling. British Journal of Educational Technology, 39(6), 1099-1120.
  17. Im, Y., and Lee, H., 2014, Development and analysis of effects of writing educational program for improving system thinking ability. Journal of Learner-Centered Curriculum and Instruction, 14, 407-427.
  18. Jeon, J., and Lee, H., 2015, The development and application of STEAM education program based on systems thinking for high school students. Journal of the Korean Association for Science Education, 35(6), 1007-1018.
  19. Jo, J., 2019, Test of Construct Equivalence of the Career Indecision Inventory for Korean University Students and the Comparison of Latent Means between Male and Female Students. Journal of Education Culture, 25(2), 339-357.
  20. Jo, S., 2011, A Study of Factors Effecting on Gifted Students' Achievement: Self-determination, Learning Goal-orientation, Self-efficacy, Implicit Theory of Intelligence, and Self-regulated Learning Strategy. Journal of Gifted/Talented Education, 21(3), 611-630.
  21. Kali, Y., Orion, N., and Eylon, B., 2003, Effect of Knowledge Integration Activities on Students' Perception of the Earth's Crust as a Cyclic System. Journal of Research in Science Teaching, 40(6), 545-565.
  22. Kang, C., Lee, H., Yoon, I., and Kim, E., 2008, Analysis of conceptions related to Earth system and systems-thinking of high school student about water cycle. Journal of Science Education, 32(1), 61-72. https://doi.org/10.21796/JSE.2008.32.1.61
  23. Kim, M., and Yoo, J., 2022, Exploration of Predictors for Male and Female Students' Reading Literacy: Evidence from PISA 2018 via the Elastic Net. Asian Journal of Education, 23(4), 787-812.
  24. Kowch, E. G., 2019, Introduction to Systems Thinking and Change. In M. J. Spector, B. B. Lockee, & M. D. Childress (Eds.), Learning, Design, and Technology (pp. 1-14). Springer International Publishing. https://doi.org/10.1007/978-3-319-17727-4_133-1.
  25. Kwon, Y., Kim, W., Lee, H., Byun, J., and Lee, I., 2011, Analysis of Biology Teachers' Systems Thinking about Ecosystem. Biology Education, 39(4), 529-543. https://doi.org/10.15717/BIOEDU.2011.39.4.529
  26. Lee, H, Lee, H., and Oh, H., 2020, Path Analysis Study among Science Motivation, Science Self-Efficacy and System Thinking of Student related Medical Major. The Journal of Learner-Centered Curriculum and Instruction, 20(18), 1193-1209.
  27. Lee, H., and Lee, H., 2013, Revalidation of measuring instrument systems thinking and comparison of systems thinking between science and general high school students. Journal of the Korean Association for Science Education, 33(4), 1237-1247.
  28. Lee, H., 2019, Validity Verification of Systems Thinking Measuring Instrument of Elementary School Students' and Middle School Students': Using Item Response Analysis. Secondary Education Research, 67(2), 249-277. https://doi.org/10.25152/SER.2019.67.2.249
  29. Lee, H., 2020, An Analysis of Systems Thinking Levels of the Water Cycle in Earth System of Primary School Pre-Service Teachers. Teacher Education Research, 59(1), 21-38. https://doi.org/10.15812/TER.59.1.202003.21
  30. Lee, H., 2022, Validation of Science Self-Efficacy Scale for Pre-Service Teachers and Latent Mean Analysis According to Background Variable. Journal of Korean Elementary Science Education, 41(1), 65-78.
  31. Lee, H., and Lee, H., 2016, Effects of Systems Thinking on High School Students' Science Self-Efficacy. The Journal of The Korean Earth Science Society, 37(3), 133-145.
  32. Lee, H., and Lee, H., 2017, Analysis and Effects of High School Students' Systems Thinking Using Iceberg (IB) Model. Journal of the Korean Association for Science Education, 37(4), 611-624.
  33. Lee, H., Jeon, J., and Lee, H., 2018, Development of Framework and Rubric for Measuring Students' Level of Systems Thinking. Journal of the Korean Association for Science Education, 38(3), 355-367.
  34. Lee, H., Jeon, J., and Lee, H., 2019, Verification the Systems Thinking Factor Structure and Comparison of Systems Thinking Based on Preferred Subjects about Elementary School Students'. Journal of the Korean Association for Science Education, 39(2), 161-171.
  35. Lee, H., Kwon, H., Park, K., and Lee, H., 2013, An instrument development and validation for measuring high school students' systems thinking. Journal of the Korean Association for Science Education, 33(3), 995-1006.
  36. Lee, H., Kwon, Y., Oh, H., and Lee, H., 2011, Development and application of the educational program to increase high school students' systems thinking skills: Focus on global warming. Journal of the Korean Earth Science Society, 32, 784-797.
  37. Lee, H., Nguyen, T., Lee, H., Jeon, J., and Park, B., 2023, The Validation of the Systems Thinking Assessment Tool for Measuring the Higher-order Thinking Ability of Vietnamese High School Students. The Journal of The Korean Earth Science Society, 44(4), 318-330.
  38. Lee, H., Park, B., Jeon, J., and Lee, H., 2024, Validation of the Revised Systems Thinking Measuring Instrument (Re_STMI) for High School Students. Brain, Digital, & Learning, 14(1), 1-19.
  39. Ministry of Education [MOE]., 2022, 2022 revised curriculum: Science. Sejong: Ministry of Education.
  40. Momsen, J., Speth, E. B., Wyse, S., and Long, T., 2022, Using Systems and Systems Thinking to Unify Biology Education. CBE-Life Sciences Education, 21(2), es3.
  41. Moon, B., Jeong, J., Kyung, J., Koh, Y., Youn, S., Kim, H., and Oh, K., 2004, Related conception s to earth system and applying of systems thinking about carbon cycle of the preservice teachers. Journal of the Korean Earth Science Society, 25, 684-696.
  42. Park, B., and Bae, S., 2016, Testing Measurement Invariance and Latent Mean Analysis of the Internet Game Addiction Scale of Adolescents Depending on Gender and School stage. Journal of Youth Welfare, 18(2), 65-84.
  43. Park, K., Lee, H., Lee, H., and Jeon, J., 2019, Analysis of Systems Thinking Level of Pre-service Teachers about Carbon Cycle in Earth Systems using Rubrics of Evaluating Systems Thinking. Journal of the Korean Association for Science Education, 39(5), 599-611.
  44. Park, M., and Choi, D., 2018, The Effect of Simulation Integrated with Problem Based Learning on System Thinking, Learning Flow, Proactivity in Problem Solving and Performance Ability for Medication in Nursing Students. Journal of Digital Convergence, 16(8), 221-231. https://doi.org/10.14400/JDC.2018.16.8.221
  45. Ratinen, I., Linnanen, L., Claudelin, A., and Halonen, V., 2023, Toward Sustainable Development: Connecting Systems Thinking Competency and Carbon Footprint Knowledge. Sustainable Development, 31(3), 1593-1605.
  46. Rust, J., and Golombok, S., 2014, Modern psychometrics: The science of psychological assessment. Routledge.
  47. Senge, P. M., 1996, The fifth discipline: Fieldbook. New York: Broadway Business.
  48. Senge, P. M., 2012, Schools that learn (Updated and Revised): A fifth discipline fieldbook for educators, parents, and everyone who cares about education. New York: Doubleday.
  49. Song, J., Moon, B., and Kim, J., 2015, The Development and Application of the Teaching-Learning Program for Systems Thinking Learning in Elementary Science Classes. Journal of Korean Society of Earth Science Education, 8(3), 318-331.
  50. Ulku, S. B., and Gaye, D. C., 2024, Research Trends on Systems Thinking Approach in Science Education. International Journal of Science Education, 46(5), 485-502.
  51. Wright, B., and Linacre, J., 1994, Reasonable mean-square-fit values. Rasch Measurement Transactions, 8(3), 370.