Educational Technology International

The Korean Society for Education Technology (한국교육공학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on the Effectiveness of LMS for Improving College Student's Mathematics Performance using a Propensity Score Matching Method

  • Received : 2024.03.10
  • Accepted : 2024.04.18
  • Published : 2024.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

This study aims to verify the practical effectiveness of learning management system (LMS) by introducing a LMS enhancing digital assessment utilizing automatic item generation in order to strengthen college student's mathematics performance. Teaching assisted with digital assessment in the LMS was applied to college mathematics classes, and the research question is whether or not students in the classes utilizing the LMS perform better than the regular classes. In particular, a calculus course, which is the foundation of important artificial intelligence technology in the future, was utilized in this study. The participants of this study were 248 freshmen in science and engineering who were taking calculus courses at a small to mid-size university. A total of 156 freshmen were selected after applying a propensity score matching method (PSMM), 78 from classes utilizing the LMS and 78 from regular classes without the LMS assisted with the digital assessment. As a result, it was found that there was a statistically significant difference in the math academic growth of students who used the LMS and those who did not. In other words, when LMS was used in calculus, students' academic growth was greater. The results of this study are meaningful in that they observed students' academic growth and confirmed that LMS enables a positive role in students' academic growth. In addition, if digital assessment is strengthened and LMS that enables individualized learning analysis is introduced and implemented in educational institutions, it is expected to play a major role in strengthening students' academic performance.

Keywords

References

  1. Cha, M., & Im, H. (2023). A Study on University Professors' Perception on Educational Applicability of ChatGPT in English Classes. Culture and Convergence, 45(5), 109-118.
  2. Chang, S. (2023). ChatGPT has Changed the Future of Writing Education - Focusing on the response of writing education in the era of artificial intelligence -. Research on Writing, 56, 7-34.
  3. Cho, S., Pyo, Y., Park, J., & Seo, J. (2017). Error types and guidance measures for college students with poor basic academic ability in mathematics. Studies in Mathematical Education, 1, 91-93.
  4. Choi, M. (2019). "A Study on the Interactivity of Smart LMS in a University Class", Journal of Digital Convergence, 17(3), 395-404. https://doi.org/10.14400/JDC.2019.17.3.395
  5. Edsurge. (2016). Decording adaptive. London: Pearson. Retrieved July 12, 2018, http://d3e7x39d4i7wbe.cloudfront.net/static_assets/PearsonDecodingAdaptiveWeb.pdf.
  6. Gierl, M. J., & Haladyna, T. M. (Eds.). (2012). Automatic item generation: Theory and practice. Routledge.
  7. Gu, X., & Rosenbaum, P. R. (1993). Comparison of multivariate matching methods: Structures, distances, and algorithms. Journal of Computational and Graphical Statistics, 2, 405-420.
  8. Guo, S., & Fraser, M. W. (2015). Propensity score analysis: Statistical methods and applications (2nd ed.). ThousandOaks, CA: Sage.
  9. Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). "MatchIt: Nonparametric preprocessing for parametric causal inference,". Journal of Statistical Software, 15, 199-236.
  10. Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2011). MatchIt: Nonparametric Preprocessing for Parametric Causal Inference. Journal of Statistical Software, 42(8). 1-28.
  11. Hong, S., Kim, H., & Park, Y. (2021). Exploring the Potentials of AI Integration into K-12 Education. Journal of Korean Association for Educational Information and Media, 27(3), 875-898. https://doi.org/10.15833/KAFEIAM.27.3.875
  12. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional fit criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55.
  13. Kang, M., Heo, H., Jo, I., Shin, J., & Seo, J. (2010). Developing an educational performance indicator for new millennium learners. Journal of Research on Technology in Education, 43(2), 157-170.
  14. Kim, A. (2021). Analysis of Learner Behavior and Learning Performance using LMS Big Data in the COVID-19: Focused on J-University. The Journal of Learner-Centered Curriculum and Instruction, 21(6), 565-579.
  15. Kim, E. (2022). A Study on the Relationship Between Digital Literacy and Major Satisfaction of Nursing Students: Intermediary Role of LMS Learning Effects. The Journal of the Convergence on Culture Technology, 8(4), 27-35. https://doi.org/10.17703/JCCT.2022.8.4.27
  16. Kim, K. (2017). A study on mathematics attitude and mathematics achievement of low achievement students in university mathematics. Master's dissertation, Pukyung National University, Busan, Korea.
  17. Kim, K., & Ryu, B. (2019). Expectations for and concerns about the use of Fourth Industrial Revolution technology in education. The journal of Educational Studies, 50(3), 55-79. https://doi.org/10.15854/jes.2019.9.50.3.55
  18. Kim, S. (2021). The proposal of AI education direction reflecting cases in major countries and review of Korean textbooks. Master's dissertation, Jeonbuk National University, Jeonbuk, Korea.
  19. Kim, S., & Cho, Y. (2018). Proliferation of Online Learning and the Implications for Teaching and Learning in Future University Education. Journal of Lifelong Learning Society, 14(4), 51-78.
  20. Kim, T. (2023). Can chatGPT Become a Tool for Innovation?: ChatGPT Use Cases and Prospects. The AI Report, 1, National Information Society Agency. Daegu.
  21. Korean Ministry of Education. (2022). 2021 National Level Academic Achievement Assessment Results and Response Strategies Announced. Sejong.
  22. Lee, H., Park, H., & Yang, J. (2023). Development and Validation of Measuring Problem-Based Learning Effectiveness. Journal of Educational Technology, 39(1), 35-63.
  23. Lee, J., Lee. S., Kwon, H., & Lee, G. (2010). A Study on the relation between Mathematical Scholastic Ability and Scholastic Aptitude Test. Communications of Mathematical Education, 25(4), 629-639.
  24. Lee, J., Lee, S., Kwon, H., & Lee, G. (2011). The Effect of Basic Mathematical Ability Improvement Program on Scholastic Achievement and Scholastic Motivation: A Case Study on Engineering Freshmen in D University. Communications of Mathematical Education, 25(1), 167-184.
  25. Lee, J., Park, H., Lee, J., & Park, M. (2023). A Comparative Analysis on LMS Features for Supporting Instructors. Journal of Digital Contents Society, 24(2), 399-410. https://doi.org/10.9728/dcs.2023.24.2.399
  26. Lee, S., & Heo Y. (2023). Types of Learned helplessness Learners and Factors Influencing on Changes in the Era of Future Education based on ChatGPT. The Journal of General Education, 20, 87-119.
  27. Lee, Y., Hwang, Y., Yun, J., Yoon, J., & Hwang, I. (2022). Development of Basic Mathematical Ability Test for University and Application Effect of Extracurricular Program. The Journal of General Education, 20, 87-119. https://doi.org/10.24173/jge.2022.07.20.4
  28. Mitra, R., & Reiter, J. P. (2012). A comparison of two methods of estimating propensity scores after multiple imputation. Statistical Methods in Medical Research, 25(1), 188-204. https://doi.org/10.1177/0962280212445945
  29. Park, H., Ryoo, H., Kwon, J., & Ryoo, J. (2022). Change of Paradigm on LMS for Online Education: LMS Implementing Learning Analytics and Online Assessment. The Educational Research for Tomorrow, 35(2), 49-72.
  30. Park, J., & Pyo Y. (2013). Improvement strategies of teaching methods for university basic mathematics education courses by ability grouping. Communications of Mathematical Education, 27(1), 19-37.
  31. Park, J., & Gil, J. (2020). Edutech in the Era of the 4th Industrial Revolution. KIPS Transactions on Software and Data Engineering, 9(11), 329-331.
  32. Park, S., & Hong, H. (2022). A Study on the Improvement of School Mathematics Curriculum for SW.AI Education. The Journal of Korean Association of Computer Education, 25(2), 59-77.
  33. Pyo, Y., & Park, J. (2010). Effective management strategies of basic mathematics for Low achievement students in university general mathematics. Communications of Mathematical Education, 24(3), 525-541.
  34. Pyo, Y., & Park, J. (2011). Effective Management Strategies of University Basic Mathematics by Ability Grouping. Journal of educational research in mathematics, 21(1), 87-103.
  35. Rubin, D. B. (2007). The design versus the analysis of observational studies for causal effects: Parallels with the design of randomized trial. Statistics in Medicine, 26, 20-36. https://doi.org/10.1002/sim.2739
  36. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41-55. https://doi.org/10.1093/biomet/70.1.41
  37. Ryoo, J., & Kwon, J. (2021). Equity of Educational Opportunity in Higher Education focusing on Fundamental Liberal Education: Cultivating of People on the 4th Industrial Revolution. Korean Council for University Education. Seoul.
  38. Sa, S., Ryoo, H. S., & Ryoo, J. H. (2022). Application of AIG implemented within CLASS Software for Generating Cognitive Test Item Models. Educational Technology International, 23(2), 157-181. https://doi.org/10.23095/ETI.2022.23.2.157
  39. Sin, S. (2006). A Study on the Connection between the 7th High School Mathematics Curriculum and University Mathematical Education. Master's dissertation, Ulsan University, Ulsan, Korea.
  40. Woo, J., Kim, B., & Bu, K. (2012). A Study on the Verification of Influential Power over Learning Effect by the Learning Participation Activities: the LMS of the O Cyber University. The Journal of Korean Institute of Information Technology, 10(4), 97-103.
  41. Yang, J., & Yoon, S. (2023). Beyond ChatGPT to the Generative AI Era: Cases of Media.Content Generation Type AI Services and Ways to Securing Competitiveness. Media Issues & Trends, 55, 62-70.
  42. Yoon, S., & Yang, J. (2021). Fundamentals of AI and Data Analysis: Digital Business Survival Strategies. Pakyoungsa.