International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

Accuracy of Data-Model Fit Using Growing Levels of Invariance Models

  • Almaleki, Deyab A. (Department of Evaluation, Measurement and Research, Umm Al-Qura University)
  • Received : 2021.12.05
  • Published : 2021.12.30
Download PDF
⟨ Previous Next ⟩

Abstract

The aim of this study is to provide empirical evaluation of the accuracy of data-model fit using growing levels of invariance models. Overall model accuracy of factor solutions was evaluated by the examination of the order for testing three levels of measurement invariance (MIV) starting with configural invariance (model 0). Model testing was evaluated by the Chi-square difference test (∆𝛘2) between two groups, and root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker-Lewis index (TLI) were used to evaluate the all-model fits. Factorial invariance result revealed that stability of the models was varying over increasing levels of measurement as a function of variable-to-factor ratio (VTF), subject-to-variable ratio (STV), and their interactions. There were invariant factor loadings and invariant intercepts among the groups indicating that measurement invariance was achieved. For VTF ratio (3:1, 6:1, and 9:1), the models started to show accuracy over levels of measurement when STV ratio was 6:1. Yet, the frequency of stability models over 1000 replications increased (from 69% to 89%) as STV ratio increased. The models showed more accuracy at or above 39:1 STV.

Keywords

References

  1. M. J. Allen and W. M. Yen, "Introduction to measurement theory, Monterey, CA: Brooks/Cole, 1979," Google Scholar, 1979.
  2. K. M. Marcoulides, N. Foldnes, and S. Gronneberg, "Assessing model fit in structural equation modeling using appropriate test statistics," Structural Equation Modeling: A Multidisciplinary Journal, vol. 27, no. 3, pp. 369-379, 2020. https://doi.org/10.1080/10705511.2019.1647785
  3. Y. Rosseel, "Small sample solutions for structural equation modeling," SMALL SAMPLE SIZE SOLUTIONS, p. 226, 2020.
  4. S. C. Smid, D. McNeish, M. Miocevic, and R. van de Schoot, "Bayesian versus frequentist estimation for structural equation models in small sample contexts: A systematic review," Structural Equation Modeling: A Multidisciplinary Journal, vol. 27, no. 1, pp. 131-161, 2020. https://doi.org/10.1080/10705511.2019.1577140
  5. S. Zitzmann and M. Hecht, "Going beyond convergence in Bayesian estimation: Why precision matters too and how to assess it," Structural Equation Modeling: A Multidisciplinary Journal, vol. 26, no. 4, pp. 646-661, 2019. https://doi.org/10.1080/10705511.2018.1545232
  6. L. Crocker and J. Algina, Introduction to classical and modern test theory. ERIC, 1986.
  7. E. G. W. Velicer, "Relation of sample size to the stability of component patterns," Psychological Bulletin, vol. 103, pp. 265-275, 1988. https://doi.org/10.1037//0033-2909.103.2.265
  8. Y. A. Wang and M. Rhemtulla, "Power analysis for parameter estimation in structural equation modeling: A discussion and tutorial," 2020.
  9. J. C. Westland, "Lower bounds on sample size in structural equation modeling," Electronic commerce research and applications, vol. 9, no. 6, pp. 476-487, 2010. https://doi.org/10.1016/j.elerap.2010.07.003
  10. D. L. Bandalos and P. Gagne, "Simulation methods in structural equation modeling.," 2012.
  11. T. A. Brown, Confirmatory factor analysis for applied research. Guilford publications, 2015.
  12. P. F. M. Bullon, "Failing to replicate: Hypothesis testing as a crucial key to make direct replications more credible and predictable," 2015.
  13. O. P. John and S. Srivastava, "The Big Five trait taxonomy: History, measurement, and theoretical perspectives," Handbook of personality: Theory and research, vol. 2, no. 1999, pp. 102-138, 1999.
  14. J. Wang and X. Wang, Structural equation modeling: Applications using Mplus. John Wiley & Sons, 2019.
  15. D. Moody, "The Method Evaluation Model: A Theoretical Model for Validating Information Systems Design Methods," ECIS 2003 Proceedings, Jan. 2003, [Online]. Available: https://aisel.aisnet.org/ecis2003/79.
  16. D. Almaleki, "Empirical Evaluation of Different Features of Design in Confirmatory Factor Analysis," 2016.
  17. C. S. Wardley, E. B. Applegate, A. D. Almaleki, and J. A. Van Rhee, "A comparison of Students' perceptions of stress in parallel problem-based and lecture-based curricula," The Journal of Physician Assistant Education, vol. 27, no. 1, pp. 7-16, 2016. https://doi.org/10.1097/JPA.0000000000000060
  18. C. S. Wardley, E. B. Applegate, A. D. Almaleki, and J. A. Van Rhee, "Is Student Stress Related to Personality or Learning Environment in a Physician Assistant Program?," The Journal of Physician Assistant Education, vol. 30, no. 1, pp. 9-19, 2019. https://doi.org/10.1097/JPA.0000000000000241
  19. D. Almaleki, "Examinee Characteristics and their Impact on the Psychometric Properties of a Multiple Choice Test According to the Item Response Theory (IRT)," Engineering, Technology & Applied Science Research, vol. 11, no. 2, pp. 6889-6901, 2021. https://doi.org/10.48084/etasr.4056
  20. D. Almaleki, "Stability of the Data-Model Fit over Increasing Levels of Factorial Invariance for Different Features of Design in Factor Analysis," Engineering, Technology & Applied Science Research, vol. 11, no. 2, pp. 6849-6856, 2021. https://doi.org/10.48084/etasr.4047
  21. D. Almaleki, "The Precision of the Overall Data-Model Fit for Different Design Features in Confirmatory Factor Analysis," Engineering, Technology & Applied Science Research, vol. 11, no. 1, pp. 6766-6774, 2021. https://doi.org/10.48084/etasr.4025
  22. D. A. Almaleki, "Challenges Experienced Use of Distance-Learning by High School Teachers Responses to Students with Depression," International Journal of Computer Science and Network Security, vol. 21, no. 5, pp. 192-198, May 2021, doi: 10.22937/IJCSNS.2021.21.5.27.
  23. D. A. Almaleki, "The Psychometric Properties of Distance-Digital Subjective Happiness Scale," International Journal of Computer Science and Network Security, vol. 21, no. 5, pp. 211-216, May 2021, doi: 10.22937/IJCSNS.2021.21.5.29.
  24. D. A. Almaleki, R. A. Alhajaji, and M. A. Alharbi, "Measuring Students' Interaction in Distance Learning Through the Electronic Platform and its Impact on their Motivation to Learn During Covid-19 Crisis," International Journal of Computer Science and Network Security, vol. 21, no. 5, pp. 98-112, May 2021, doi: 10.22937/IJCSNS.2021.21.5.16.
  25. D. A. Almaleki, W. W. Khayat, T. F. Yally, and A. A. Al-hajjaji, "The Effectiveness of the Use of Distance-Evaluation Tools and Methods among Students with Learning-Difficulties from the Teachers' Point of View," International Journal of Computer Science and Network Security, vol. 21, no. 5, pp. 243-255, May 2021, doi: 10.22937/IJCSNS.2021.21.5.34.
  26. "Evaluating Psychological Experiences of Saudi Students in Distance ." -Learning https://scholar.google.com/citations?view_op=view_citation&hl=en&user=RWnye6UAAAAJ&citation_for_view=RWnye6UAAAAJ:kNdYIx-mwKoC (accessed Oct. 13, 2021).
  27. "Factor Structure, Validity and Reliability of The Teacher Satisfaction Scale (TSS) In Distance-Learning During Covid-19 Crisis: Invariance Across Some Teachers' Characteristics." https://scholar.google.com/citations?view_op=view_citation&hl=en&user=RWnye6UAAAAJ&citation_for_view=RWnye6UAAAAJ:MXK_kJrjxJIC (accessed Oct. 13, 2021).
  28. " stimating the Ability on The Effect of Methods of E The Accuracy and Items Parameters According to 3PL Model." https://scholar.google.com/citations?view_op=view_citation&hl=en&user=RWnye6UAAAAJ&citation_for_view=RWnye6UAAAAJ:8k81kl-MbHgC (accessed Oct. 13, 2021).
  29. " Digital The Psychometric Properties of Distance-Subjective Happiness Scale." https://scholar.google.com/citations?view_op=view_citation&hl=en&user=RWnye6UAAAAJ&citation_for_view=RWnye6UAAAAJ:UebtZRa9Y70C (accessed Oct. 13, 2021).
  30. L. H. Lam, T. D. H. Phuc, and N. H. Hieu, "Simulation Models For Three-Phase Grid Connected PV Inverters Enabling Current Limitation Under Unbalanced Faults," Engineering, Technology & Applied Science Research, vol. 10, no. 2, pp. 5396-5401, Apr. 2020, doi: 10.48084/etasr.3343.
  31. A. H. Blasi and M. Alsuwaiket, "Analysis of Students' Misconducts in Higher Education using Decision Tree and ANN Algorithms," Engineering, Technology & Applied Science Research, vol. 10, no. 6, pp. 6510-6514, Dec. 2020, doi: 10.48084/etasr.3927.
  32. D. Almaleki, "The Precision of the Overall Data-Model Fit for Different Design Features in Confirmatory Factor Analysis," Engineering, Technology & Applied Science Research, vol. 11, no. 1, pp. 6766-6774, Feb. 2021, doi: 10.48084/etasr.4025.
  33. B. Thompson, "Ten commandments of structural equation modeling.," in US Dept of Education, Office of Special Education Programs (OSEP) Project Directors' Conference, 1998, Washington, DC, US; A previous version of this chapter was presented at the aforementioned conference and at the same annual conference held in 1999., 2000.
  34. K. B. Coughlin, "An analysis of factor extraction strategies: A comparison of the relative strengths of principal axis, ordinary least squares, and maximum likelihood in research contexts that include both categorical and continuous variables," 2013.
  35. J. J. Hox, C. J. Maas, and M. J. Brinkhuis, "The effect of estimation method and sample size in multilevel structural equation modeling," Statistica neerlandica, vol. 64, no. 2, pp. 157-170, 2010. https://doi.org/10.1111/j.1467-9574.2009.00445.x
  36. A. J. Morin, N. D. Myers, and S. Lee, "Modern Factor Analytic Techniques: Bifactor Models, Exploratory Structural Equation Modeling (ESEM), and Bifactor-ESEM," Handbook of sport psychology, pp. 1044-1073, 2020.
  37. D. Almaleki, "Empirical Evaluation of Different Features of Design in Confirmatory Factor Analysis," 2016.
  38. R. K. Henson and J. K. Roberts, "Use of exploratory factor analysis in published research: Common errors and some comment on improved practice," Educational and Psychological measurement, vol. 66, no. 3, pp. 393-416, 2006. https://doi.org/10.1177/0013164405282485
  39. A. W. Meade and D. J. Bauer, "Power and precision in confirmatory factor analytic tests of measurement invariance," Structural Equation Modeling: A Multidisciplinary Journal, vol. 14, no. 4, pp. 611-635, 2007. https://doi.org/10.1080/10705510701575461
  40. J. C. de Winter*, D. Dodou*, and P. A. Wieringa, "Exploratory factor analysis with small sample sizes," Multivariate behavioral research, vol. 44, no. 2, pp. 147-181, 2009. https://doi.org/10.1080/00273170902794206
  41. L. R. Fabrigar, D. T. Wegener, R. C. MacCallum, and E. J. Strahan, "Evaluating the use of exploratory factor analysis in psychological research.," Psychological methods, vol. 4, no. 3, p. 272, 1999. https://doi.org/10.1037//1082-989X.4.3.272
  42. R. Jacobucci, A. M. Brandmaier, and R. A. Kievit, "A practical guide to variable selection in structural equation modeling by using regularized multiple-indicators, multiple-causes models," Advances in methods and practices in psychological science, vol. 2, no. 1, pp. 55-76, 2019. https://doi.org/10.1177/2515245919826527
  43. E. Guadagnoli and W. F. Velicer, "Relation of sample size to the stability of component patterns.," Psychological bulletin, vol. 103, no. 2, p. 265, 1988. https://doi.org/10.1037//0033-2909.103.2.265
  44. A. B. Costello and J. Osborne, "Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis," Practical assessment, research, and evaluation, vol. 10, no. 1, p. 7, 2005.
  45. X. An and Y.-F. Yung, "Item Response Theory: What It Is and How You Can Use the IRT Procedure to Apply It," p. 14.
  46. F. B. Bryant and P. R. Yarnold, "Principal-components analysis and exploratory and confirmatory factor analysis.," 1995.
  47. C. M. Ringle, M. Sarstedt, R. Mitchell, and S. P. Gudergan, "Partial least squares structural equation modeling in HRM research," The International Journal of Human Resource Management, vol. 31, no. 12, pp. 1617-1643, 2020. https://doi.org/10.1080/09585192.2017.1416655