Browse > Article
http://dx.doi.org/10.22937/IJCSNS.2021.21.7.12

The Effect of Methods of Estimating the Ability on The Accuracy and Items Parameters According to 3PL Model  

Almaleki, Deyab A. (Department of Evaluation, Measurement and Research, Umm Al-Qura University)
Alomrany, Ahoud Ghazi (Department of Evaluation, Measurement and Research, Umm Al-Qura University)
Publication Information
International Journal of Computer Science & Network Security / v.21, no.7, 2021 , pp. 93-102 More about this Journal
Abstract
This study aimed to test method on the accuracy of estimating the items parameters and ability, using the Three Parameter Logistic. To achieve the objectives of the study, an achievement test in chemistry was constructed for third-year secondary school students in the course of "natural sciences". A descriptive approach was employed to conduct the study. The test was applied to a sample of (507) students of the third year of secondary school in the "Natural Sciences Course". The study's results revealed that the (EAP) method showed a higher degree of accuracy in the estimation of the difficulty parameter and the abilities of persons higher than the MML method. There were no statistically significant differences in the accuracy of the parameter estimation of discrimination and guessing regarding the difference of the two methods: (MML) and (EAP).
Keywords
Estimation-Accuracy; Three-Parameter Logistic Model (3PL); Items Parameters; Accuracy;
Citations & Related Records
연도 인용수 순위
  • Reference
1 C. DeMars, "Group differences based on IRT scores: Does the model matter?," Educ. Psychol. Meas., vol. 61, no. 1, pp. 60-70, 2001.   DOI
2 J. Chen and J. Choi, "A comparison of maximum likelihood and expected a posteriori estimation for polychoric correlation using Monte Carlo simulation," J. Mod. Appl. Stat. Methods, vol. 8, no. 1, p. 32, 2009.
3 D. Almaleki, "Examinee Characteristics and their Impact on the Psychometric Properties of a Multiple Choice Test According to the Item Response Theory (IRT)," Eng. Technol. Appl. Sci. Res., vol. 11, no. 2, pp. 6889-6901, 2021.   DOI
4 D. Almaleki, "Empirical Evaluation of Different Features of Design in Confirmatory Factor Analysis," 2016.
5 M. Wu and R. Adams, Applying the Rasch model to psychosocial measurement: A practical approach. Educational Measurement Solutions Melbourne, 2007.
6 D. R. Divgi, "A minimum chi-square method for developing a common metric in item response theory," Appl. Psychol. Meas., vol. 9, no. 4, pp. 413-415, 1985.   DOI
7 F. M. Lord, "Maximum likelihood and Bayesian parameter estimation in item response theory," J. Educ. Meas., pp. 157-162, 1986.
8 B. Zhuang, S. Wang, S. Zhao, and M. Lu, "Computed tomography angiography-derived fractional flow reserve (CT-FFR) for the detection of myocardial ischemia with invasive fractional flow reserve as reference: systematic review and meta-analysis," Eur. Radiol., vol. 30, no. 2, pp. 712-725, 2020.   DOI
9 C. E. Cantrell, "Item Response Theory: Understanding the One-Parameter Rasch Model.," 1997.
10 R. K. Hambleton and H. J. Rogers, "Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods," Appl. Meas. Educ., vol. 2, no. 4, pp. 313-334, 1989.   DOI
11 C. Magno, "Demonstrating the difference between classical test theory and item response theory using derived test data," Int. J. Educ. Psychol. Assess., vol. 1, no. 1, pp. 1-11, 2009.
12 T. Strachan et al., "Using a Projection IRT Method for Vertical Scaling When Construct Shift Is Present," J. Educ. Meas., 2020.
13 W.-C. Lee, S. Y. Kim, J. Choi, and Y. Kang, "IRT Approaches to Modeling Scores on Mixed-Format Tests," J. Educ. Meas., vol. 57, no. 2, pp. 230-254, 2020.   DOI
14 D. R. Crisan, J. N. Tendeiro, and R. R. Meijer, "Investigating the practical consequences of model misfit in unidimensional IRT models," Appl. Psychol. Meas., vol. 41, no. 6, pp. 439-455, 2017.   DOI
15 R. G. Lim and F. Drasgow, "Evaluation of two methods for estimating item response theory parameters when assessing differential item functioning.," J. Appl. Psychol., vol. 75, no. 2, p. 164, 1990.   DOI
16 C. J. Maas and J. J. Hox, "Sufficient sample sizes for multilevel modeling," Methodology, vol. 1, no. 3, pp. 86-92, 2005.   DOI
17 K. M. Marcoulides, N. Foldnes, and S. Gronneberg, "Assessing model fit in structural equation modeling using appropriate test statistics," Struct. Equ. Model. Multidiscip. J., vol. 27, no. 3, pp. 369-379, 2020.   DOI
18 L. R. Bonetto, J. S. Crespo, R. Guegan, V. I. Esteves, and M. Giovanela, "Removal of methylene blue from aqueous solutions using a solid residue of the apple juice industry: full factorial design, equilibrium, thermodynamics and kinetics aspects," J. Mol. Struct., vol. 1224, p. 129296, 2021.   DOI
19 M. N. Morshed, M. N. Pervez, N. Behary, N. Bouazizi, J. Guan, and V. A. Nierstrasz, "Statistical modeling and optimization of heterogeneous Fenton-like removal of organic pollutant using fibrous catalysts: a full factorial design," Sci. Rep., vol. 10, no. 1, pp. 1-14, 2020.   DOI
20 H. Swaminathan, R. K. Hambleton, and J. Algina, "Reliability of criterion-referenced tests: A decision-theoretic formulation," J. Educ. Meas., vol. 11, no. 4, pp. 263-267, 1974.   DOI
21 H. Swaminathan, R. K. Hambleton, and H. J. Rogers, "21 Assessing the Fit of Item Response Theory Models," Handb. Stat., vol. 26, pp. 683-718, 2006.   DOI
22 G. J. Mellenbergh, "Item bias and item response theory," Int. J. Educ. Res., vol. 13, no. 2, pp. 127-143, 1989.   DOI
23 G. H. Fischer and I. W. Molenaar, Rasch models: Foundations, recent developments, and applications. Springer Science & Business Media, 2012.
24 D. A. Almaleki, W. W. Khayat, T. F. Yally, and A. A. Alhajjaji, "The Effectiveness of the Use of DistanceEvaluation Tools and Methods among Students with Learning-Difficulties from the Teachers' Point of View," Int. J. Comput. Sci. Netw. Secur., vol. 21, no. 5, pp. 243-255, May 2021, doi: 10.22937/IJCSNS.2021.21.5.34.   DOI
25 R. K. Hambleton, W. J. van der Linden, and C. S. Wells, "IRT models for the analysis of polytomously scored data," Handb. Polytomous Item Response Theory Models, pp. 21-42, 2010.
26 S.-S. Lee and J. Kim, "An Exploratory study on Student-Intelligent Robot Teacher relationship recognized by Middle School Students," J. Digit. Converg., vol. 18, no. 4, pp. 37-44, 2020.   DOI
27 S. Tibi, A. A. Edwards, C. Schatschneider, L. J. Lombardino, J. R. Kirby, and S. H. Salha, "IRT analyses of Arabic letter knowledge in Kindergarten," Read. Writ., pp. 1-26, 2020.
28 F. M. Lord, Applications of item response theory to practical testing problems. Routledge, 2012.
29 M. L. Stocking and F. M. Lord, "Developing a common metric in item response theory," Appl. Psychol. Meas., vol. 7, no. 2, pp. 201-210, 1983.   DOI
30 H. Kishino, T. Miyata, and M. Hasegawa, "Maximum likelihood inference of protein phylogeny and the origin of chloroplasts," J. Mol. Evol., vol. 31, no. 2, pp. 151-160, 1990.   DOI
31 G. J. Mellenbergh, "Item bias and item response theory," Int. J. Educ. Res., vol. 13, no. 2, pp. 127-143, 1989.   DOI
32 H. Kishino and M. Hasegawa, "Evaluation of the maximum likelihood estimate of the evolutionary tree topologies from DNA sequence data, and the branching order in Hominoidea," J. Mol. Evol., vol. 29, no. 2, pp. 170-179, 1989.   DOI
33 C. S. Wardley, E. B. Applegate, A. D. Almaleki, and J. A. Van Rhee, "A comparison of Students' perceptions of stress in parallel problem-based and lecture-based curricula," J. Physician Assist. Educ., vol. 27, no. 1, pp. 7-16, 2016.   DOI
34 H. Swaminathan, R. K. Hambleton, S. G. Sireci, D. Xing, and S. M. Rizavi, "Small sample estimation in dichotomous item response models: Effect of priors based on judgmental information on the accuracy of item parameter estimates," Appl. Psychol. Meas., vol. 27, no. 1, pp. 27-51, 2003.   DOI
35 R. K. Hambleton, H. Swaminathan, and H. J. Rogers, Fundamentals of item response theory, vol. 2. Sage, 1991.
36 D. Almaleki, "Stability of the Data-Model Fit over Increasing Levels of Factorial Invariance for Different Features of Design in Factor Analysis," Eng. Technol. Appl. Sci. Res., vol. 11, no. 2, pp. 6849-6856, 2021.   DOI
37 D. Almaleki, "The Precision of the Overall Data-Model Fit for Different Design Features in Confirmatory Factor Analysis," Eng. Technol. Appl. Sci. Res., vol. 11, no. 1, pp. 6766-6774, 2021.   DOI
38 J. Y. Park, F. Cornillie, H. L. van der Maas, and W. Van Den Noortgate, "A multidimensional IRT approach for dynamically monitoring ability growth in computerized practice environments," Front. Psychol., vol. 10, p. 620, 2019.   DOI
39 R. K. Hambleton and A. Kanjee, "Increasing the validity of cross-cultural assessments: Use of improved methods for test adaptations," Eur. J. Psychol. Assess., vol. 11, no. 3, pp. 147-157, 1995.   DOI
40 D. A. Almaleki, R. A. Alhajaji, and M. A. Alharbi, "Measuring Students' Interaction in Distance Learning Through the Electronic Platform and its Impact on their Motivation to Learn During Covid-19 Crisis," Int. J. Comput. Sci. Netw. Secur., vol. 21, no. 5, pp. 98-112, May 2021, doi: 10.22937/IJCSNS.2021.21.5.16.   DOI
41 E.-Y. Mun, Y. Huo, H. R. White, S. Suzuki, and J. de la Torre, "Multivariate higher-order IRT model and MCMC algorithm for linking individual participant data from multiple studies," Front. Psychol., vol. 10, p. 1328, 2019.   DOI
42 A. Preti, M. Vellante, and D. R. Petretto, "The psychometric properties of the 'Reading the Mind in the Eyes' Test: an item response theory (IRT) analysis," Cognit. Neuropsychiatry, vol. 22, no. 3, pp. 233-253, 2017.   DOI
43 C. S. Wardley, E. B. Applegate, A. D. Almaleki, and J. A. Van Rhee, "Is Student Stress Related to Personality or Learning Environment in a Physician Assistant Program?," J. Physician Assist. Educ., vol. 30, no. 1, pp. 9-19, 2019.   DOI
44 R. K. Hambleton and W. J. Van der Linden, Advances in item response theory and applications: An introduction. Sage Publications Sage CA: Thousand Oaks, CA, 1982.
45 G. C. Foster, H. Min, and M. J. Zickar, "Review of item response theory practices in organizational research: Lessons learned and paths forward," Organ. Res. Methods, vol. 20, no. 3, pp. 465-486, 2017.   DOI
46 G. Rajlic, "Violations of unidimensionality and local independence in measures intended as unidimensional: assessing levels of violations and the accuracy in unidimensional IRT model estimates," PhD Thesis, University of British Columbia, 2019.
47 R. Liu, A. C. Huggins-Manley, and O. Bulut, "Retrofitting diagnostic classification models to responses from IRT-based assessment forms," Educ. Psychol. Meas., vol. 78, no. 3, pp. 357-383, 2018.   DOI
48 M. A. Tanner and W. H. Wong, "The calculation of posterior distributions by data augmentation," J. Am. Stat. Assoc., vol. 82, no. 398, pp. 528-540, 1987.   DOI
49 S.-K. Chen, L. Hou, and B. G. Dodd, "A comparison of maximum likelihood estimation and expected a posteriori estimation in CAT using the partial credit model," Educ. Psychol. Meas., vol. 58, no. 4, pp. 569-595, 1998.   DOI
50 W. Ma, N. Minchen, and J. de la Torre, "Choosing between CDM and unidimensional IRT: The proportional reasoning test case," Meas. Interdiscip. Res. Perspect., vol. 18, no. 2, pp. 87-96, 2020.   DOI
51 C. L. Azevedo, D. F. Andrade, and J.-P. Fox, "A Bayesian generalized multiple group IRT model with model-fit assessment tools," Comput. Stat. Data Anal., vol. 56, no. 12, pp. 4399-4412, 2012.   DOI
52 I. Paek, M. Cui, N. Ozturk Gubes, and Y. Yang, "Estimation of an IRT model by Mplus for dichotomously scored responses under different estimation methods," Educ. Psychol. Meas., vol. 78, no. 4, pp. 569-588, 2018.   DOI
53 K. Matlock Cole and I. Paek, "PROC IRT: A SAS procedure for item response theory," Appl. Psychol. Meas., vol. 41, no. 4, pp. 311-320, 2017.   DOI
54 K. K. Tatsuoka, "Rule space: An approach for dealing with misconceptions based on item response theory," J. Educ. Meas., pp. 345-354, 1983.
55 G. L. Candell and F. Drasgow, "An iterative procedure for linking metrics and assessing item bias in item response theory," Appl. Psychol. Meas., vol. 12, no. 3, pp. 253-260, 1988.   DOI
56 I. W. Molenaar, "Some background for item response theory and the Rasch model," in Rasch models, Springer, 1995, pp. 3-14.
57 S. E. Embretson and S. P. Reise, Item response theory. Psychology Press, 2013.
58 T. Strachan, E. Ip, Y. Fu, T. Ackerman, S.-H. Chen, and J. Willse, "Robustness of projective IRT to misspecification of the underlying multidimensional model," Appl. Psychol. Meas., vol. 44, no. 5, pp. 362-375, 2020.   DOI
59 D. A. Almaleki, "Challenges Experienced Use of Distance-Learning by High School Teachers Responses to Students with Depression," Int. J. Comput. Sci. Netw. Secur., vol. 21, no. 5, pp. 192-198, May 2021, doi: 10.22937/IJCSNS.2021.21.5.27.   DOI
60 D. A. Almaleki, "The Psychometric Properties of Distance-Digital Subjective Happiness Scale," Int. J. Comput. Sci. Netw. Secur., vol. 21, no. 5, pp. 211-216, May 2021, doi: 10.22937/IJCSNS.2021.21.5.29.   DOI
61 J. J. Hox, C. J. Maas, and M. J. Brinkhuis, "The effect of estimation method and sample size in multilevel structural equation modeling," Stat. Neerlandica, vol. 64, no. 2, pp. 157-170, 2010.   DOI
62 R. K. Hambleton and R. W. Jones, "Comparison of classical test theory and item response theory and their applications to test development," Educ. Meas. Issues Pract., vol. 12, no. 3, pp. 38-47, 1993.