International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

The Effect of Methods of Estimating the Ability on The Accuracy and Items Parameters According to 3PL Model

  • Almaleki, Deyab A. (Department of Evaluation, Measurement and Research, Umm Al-Qura University) ;
  • Alomrany, Ahoud Ghazi (Department of Evaluation, Measurement and Research, Umm Al-Qura University)
  • Received : 2021.07.05
  • Published : 2021.07.30
Download PDF
⟨ Previous Next ⟩

Abstract

This study aimed to test method on the accuracy of estimating the items parameters and ability, using the Three Parameter Logistic. To achieve the objectives of the study, an achievement test in chemistry was constructed for third-year secondary school students in the course of "natural sciences". A descriptive approach was employed to conduct the study. The test was applied to a sample of (507) students of the third year of secondary school in the "Natural Sciences Course". The study's results revealed that the (EAP) method showed a higher degree of accuracy in the estimation of the difficulty parameter and the abilities of persons higher than the MML method. There were no statistically significant differences in the accuracy of the parameter estimation of discrimination and guessing regarding the difference of the two methods: (MML) and (EAP).

Keywords

References

  1. S.-S. Lee and J. Kim, "An Exploratory study on Student-Intelligent Robot Teacher relationship recognized by Middle School Students," J. Digit. Converg., vol. 18, no. 4, pp. 37-44, 2020. https://doi.org/10.14400/JDC.2020.18.4.037
  2. S. Tibi, A. A. Edwards, C. Schatschneider, L. J. Lombardino, J. R. Kirby, and S. H. Salha, "IRT analyses of Arabic letter knowledge in Kindergarten," Read. Writ., pp. 1-26, 2020.
  3. R. K. Hambleton and W. J. Van der Linden, Advances in item response theory and applications: An introduction. Sage Publications Sage CA: Thousand Oaks, CA, 1982.
  4. F. M. Lord, Applications of item response theory to practical testing problems. Routledge, 2012.
  5. W. Ma, N. Minchen, and J. de la Torre, "Choosing between CDM and unidimensional IRT: The proportional reasoning test case," Meas. Interdiscip. Res. Perspect., vol. 18, no. 2, pp. 87-96, 2020. https://doi.org/10.1080/15366367.2019.1697122
  6. G. C. Foster, H. Min, and M. J. Zickar, "Review of item response theory practices in organizational research: Lessons learned and paths forward," Organ. Res. Methods, vol. 20, no. 3, pp. 465-486, 2017. https://doi.org/10.1177/1094428116689708
  7. R. Liu, A. C. Huggins-Manley, and O. Bulut, "Retrofitting diagnostic classification models to responses from IRT-based assessment forms," Educ. Psychol. Meas., vol. 78, no. 3, pp. 357-383, 2018. https://doi.org/10.1177/0013164416685599
  8. C. L. Azevedo, D. F. Andrade, and J.-P. Fox, "A Bayesian generalized multiple group IRT model with model-fit assessment tools," Comput. Stat. Data Anal., vol. 56, no. 12, pp. 4399-4412, 2012. https://doi.org/10.1016/j.csda.2012.03.017
  9. G. Rajlic, "Violations of unidimensionality and local independence in measures intended as unidimensional: assessing levels of violations and the accuracy in unidimensional IRT model estimates," PhD Thesis, University of British Columbia, 2019.
  10. C. DeMars, "Group differences based on IRT scores: Does the model matter?," Educ. Psychol. Meas., vol. 61, no. 1, pp. 60-70, 2001. https://doi.org/10.1177/00131640121971068
  11. J. Chen and J. Choi, "A comparison of maximum likelihood and expected a posteriori estimation for polychoric correlation using Monte Carlo simulation," J. Mod. Appl. Stat. Methods, vol. 8, no. 1, p. 32, 2009.
  12. K. Matlock Cole and I. Paek, "PROC IRT: A SAS procedure for item response theory," Appl. Psychol. Meas., vol. 41, no. 4, pp. 311-320, 2017. https://doi.org/10.1177/0146621616685062
  13. D. Almaleki, "Examinee Characteristics and their Impact on the Psychometric Properties of a Multiple Choice Test According to the Item Response Theory (IRT)," Eng. Technol. Appl. Sci. Res., vol. 11, no. 2, pp. 6889-6901, 2021. https://doi.org/10.48084/etasr.4056
  14. D. Almaleki, "Empirical Evaluation of Different Features of Design in Confirmatory Factor Analysis," 2016.
  15. K. K. Tatsuoka, "Rule space: An approach for dealing with misconceptions based on item response theory," J. Educ. Meas., pp. 345-354, 1983.
  16. M. Wu and R. Adams, Applying the Rasch model to psychosocial measurement: A practical approach. Educational Measurement Solutions Melbourne, 2007.
  17. G. H. Fischer and I. W. Molenaar, Rasch models: Foundations, recent developments, and applications. Springer Science & Business Media, 2012.
  18. B. Zhuang, S. Wang, S. Zhao, and M. Lu, "Computed tomography angiography-derived fractional flow reserve (CT-FFR) for the detection of myocardial ischemia with invasive fractional flow reserve as reference: systematic review and meta-analysis," Eur. Radiol., vol. 30, no. 2, pp. 712-725, 2020. https://doi.org/10.1007/s00330-019-06470-8
  19. D. R. Divgi, "A minimum chi-square method for developing a common metric in item response theory," Appl. Psychol. Meas., vol. 9, no. 4, pp. 413-415, 1985. https://doi.org/10.1177/014662168500900410
  20. F. M. Lord, "Maximum likelihood and Bayesian parameter estimation in item response theory," J. Educ. Meas., pp. 157-162, 1986.
  21. G. L. Candell and F. Drasgow, "An iterative procedure for linking metrics and assessing item bias in item response theory," Appl. Psychol. Meas., vol. 12, no. 3, pp. 253-260, 1988. https://doi.org/10.1177/014662168801200304
  22. G. J. Mellenbergh, "Item bias and item response theory," Int. J. Educ. Res., vol. 13, no. 2, pp. 127-143, 1989. https://doi.org/10.1016/0883-0355(89)90002-5
  23. C. J. Maas and J. J. Hox, "Sufficient sample sizes for multilevel modeling," Methodology, vol. 1, no. 3, pp. 86-92, 2005. https://doi.org/10.1027/1614-2241.1.3.86
  24. I. Paek, M. Cui, N. Ozturk Gubes, and Y. Yang, "Estimation of an IRT model by Mplus for dichotomously scored responses under different estimation methods," Educ. Psychol. Meas., vol. 78, no. 4, pp. 569-588, 2018. https://doi.org/10.1177/0013164417715738
  25. L. R. Bonetto, J. S. Crespo, R. Guegan, V. I. Esteves, and M. Giovanela, "Removal of methylene blue from aqueous solutions using a solid residue of the apple juice industry: full factorial design, equilibrium, thermodynamics and kinetics aspects," J. Mol. Struct., vol. 1224, p. 129296, 2021. https://doi.org/10.1016/j.molstruc.2020.129296
  26. K. M. Marcoulides, N. Foldnes, and S. Gronneberg, "Assessing model fit in structural equation modeling using appropriate test statistics," Struct. Equ. Model. Multidiscip. J., vol. 27, no. 3, pp. 369-379, 2020. https://doi.org/10.1080/10705511.2019.1647785
  27. I. W. Molenaar, "Some background for item response theory and the Rasch model," in Rasch models, Springer, 1995, pp. 3-14.
  28. C. E. Cantrell, "Item Response Theory: Understanding the One-Parameter Rasch Model.," 1997.
  29. C. Magno, "Demonstrating the difference between classical test theory and item response theory using derived test data," Int. J. Educ. Psychol. Assess., vol. 1, no. 1, pp. 1-11, 2009.
  30. S. E. Embretson and S. P. Reise, Item response theory. Psychology Press, 2013.
  31. T. Strachan et al., "Using a Projection IRT Method for Vertical Scaling When Construct Shift Is Present," J. Educ. Meas., 2020.
  32. W.-C. Lee, S. Y. Kim, J. Choi, and Y. Kang, "IRT Approaches to Modeling Scores on Mixed-Format Tests," J. Educ. Meas., vol. 57, no. 2, pp. 230-254, 2020. https://doi.org/10.1111/jedm.12248
  33. T. Strachan, E. Ip, Y. Fu, T. Ackerman, S.-H. Chen, and J. Willse, "Robustness of projective IRT to misspecification of the underlying multidimensional model," Appl. Psychol. Meas., vol. 44, no. 5, pp. 362-375, 2020. https://doi.org/10.1177/0146621620909894
  34. D. R. Crisan, J. N. Tendeiro, and R. R. Meijer, "Investigating the practical consequences of model misfit in unidimensional IRT models," Appl. Psychol. Meas., vol. 41, no. 6, pp. 439-455, 2017. https://doi.org/10.1177/0146621617695522
  35. M. N. Morshed, M. N. Pervez, N. Behary, N. Bouazizi, J. Guan, and V. A. Nierstrasz, "Statistical modeling and optimization of heterogeneous Fenton-like removal of organic pollutant using fibrous catalysts: a full factorial design," Sci. Rep., vol. 10, no. 1, pp. 1-14, 2020. https://doi.org/10.1038/s41598-019-56847-4
  36. M. L. Stocking and F. M. Lord, "Developing a common metric in item response theory," Appl. Psychol. Meas., vol. 7, no. 2, pp. 201-210, 1983. https://doi.org/10.1177/014662168300700208
  37. H. Swaminathan, R. K. Hambleton, and J. Algina, "Reliability of criterion-referenced tests: A decision-theoretic formulation," J. Educ. Meas., vol. 11, no. 4, pp. 263-267, 1974. https://doi.org/10.1111/j.1745-3984.1974.tb00998.x
  38. R. K. Hambleton and H. J. Rogers, "Detecting potentially biased test items: Comparison of IRT area and Mantel-Haenszel methods," Appl. Meas. Educ., vol. 2, no. 4, pp. 313-334, 1989. https://doi.org/10.1207/s15324818ame0204_4
  39. H. Kishino, T. Miyata, and M. Hasegawa, "Maximum likelihood inference of protein phylogeny and the origin of chloroplasts," J. Mol. Evol., vol. 31, no. 2, pp. 151-160, 1990. https://doi.org/10.1007/BF02109483
  40. R. G. Lim and F. Drasgow, "Evaluation of two methods for estimating item response theory parameters when assessing differential item functioning.," J. Appl. Psychol., vol. 75, no. 2, p. 164, 1990. https://doi.org/10.1037//0021-9010.75.2.164
  41. H. Kishino and M. Hasegawa, "Evaluation of the maximum likelihood estimate of the evolutionary tree topologies from DNA sequence data, and the branching order in Hominoidea," J. Mol. Evol., vol. 29, no. 2, pp. 170-179, 1989. https://doi.org/10.1007/BF02100115
  42. H. Swaminathan, R. K. Hambleton, and H. J. Rogers, "21 Assessing the Fit of Item Response Theory Models," Handb. Stat., vol. 26, pp. 683-718, 2006. https://doi.org/10.1016/S0169-7161(06)26021-8
  43. H. Swaminathan, R. K. Hambleton, S. G. Sireci, D. Xing, and S. M. Rizavi, "Small sample estimation in dichotomous item response models: Effect of priors based on judgmental information on the accuracy of item parameter estimates," Appl. Psychol. Meas., vol. 27, no. 1, pp. 27-51, 2003. https://doi.org/10.1177/0146621602239475
  44. R. K. Hambleton, H. Swaminathan, and H. J. Rogers, Fundamentals of item response theory, vol. 2. Sage, 1991.
  45. C. S. Wardley, E. B. Applegate, A. D. Almaleki, and J. A. Van Rhee, "A comparison of Students' perceptions of stress in parallel problem-based and lecture-based curricula," J. Physician Assist. Educ., vol. 27, no. 1, pp. 7-16, 2016. https://doi.org/10.1097/JPA.0000000000000060
  46. D. Almaleki, "Stability of the Data-Model Fit over Increasing Levels of Factorial Invariance for Different Features of Design in Factor Analysis," Eng. Technol. Appl. Sci. Res., vol. 11, no. 2, pp. 6849-6856, 2021. https://doi.org/10.48084/etasr.4047
  47. D. Almaleki, "The Precision of the Overall Data-Model Fit for Different Design Features in Confirmatory Factor Analysis," Eng. Technol. Appl. Sci. Res., vol. 11, no. 1, pp. 6766-6774, 2021. https://doi.org/10.48084/etasr.4025
  48. C. S. Wardley, E. B. Applegate, A. D. Almaleki, and J. A. Van Rhee, "Is Student Stress Related to Personality or Learning Environment in a Physician Assistant Program?," J. Physician Assist. Educ., vol. 30, no. 1, pp. 9-19, 2019. https://doi.org/10.1097/JPA.0000000000000241
  49. J. Y. Park, F. Cornillie, H. L. van der Maas, and W. Van Den Noortgate, "A multidimensional IRT approach for dynamically monitoring ability growth in computerized practice environments," Front. Psychol., vol. 10, p. 620, 2019. https://doi.org/10.3389/fpsyg.2019.00620
  50. G. J. Mellenbergh, "Item bias and item response theory," Int. J. Educ. Res., vol. 13, no. 2, pp. 127-143, 1989. https://doi.org/10.1016/0883-0355(89)90002-5
  51. R. K. Hambleton and A. Kanjee, "Increasing the validity of cross-cultural assessments: Use of improved methods for test adaptations," Eur. J. Psychol. Assess., vol. 11, no. 3, pp. 147-157, 1995. https://doi.org/10.1027/1015-5759.11.3.147
  52. R. K. Hambleton, W. J. van der Linden, and C. S. Wells, "IRT models for the analysis of polytomously scored data," Handb. Polytomous Item Response Theory Models, pp. 21-42, 2010.
  53. M. A. Tanner and W. H. Wong, "The calculation of posterior distributions by data augmentation," J. Am. Stat. Assoc., vol. 82, no. 398, pp. 528-540, 1987. https://doi.org/10.1080/01621459.1987.10478458
  54. E.-Y. Mun, Y. Huo, H. R. White, S. Suzuki, and J. de la Torre, "Multivariate higher-order IRT model and MCMC algorithm for linking individual participant data from multiple studies," Front. Psychol., vol. 10, p. 1328, 2019. https://doi.org/10.3389/fpsyg.2019.01328
  55. S.-K. Chen, L. Hou, and B. G. Dodd, "A comparison of maximum likelihood estimation and expected a posteriori estimation in CAT using the partial credit model," Educ. Psychol. Meas., vol. 58, no. 4, pp. 569-595, 1998. https://doi.org/10.1177/0013164498058004002
  56. D. A. Almaleki, W. W. Khayat, T. F. Yally, and A. A. Alhajjaji, "The Effectiveness of the Use of DistanceEvaluation Tools and Methods among Students with Learning-Difficulties from the Teachers' Point of View," Int. J. Comput. Sci. Netw. Secur., vol. 21, no. 5, pp. 243-255, May 2021, doi: 10.22937/IJCSNS.2021.21.5.34.
  57. D. A. Almaleki, R. A. Alhajaji, and M. A. Alharbi, "Measuring Students' Interaction in Distance Learning Through the Electronic Platform and its Impact on their Motivation to Learn During Covid-19 Crisis," Int. J. Comput. Sci. Netw. Secur., vol. 21, no. 5, pp. 98-112, May 2021, doi: 10.22937/IJCSNS.2021.21.5.16.
  58. D. A. Almaleki, "The Psychometric Properties of Distance-Digital Subjective Happiness Scale," Int. J. Comput. Sci. Netw. Secur., vol. 21, no. 5, pp. 211-216, May 2021, doi: 10.22937/IJCSNS.2021.21.5.29.
  59. D. A. Almaleki, "Challenges Experienced Use of Distance-Learning by High School Teachers Responses to Students with Depression," Int. J. Comput. Sci. Netw. Secur., vol. 21, no. 5, pp. 192-198, May 2021, doi: 10.22937/IJCSNS.2021.21.5.27.
  60. A. Preti, M. Vellante, and D. R. Petretto, "The psychometric properties of the 'Reading the Mind in the Eyes' Test: an item response theory (IRT) analysis," Cognit. Neuropsychiatry, vol. 22, no. 3, pp. 233-253, 2017. https://doi.org/10.1080/13546805.2017.1300091
  61. J. J. Hox, C. J. Maas, and M. J. Brinkhuis, "The effect of estimation method and sample size in multilevel structural equation modeling," Stat. Neerlandica, vol. 64, no. 2, pp. 157-170, 2010. https://doi.org/10.1111/j.1467-9574.2009.00445.x
  62. R. K. Hambleton and R. W. Jones, "Comparison of classical test theory and item response theory and their applications to test development," Educ. Meas. Issues Pract., vol. 12, no. 3, pp. 38-47, 1993.