A new approach to computerized adaptive testing

A new approach to computerized adaptive testing is presented on the basis of discrete-state discrete-time Markov processes. This approach is based on an extension of the G. Rasch model used in the Item Response Theory (IRT) and has decisive advantages over the adaptive IRT testing. This approach has a number of competitive advantages: takes into account all the observed history of performing test items that includes the distribution of successful and unsuccessful item solutions; incorporates time spent on performing test items; forecasts results in the future behavior of the subjects; allows for self-learning and changing subject abilities during a testing procedure; contains easily available model identification procedure based on simply accessible observation data. Markov processes and the adaptive transitions between the items remain hidden for the subjects who have access to the items only and do not know all the intrinsic mathematical details of a testing procedure. The developed model of adaptive testing is easily generalized for the case of polytomous items and multidimensional items and model structures.

Testing procedures are increasingly used in many contemporary applications requiring assessment of people or machine’s behavior. According to conventional models of testing based on classical test theory for measuring the examinee’s level in a specific skill or ability as preciselyas possible these procedures usually should implement a big number of items that makes testing difficult to use.

1. Burden R.L., Faires J.D. Numerical Analysis. Brooks Cole, 2011. 895 p.
2. Kohonen T. Self-Organizing Maps. Springer, 2001. 501 p.
3. Kuravsky L. S., Marmalyuk P. A., Yuryev G. A., Dumin P. N. A Numerical Technique for the Identification of Discrete-State Continuous-Time Markov Models. Applied Mathematical Sciences, 2015, vol. 9, no. 8, pp. 379–391. URL: https://doi.org/10.12988/ams. 2015.410882.
4. Kuravsky L.S., Margolis A.A., Marmalyuk P.A., Panfilova A.S., Yuryev G.A., Dumin P.N. A Probabilistic Model of Adaptive Training. Applied Mathematical Sciences, 2016, vol. 10, no. 48, pp. 2369—2380. http:// dx.doi.org/10.12988/ams.2016.65168.
5. Kuravsky L.S., Margolis A.A., Marmalyuk P.A., Yuryev G.A., Dumin P.N. Trained Markov Models to Optimize the Order of Tasks in Psychological Testing. Neirokomp'yutery: Razrabotka, Primenenie [Neurocomputers: Development, Application], 2013, no. 4, pp. 28–38 (In Rus.).
6. Kuravsky L.S., Marmalyuk P.A., Baranov S.N., Alkhimov V.I., Yuryev G.A., Artyukhina S.V. A New Technique for Testing Professional Skills and Competencies and Examples of its Practical Applications. Applied Mathematical Sciences, 2015, vol. 9, no. 21, pp. 1003—1026. https://doi.org/10.12988/ ams.2015.411899.
7. Kuravsky L.S., Yuryev G.A. Primenenie markovskikh modeley dlya testirovania resultatov [Applying Markov Models to Processing Testing Results]. Voprosy Psychologii, 2011, no. 2, pp. 112—121, (In Rus.).
8. Thompson N.A., Weiss D.J. A framework for the development of computerized adaptive tests. Practical Assessment. Research & Evaluation, 2011, vol. 16, no. 1, pp. 1—9.
9. Torre J. de la, Patz R.J. Making the Most of What We Have: A Practical Application of Multidimensional Item Response Theory in Test Scoring. Journal of Educational and Behavioral Statistics, 2005, vol. 30, no. 3, pp. 295—311. doi:10.3102/10769986030003295.
10. Wilkinson J.H. The Algebraic Eigenvalue Problem. Oxford, Clarendon Press, 1988, 662 p.

PlumX

article metrics