A new approach to computerized adaptive testing

978

Abstract

A new approach to computerized adaptive testing is presented on the basis of discrete-state discrete-time Markov processes. This approach is based on an extension of the G. Rasch model used in the Item Response Theory (IRT) and has decisive advantages over the adaptive IRT testing. This approach has a number of competitive advantages: takes into account all the observed history of performing test items that includes the distribution of successful and unsuccessful item solutions; incorporates time spent on performing test items; forecasts results in the future behavior of the subjects; allows for self-learning and changing subject abilities during a testing procedure; contains easily available model identification procedure based on simply accessible observation data. Markov processes and the adaptive transitions between the items remain hidden for the subjects who have access to the items only and do not know all the intrinsic mathematical details of a testing procedure. The developed model of adaptive testing is easily generalized for the case of polytomous items and multidimensional items and model structures.

General Information

Keywords: Markov processes, adaptive testing, IRT, computerized adaptive testing

Journal rubric: Research Methods

Article type: scientific article

DOI: https://doi.org/10.17759/exppsy.2017100303

For citation: Kuravsky L.S., Artemenkov S.L., Yuryev G.A., Grigorenko E.L. A new approach to computerized adaptive testing. Eksperimental'naâ psihologiâ = Experimental Psychology (Russia), 2017. Vol. 10, no. 3, pp. 33–45. DOI: 10.17759/exppsy.2017100303.

A Part of Article

Testing procedures are increasingly used in many contemporary applications requiring assessment of people or machine’s behavior. According to conventional models of testing based on classical test theory for measuring the examinee’s level in a specific skill or ability as preciselyas possible these procedures usually should implement a big number of items that makes testing difficult to use.

References

  1. Burden R.L., Faires J.D. Numerical Analysis. Brooks Cole, 2011. 895 p.
  2. Kohonen T. Self-Organizing Maps. Springer, 2001. 501 p.
  3. Kuravsky L. S., Marmalyuk P. A., Yuryev G. A., Dumin P. N. A Numerical Technique for the Identification of Discrete-State Continuous-Time Markov Models. Applied Mathematical Sciences, 2015, vol. 9, no. 8, pp. 379–391. URL: https://doi.org/10.12988/ams. 2015.410882.
  4. Kuravsky L.S., Margolis A.A., Marmalyuk P.A., Panfilova A.S., Yuryev G.A., Dumin P.N. A Probabilistic Model of Adaptive Training. Applied Mathematical Sciences, 2016, vol. 10, no. 48, pp. 2369—2380. http:// dx.doi.org/10.12988/ams.2016.65168.
  5. Kuravsky L.S., Margolis A.A., Marmalyuk P.A., Yuryev G.A., Dumin P.N. Trained Markov Models to Optimize the Order of Tasks in Psychological Testing. Neirokomp'yutery: Razrabotka, Primenenie [Neurocomputers: Development, Application], 2013, no. 4, pp. 28–38 (In Rus.).
  6. Kuravsky L.S., Marmalyuk P.A., Baranov S.N., Alkhimov V.I., Yuryev G.A., Artyukhina S.V. A New Technique for Testing Professional Skills and Competencies and Examples of its Practical Applications. Applied Mathematical Sciences, 2015, vol. 9, no. 21, pp. 1003—1026. https://doi.org/10.12988/ ams.2015.411899.
  7. Kuravsky L.S., Yuryev G.A. Primenenie markovskikh modeley dlya testirovania resultatov [Applying Markov Models to Processing Testing Results]. Voprosy Psychologii, 2011, no. 2, pp. 112—121, (In Rus.).
  8. Thompson N.A., Weiss D.J. A framework for the development of computerized adaptive tests. Practical Assessment. Research & Evaluation, 2011, vol. 16, no. 1, pp. 1—9.
  9. Torre J. de la, Patz R.J. Making the Most of What We Have: A Practical Application of Multidimensional Item Response Theory in Test Scoring. Journal of Educational and Behavioral Statistics, 2005, vol. 30, no. 3, pp. 295—311. doi:10.3102/10769986030003295.
  10. Wilkinson J.H. The Algebraic Eigenvalue Problem. Oxford, Clarendon Press, 1988, 662 p.

Information About the Authors

Lev S. Kuravsky, Doctor of Engineering, professor, Dean of the Computer Science Faculty, Moscow State University of Psychology and Education, Moscow, Russia, ORCID: https://orcid.org/0000-0002-3375-8446, e-mail: l.s.kuravsky@gmail.com

Sergei L. Artemenkov, PhD in Engineering, Professor, Head of the Department of Applied Informatics and Multimedia Technologies, Head of the Center of Information Technologies for Psychological Research of the Faculty of Information Technologies, Moscow State University of Psychology and Education, Moscow, Russia, ORCID: https://orcid.org/0000-0002-1619-2209, e-mail: slart@inbox.ru

Grigory A. Yuryev, PhD in Physics and Matematics, Associate Professor, Head of Department of the Computer Science Faculty, Leading Researcher, Youth Laboratory Information Technologies for Psychological Diagnostics, Moscow State University of Psychology and Education, Moscow, Russia, ORCID: https://orcid.org/0000-0002-2960-6562, e-mail: g.a.yuryev@gmail.com

Elena L. Grigorenko, PhD, Hugh Roy and Lillie Cranz Cullen Distinguished Professor of Psychology, University of Houston, Houston, TX, USA; Adjunct Senior Research Scientist, Moscow State University of Psychology and Education, Moscow, Russia; Professor and Acting Director, Center for Cognitive Sciences, Sirius University of Science and Technology, Federal territory "Sirius", Russia; Adjunct Professor, Child Study Center and Adjunct Senior Research Scientist, Haskins Laboratories, Yale University, New Haven, CT, USA; Research Certified Professor, Baylor College of Medicine, Member of the editorial boards of the journals “Clinical and Special Educatiom”, “Experimental Psychology” and “Psychological Science and Education”, Houston, USA, ORCID: https://orcid.org/0000-0001-9646-4181, e-mail: elena.grigorenko@times.uh.edu

Metrics

Views

Total: 1963
Previous month: 9
Current month: 5

Downloads

Total: 978
Previous month: 2
Current month: 3