Russian Psychological Issues PsyJournals.ru
OPEN ACCESS JOURNALS
JournalsTopicsAuthorsEditor's Choice Manuscript SubmissionAbout PsyJournals.ruContact Us

Digital Humanities and Technology in Education (DHTE 2021)

Publisher: Moscow State University of Psychology and Education

ISBN: 978-5-94051-240-0

Published in 2021

 

The Application of Item Response Theory (IRT) for AUDEE Scale Psychometric Assessment

Radchikova N.P.
PhD in Psychology, Associate Professor of the Department of Developmental Psychology, Faculty of Pre-School Pedagogy and Psychology, Moscow Pedagogical State University (MPGU), Moscow, Russia
ORCID: https://orcid.org/0000-0002-5139-8288
e-mail: nataly.radchikova@gmail.com

Sorokova M.G.
Doctor of Education, PhD in Physics and Mathematics, Head of Scientific and Practical Center for Comprehensive Support of Psychological Research «PsyDATA», Professor, Chair of Applied Mathematics, Faculty of Information Technology, Moscow State University of Psychology and Education, Moscow, Russia
ORCID: https://orcid.org/0000-0002-1000-6487
e-mail: sorokovamg@mgppu.ru

Odintsova M.A.
PhD in Psychology, Head of the Department of Psychology and Pedagogy of Distance Learning, Faculty of Distance Learning, Moscow State University of Psychology and Education, Moscow, Russia
ORCID: https://orcid.org/0000-0003-3106-4616
e-mail: mari505@mail.ru

Gusarova E.S.
MA in Psychology, Moderator, Scientific and Practical Center for Comprehensive Support of Psychological Research PsyDATA, MSUPE, Moscow State University of Psychology & Education, Moscow, Moscow, Russia
ORCID: https://orcid.org/0000-0001-6968-5253
e-mail: bondarevaes@fdomgppu.ru

Abstract
Using item response theory (IRT), we examine AUDEE Scale that measures six different aspects of university digital educational environment (DEE): DEE Learning Process Satisfaction, DEE Communication satisfaction and Learning Motivation, DEE Stress Tension, Need for Support in DEE Learning Activity, DEE Dishonest Strategy Prevalence, and DEE Accessibility. Five-point Likert scale is used for all 38 questions of the AUDEE Scale. The study involved 406 MSUPE students who completed e-courses in the online format (90.1 % female). The average age of the participants was 28,7 ± 9,6 years (median = 24 years, minimum = 19 years, maximum = 72 years). The application of the graded response model (GRM) separately to each of the six subscales of the AUDEE Scale showed that all the items have at least moderate item discrimination, and most of the items have high item discrimination. The threshold values for each question of the AUDEE Scale, corresponding to the relative indicator of difficulty, increase monotonically. Test information functions for each subscale of the AUDEE Scale indicate that almost all measurements of the components of the AUDEE Scale are quite accurate in the range of mean values, and only for the scale 6 “DEE Accessibility” the estimation accuracy is high for the lowest values and drops sharply for medium and high values.

Keywords: digital educational environment (DEE), AUDEE scale, Item Response Theory (IRT), item difficulty, item discrimination, test information function

Column: Modeling and Data Analysis for Digital Education

For Reference

Funding

The reported study was funded by the Moscow State Univer¬sity of Psychology and Education (MSUPE) in the framework of the re¬search project “Digital Technologies in Higher Education: Development of Technology for Individualizing Education Using E-Courses

References
  1. Avanesov V.S. Istoki i osnovnye ponyatiya matematicheskoi teorii pedagogicheskikh izmerenii (Item Response Theory) [The Origins and Basic Concepts of Item Response Theory] // Pedagogichesk­ie Izmereniya = Pedagogical measurements, 2007, no. 3, pp. 3–36. (In Russ.)
  2. Kroker L., Algina D. Vvedenie v klassicheskuyu i sovremennuyu te­oriyu testov [Introduction to classical and modern test theory]. Mos­cow: Logos, 2010. 668 p. (In Russ.)
  3. Neiman Yu.M., Khlebnikov V.A. Vvedenie v teoriyu modelirovaniya i parametrizatsii pedagogicheskikh testov [Introduction to the Theo­ry of Modeling and Parametrization of Educational Tests]. Moscow: Prometei, 2000. 168 p. (In Russ.)
  4. Sorokova M.G., Odintsova M.A., Radchikova N.P. Shkala otsenki tsifrovoi obrazovatel’noi sredy (TsOS) universiteta [Scale for Assess­ing University Digital Educational Environment (AUDEE Scale)] // Psikhologicheskaya nauka i obrazovanie = Psychological Science and Education, 2021, Vol. 26, no. 2, pp. 52–65. DOI: https://doi.org/10.17759/pse.2021260205 (In Russ., Abstr. in Engl.).
  5. Baker F.B. The Basis of Item Response Theory, 2nd ed. ERIC Clear­inghouse on Assessment and Evaluation, College Park, 2001. 186 p.
  6. Burnham K.P., Anderson D.R. Model selection and multimodel in­ference: A practical information-theoretic approach (2nd ed.). New York: Springer-Verlag, 2002. 488 p.
  7. Kang T., Chen T.T. Performance of the generalized S-X2 item fit in­dex for polytomous IRT models // Journal of Educational Measure­ment, 2008, no. 45 (4), pp. 391–406.
  8. Muraki E. A generalized partial credit model: Application of an EM algorithm. ETS Research Report Series, 1992. pp. 159–176.
  9. Orlando M., Thissen D. Likelihood-based item-fit indices for dichot­omous item response theory models // Applied Psychological Meas­urement, 2000, no. 24 (1), pp. 50–64.
  10. Samejima F. Graded Response Model. In W.J. van der Linden, & R.K. Hambleton (Eds.). Handbook of Modern Item Response The­ory. New York, NY: Springer, 1997. pp. 85–100.
  11. Sorokova, Marina; Odintsova, Maria; Radchikova, Nataly (2021): Scale for Assessing University Digital Educational Environment (AUDEE Scale): validation database. Psychological Research Data & Tools Repository. Dataset. https://doi.org/10.25449/ruspsyda­ta.14742816
 
About PsyJournals.ru

© 2007–2022 Portal of Russian Psychological Publications. All rights reserved

PsyJournals.ru in Russian

Publisher: Moscow State University of Psychology and Education

Catalogue of academic journals in psychology & education MSUPE

Creative Commons License Open Access Repository     Webometrics Ranking of Repositories

RSS Psyjournals at Youtube ??????.???????