Academic Outcomes of Students in University Digital Environment at Different Levels of Higher Education: Who Is More Successful?

The problem of empirical assessment of various aspects of learning in the digital educational environment seems to be of particular relevance.At the same time, it is emphasized that there is a lack of comparative studies of students’ educational outcomes at different levels of higher education who have completed e-courses.The study was conducted at the Moscow State University of Psychology and Education (MSUPE), the sample size is N = 424 students.Subject of the study is immediate and long-term academic achievements of students in two levels of higher education who completed e-courses.Purpose of the study is to evaluate the differences in the academic achievements and knowledge retention of graduate and undergraduate students.Key findings of students’ academic outcomes comparative analysis are the following: (1) No differences were found between graduate and undergraduate students in the pretest, final test and overall e-course grade indicators.(2) The same tendency was revealed in students of both groups: pretest scores are low, posttest scores significantly and strongly increase, and then after 1.5—4 months they significantly decrease, while remaining significantly higher than the input indicators.The knowledge retention scores are very scattered in comparison with the direct ones.(3) The gain score effect size and the improvement index are significant for the final test only without adjusting for clustering, i.e.ignoring the fact that the sample consists of several student groups.A median graduate student would have a higher score than a median undergraduate student.Cluster-level effect size is not statistically significant.Cluster-level effect size for overall e-course grade indicators with difference-in-differences adjustment is also not reliable.(4) The knowledge retention scores in both students’ categories do not differ.The gain score effect sizes for knowledge retention, taking into account both the final test and the pretest, are not significant.(5) The psychometric characteristics of the academic achievement test in the field of empirical data quantitative analysis can be considered satisfactory.

1. Gosudarstvennaya programma Rossiiskoi Federatsii «Razvitie obrazovaniya» [The state program of the Russian Federation “Development of education”].Bank dokumentov.Ministerstvo prosveshcheniya Rossiiskoi Federatsii [Bank of documents.Ministry of Education of the Russian Federation].Available at: https://docs.edu.gov.ru/document/3a928e13b4d29 2f8f71513a2c02086a3/ (Accessed: 05.06.2020).(In Russ.).
2. Dmitrievskaya N.A., Goremykina G.I.Modelirovanie sistemy upravleniya po rezul’tatam deyatel’nosti smart-universiteta v usloviyakh tsifrovizatsii ekonomiki i obshchestva [Modeling a management system based on the results of a smart university in the context of digitalization of the economy and society].Materialy mezhdunarodnoi konferentsii “E-Learning Stakeholders and Researchers Summit” (g.Moskva, 5—6 dekabrya 2018 g.) [Materials of the international conference “E-Learning Stakeholders and Researchers Summit”].Moscow: Publ.Izd.dom Vysshei shkoly ekonomiki, 2018, pp.39—46.(In Russ.).
3. Margolis A.A.Chto smeshivaet smeshannoe obuchenie? [What Kind of Blending Makes Blended Learning?].Psikhologicheskaya nauka i obrazovanie = Psychological Science and Education, 2018.Vol.23, no.3, pp.5—19.DOI:10.17759/pse.2018230301 (In Russ.).
4. Sorokova M.G.Matematicheskie metody v psikhologo-pedagogicheskikh issledovaniyakh: Uchebnoe posobie [Mathematical methods in psycho-educational researches].Moscow: Publ.Neolit, 2020.216 p.DOI:10.17759/psychlib/978-5-6043562-0-3 (In Russ.).
5. Sorokova M.G.Digital Educational Environment in University: Who is More Comfortable Studying in It? Psikhologicheskaya nauka i obrazovanie = Psychological Science and Education, 2020.Vol.25, no.2, pp.44—58.DOI:10.17759/pse.2020250204 (In Russ.).
6. Sorokova M.G.Elektronnyi kurs kak tsifrovoi obrazovatel’nyi resurs smeshannogo obucheniya v usloviyakh vysshego obrazovaniya [E-Course as Blended Learning Digital Educational Resource in University].Psikhologicheskaya nauka i obrazovanie = Psychological Science and Education, 2020.Vol.25, no.1, pp.36—50.DOI:10.17759/pse.2020250104 (In Russ.).
7. Chkhutiashvili L.V.Gosudarstvennaya politika v sfere onlain-obrazovaniya [State policy in the field of online-education].Materialy mezhdunarodnoi konferentsii “E-Learning Stakeholders and Researchers Summit” (g.Moskva, 5—6 dekabrya 2018 g.) [Materials of the international conference “E-Learning Stakeholders and Researchers Summit”].Moscow: Publ.Izd.dom Vysshei shkoly ekonomiki, 2018, pp.18—21.(In Russ.).
8. Akcayır G., Akcayır M.The flipped classroom: A review of its advantages and challenges.Computers & Education, Vol.126, pp.334—345.DOI:10.1016/j.compedu.2018.07.021
9. Arıf S., Omar İ.Effectiveness of Flipped Classroom in Teaching Basic English Courses.Yükseköğretim Dergisi, 2019.Vol.9, no.3, pp.279—289.DOI:10.2399/ yod.19.003
10. Bouwmeestera R.A.M., de Kleijnb R.A.M., van den Bergc I.E.T., ten Cated O.Th.J., van Rijena H.V.M., Westervelde H.E.Flipping the medical classroom: Effect on workload, interactivity, motivation and retention of knowledge.Computers & Education, 2019.Vol.139, pp. 118—128.DOI:10.1016/j.compedu.2019.05.002
11. Chen F., Lui A.M., Martinelli S.M.A systematic review of the effectiveness of flipped classrooms in medical education.Medical Education, 2017.Vol.51, no.6, pp.585—597.DOI:10.1111/medu.13272
12. Chen K.S., Monrouxe L., Lu Y.H., Jenq C.C., Chang Y.J., Chang Y.C.et al.Academic outcomes of flipped classroom learning: A meta-analysis.Medical Education, 2018.Vol.52, no.9, pp.910—924.DOI:10.1111/medu.13616
13. De Kleijn R.A.M., Bouwmeester R.A.M., Ritzen M.M.J., Ramaekers S.P.J., Van Rijen H.V.M.Students’ motives for using online formative assessments when preparing for summative assessments.Medical Teacher, 2013.Vol.35, no.12, pp.1—7.DOI:10.3109/0 142159X.2013.826794
14. Hew K.F., Lo C.K.Flipped classroom improves student learning in health professions education: a meta-analysis.BMC Med Educ, 2018.Vol.18, 38.DOI:10.1186/s12909-018-1144-z
15. IES What Works Clearinghouse.Handbooks and other resources.Available at: https://ies.ed.gov/ncee/ wwc/Handbooks (Accessed 05.06.2020).
16. Morton D.A., Colbert-Getz J.M.Measuring the impact of the flipped anatomy classroom: The importance of categorizing an assessment by Bloom’s taxonomy.Anatomical Sciences Education, 2017.Vol. 10, no.2, pp.170—175.DOI:10.1002/ase.1635
17. Prinsloo P.Big data in education.The digital future of learning, policy and practice.International Studies in Sociology of Education, 2020.Vol.29, no. 1—2, pp. 183—186.DOI:10.1080/09620214.201 9.1690546
18. Rajaram K.Flipped classrooms: Scaffolding support system with real-time learning interventions.Asian Journal of the Scholarship of Teaching and Learning, 2019.Vol.9, no.1, pp.30—58.Available at: http:// www.nus.edu.sg/cdtl/engagement/publications/ajsotl-home/archive-of-past-issues/v9n1/flipped-classrooms-providing-a-scaffolding-support-system-with-real-time-learning-interventions (Accessed 02.04.2020).
19. Røe Y., Rowe M., Ødegaard N.B.et al.Learning with technology in physiotherapy education: design, implementation and evaluation of a flipped classroom teaching approach.BMC Med Educ, Vol.19, 291.DOI:10.1186/s12909-019-1728-2
20. Sorokova M.(2020) “Educational outcomes of graduate and undergraduate students who completed e-courses in mathematical methods in psychological and educational researches”, Mendeley Data, v1 http:// dx.doi.org/10.17632/hvfkdpfwnr.1
21. Wang K., Zhu C.MOOC-based flipped learning in higher education: students’ participation, experience and learning performance.Int J Educ Technol High Educ, 2019.Vol.16, 33.DOI:10.1186/s41239-019- 0163-0
22. Williamson B.Making markets through digital platforms: Pearson, edu-business, and the (e)valuation of higher education.Critical Studies in Education, 2020.DOI:10.1080/17508487.2020.1737556