Modeling and forecasting digital cognitive burnout in students based on multimodal time series and a cross-modal transformer architecture

The article discusses the problem of predicting digital cognitive burnout (CKD) in students, a dynamic psychological state formed in a digital educational environment. The problem is formalized as a multidimensional regression problem on multimodal time series, the purpose of which is to predict the trajectory of changes in indicators on the MBI-SS scale over a given time horizon. The author proposes a mathematical model based on a hierarchical cross-modal transformer architecture, including specialized modal encoders, a cross-modal attention mechanism, and a recurrent predictive decoder. To jointly solve the problems of regression (forecasting) and classification (determining the current level of burnout), a composite loss functional has been introduced that takes into account not only the accuracy of the forecast, but also the smoothness of its temporal dynamics, as well as the correctness of the classification. Experimental verification of the model on synthetic data demonstrated a decrease in the average absolute prediction error compared to the basic approaches and confirmed the interpretability of intermodal interactions based on the analysis of attention matrices. The results obtained form the formal computational basis for the development of early detection systems for the risk of digital cognitive burnout in educational platforms.

1. Выготский, Л. С. (1982). Проблемы общей психологии (Собр. соч., Т. 2). Москва: Педагогика.
   Vygotsky, L. S. (1982). Problemy obshchei psikhologii [Problems of general psychology] (Vol. 2). Pedagogika.
2. Карпов, А. В. (2021). Методологические основы психологического анализа информационной деятельности. Филигрань.
   Karpov, A. V. (2021). Metodologicheskie osnovy psikhologicheskogo analiza informatsionnoi deiatel'nosti [Methodological foundations of the psychological analysis of informational activity]. Filigran'.
3. Карташева, М. И. (2022) Роль системы Я в структуре ментальной регуляции психических состояний в процессе учебной деятельности // Ярославский педагогический вестник. 2022. № 1 (124). С. 162-174. http://dx.doi.org/10.20323/1813-145X-2022-1-124-162-174.
   Kartasheva, M. I. (2022). The role of the self-system in the structure of mental states regulation during the educational activities. Yaroslavl Pedagogical Bulletin, (1), 162–174. https://doi.org/10.20323/1813-145X-2022-1-124-162-174
4. Коровникова, Н.А. Искусственный интеллект в образовательном пространстве: проблемы и перспективы // Социальные новации и социальные науки. – Москва: ИНИОН РАН, 2021. – № 2. – С. 98–113
   Korovnikova, N.A. (2021). Artificial intelligence in the educational space: Problems and prospects. Social Innovations and Social Sciences, 2, 98–113. (In Russ.). Moscow: INION RAS.
5. Солдатова, Г.У. (2018). Цифровая социализация в культурно-исторической парадигме: изменяющийся ребенок в изменяющемся мире. Социальная психология и общество, 9(3), 71–80. https://doi.org/10.17759/sps.2018090308
   Soldatova, G.U. (2018). Digital socialization in the cultural-historical paradigm: A changing child in a changing world. Social Psychology and Society, 9(3), 71–80. (In Russ.). https://doi.org/10.17759/sps.2018090308
6. Юрьева, Н.Е. (2025). Искусственный интеллект в психодиагностике: когнитивные состояния в цифровой образовательной среде. Моделирование и анализ данных, 15(3), 47–55. https://doi.org/10.17759/mda.2025150303
   Yuryeva, N.E. (2025). Artificial intelligence in psychodiagnostics: cognitive states in a digital educational environment. Modelling and Data Analysis, 15(3), 47–55. (In Russ.). https://doi.org/10.17759/mda.2025150303
7. Baker, R. S. (2019). Challenges for the future of educational data mining: The Baker Learning Analytics Prizes. Journal of Educational Data Mining, 11(1), 1–17. https://doi.org/10.5281/zenodo.3554745
8. Baltrušaitis, T., Ahuja, C., & Morency, L.-P. (2019). Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2), 423–443. DOI: 10.1109/TPAMI.2018.2798607
9. D’Mello, S., & Kory, J. (2015). A review and meta‑analysis of multimodal affect detection systems. ACM Computing Surveys, 47(3), Article 43. https://doi.org/10.1145/2682899. Открытый PDF (author version): https://people.ict.usc.edu/~gratch/CSCI534/Readings/dmello-MMML-Survey.pdf
10. Grafsgaard, J. F., Wiggins, J. B., Boyer, K. E., Wiebe, E. N., & Lester, J. C. (2014). Predicting learning and affect from multimodal data streams in task-oriented tutorial dialogue. In J. Stamper, Z. Pardос, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (EDM 2014) (pp. 122–129). International Educational Data Mining Society. https://learndialogue.org/pdf/LearnDialogue-Grafsgaard-EDM-2014.pdf
11. Hamaker, E. L., Kuiper, R. M., & Grasman, R. P. P. P. (2015). A critique of the cross-lagged panel model. Psychological Methods, 20(1), 102–116. https://doi.org/10.1037/a0038889
12. He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) (pp. 1026–1034). https://doi.org/10.1109/ICCV.2015.123
13. Jain, S., & Wallace, B. C. (2019). Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (pp. 3543–3556). https://arxiv.org/abs/1902.10186
14. Kuppens, P., & Verduyn, P. (2017). Emotion dynamics. Current Opinion in Psychology, 17, 22–26. https://doi.org/10.1016/j.copsyc.2017.06.004
15. Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019. https://doi.org/10.48550/arXiv.1711.05101
16. Reis, R. C. D., Isotani, S., Rodriguez, C. L., Jaques, P. A., & Bittencourt, I. I. (2018). Affective states in computer‑supported collaborative learning: Studying the past to drive the future. Computers & Education, 120, 29–50. https://doi.org/10.1016/j.compedu.2018.01.015. Открытый PDF: https://isiarticles.com/bundles/Article/pre/pdf/126884.pdf
17. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30 (pp. 5998–6008). Curran Associates, Inc. URL: https://arxiv.org/abs/1706.03762
18. Wang, J., Yu, Y. (2025). Machine learning approach to student performance prediction of online learning. PLOS ONE, 20(1), e0299018. https://doi.org/10.1371/journal.pone.0299018