Evaluation of texts complexity and interest to them in Internet

Context and Relevance. Human understanding of texts is becoming increasingly important, especially in the context of learning at all types and levels, understanding instructions, jurisprudence, and various online communications. Objective: to develop an adequate system for assessing text complexity and propose a pilot experiment design that takes into account the factor of interest in it. Hypothesis. 1) The number of syntactic relations per sentence determines the subjective assessment of text complexity. 2) Text complexity and interest in it are negatively related. Methods and materials. A method for assessing text complexity is proposed based on nine statistical indicators — four original ones: the number of letters, words, syntactic relations (‘rels’), and sentences, and five derivatives from them. An experimental design was developed that includes a subjective assessment of text complexity and interest in it. Texts from four authors from Yandex Zen were used for the pilot study. From these, eight stimuli were obtained by either simplifying or complicating them. Simplification consisted of reducing the number of syntactic relations per sentence, while complexity consisted of increasing it. The stimulus length was approximately 1000 characters. Texts and assessment tools were presented online on a special temporary website. Participation was completely anonymous; responses from 33 unique IP addresses were recorded. Results. Overall, increasing the number of syntactic entities per sentence leads to subjective text complexity, although vocabulary, subject matter, and the presence of specific syntactic entities in the text play a significant role. Conclusions. The proposed method appears promising for studying resource management, attention, and concentration processes, provided that variables are controlled in a laboratory setting and psychophysiological methods such as eye tracking and EEG are incorporated.

1. Алмаев, Н.А., Бессонова, Ю.В., Мурашева, О.В. (2020). Текст. Взгляд. Мотивация. Методические вопросы. М.: Изд-во ИП РАН.
   Almaev, N.A., Bessonova, Yu.V., Murasheva, O.V. (2020). View. Motivation. Methodological issues. Moscow: Publishing house of the Institute of Sociology of the Russian Academy of Sciences. (In Russ.)
2. Алмаев, Н.А., Мурашева, О.В., Петрович, Д.Л. (2020). ЭЭГ корреляты личностных опросников: ОФДСИ Русалова, ОСТ-77 Трофимовой и ТТХ-140 Клонинджера. Вестник психофизиологии, 3, 135—138.
   Almaev, N.A., Murasheva, O.V., Petrovich, D.L. (2020). EEG correlates of personality questionnaires: Rusalov's OFDSI, Trofimova's OST-77, and Cloninger's TTX-140. Bulletin of Psychophysiology, 3, 135—138. (In Russ.)
3. Вахрушева, А.Я., Солнышкина, М.И., Куприянов, Р.В., Гафиятова, Э.В., Климагина, И.О. (2021). Лингвистическая сложность учебных текстов. Вопросы журналистики, педагогики, языкознания, 40(1), 89—99. 
   Vahrusheva, A.Ya., Solnyshkina, M.I., Kupriyanov, R.V., Gafiyatova, E.V., Klimagina, I.O. (2021). Linguistic Complexity of Educational Texts. Issues of Journalism, Pedagogy, and Linguistics, 40(1), 89—99. (In Russ.)
4. Виноградова, О.А., Смирнова, Е.А., Вилкова, А.В., Пантелеева, И.М. (2020). Синтаксическая сложность академического текста: корпусное исследование работ русскоязычных студентов, изучающих английский язык, в сравнении с текстами профессиональных авторов. Вестник РГГУ. Серия «Литературоведение. Языкознание. Культурология»,
   Vinogradova, O.A, Smirnova, E.A, Vilkova, A.V., Panteleeva, I.M. (2020). Syntactic Complexity of Academic Texts: A Corpus-Based Study of Works by Russian-Speaking Students Learning English in Comparison with Texts by Professional Authors. RSUH Bulletin. Series: Literary Studies. Linguistics. Cultural Studies, 7. (In Russ.)
5. Девяткин, Д.А., Ениколопов, С.Н., Салимовский, В.А., Чудова, Н.В. (2021). Речевые реакции на фрустрацию: автоматическая категоризация. Психологические исследования, 14(78).
   Devyatkin, D.A., Enikolopov, S.N., Salimovskij, V.A., Chudova, N.V. (2021). Speech frustration responses: an automatic categorization. Psychological research, 14(78). (In Russ.).
6. Ениколопов, С.Н., Кузнецова, Ю.М., Смирнов, И.В., Станкевич, М.А., Чудова, Н.В. (2019a). Создание инструмента автоматического анализа текста в интересах социо-гуманитарных исследований. Часть 1. Методические и методологические аспекты. Искусственный интеллект и принятие решений, 2, 28—38.
   Enikolopov, S.N., Kuznetsova, Yu.M., Smirnov, I.V., Stankevich, M.A., Chudova, N.V. (2019a). Creating an Automatic Text Analysis Tool for Social Sciences and Humanities Research. Part 1. Methodological Aspects. Artificial Intelligence and Decision Making, 2, 28—38. (In Russ.)
7. Ениколопов, С.Н., Ковалев, А.К., Кузнецова, Ю.М., Чудова, Н.В., Старостина, Е.В. (2019б). Признаки, характерные для текстов, написанных в состоянии фрустрации. Вестник Московского университета. Серия 14: Психология, 3, 66—85.
   Enikolopov, S.N., Kovalyov, A.K., Kuznetsova, Yu.M., Chudova, N.V., Starostina, E.V. (2019b). Signs characteristic of texts written in a state of frustration. Moscow University Psychology Bulletin, 3, 66—85. (In Russ.)
8. Ениколопов, С.Н., Медведева, Т.И., Воронцова, О.Ю. (2019в). Лингвистические характеристики текстов при депрессии и шизофрении. Медицинская психология в России: электронный научный журнал, 11(5).
   Enikolopov, S.N., Medvedeva, T.I., Vorontsova, O.Yu. (2019с). Linguistic text characteristics in depression and schizophrenia. Medical Psychology in Russia, 11(5). (In Russ.)
9. Ениколопов, С.Н., Медведева, Т.И., Воронцова, О.Ю., Иванова, П.О. (2019г). Оценка текстов, написанных больными эндогенными психическими заболеваниями. Психиатрия, 81(1), 56—68.
   Enikolopov, S.N., Medvedeva, T.I., Vorontsova, O.Yu., Ivanova, P.O. (2019d). Evaluation of texts written by patients with endogenous mental illnesses. Psychiatry, 81(1), 56—68. (In Russ.)
10. Кнутов, А.В., Чаплинский, А.В., Алимпеев, Д.Р. (2022). Язык нормативных правовых актов: пора ли бить тревогу? Вестник Пермского университета. Юридические науки, 57, 399—426. https://doi.org/10.17072/1995-4190-2022-57-399-426
    Knutov, A.V., Chaplinsky, A.V., Alimpeev, D.R. (2022). The Language of Regulatory Legal Acts: Is It Time to Sound the Alarm? Perm University Herald. Juridical Sciences, 57, 399—426. (In Russ.). https://doi.org/10.17072/1995-4190-2022-57-399-426
11. Кнутов, А.В., Плаксин, С.М., Чаплинский, А.В., Григорьева, Н.Л., Синятуллин, Р.Х., Успенская, А.М. (2021). Сложность российских законов. Опыт синтаксического анализа. М.: Издательский дом Высшей школы экономики.
    Knutov, A.V., Plaksin, S.M., Chaplinsky, A.V., Grigoryeva, N.L., Sinyatullin, R.H., Uspenskaya, A.M. (2021). The Complexity of Russian Laws: An Experiment in Syntactic Analysis. Moscow: Higher School of Economics Publishing House. (In Russ.)
12. Лапошина, А.Н. (2023). Лингводидактическое обоснование применения Автоматической оценки сложности учебного текста в преподавании РКИ: Дис. ... канд. пед. наук. М.
    Laposhina, A.N. (2023). A linguodidactic rationale for the use of automatic assessment of educational text complexity in teaching Russian as a foreign language: Cand. Sciences (Ped.). Moscow. (In Russ.)
13. Ромич, М., Бороевич, С. (2025). Типографские характеристики и обработка латинских и кириллических слов. Экспериментальная психология, 18(1), 108—118.
    Romich, M., Boroevich, S. (2025). Typographic characteristics and processing of Latin and Cyrillic words. Experimental Psychology (Russia), 18(1), 108—118. (In Russ.)
14. Савчук, С.О., Архангельский, Т.А., Бонч-Осмоловская, А.А., Донина, О.В., Кузнецова, Ю.Н., Ляшевская, О.Н., Орехов, Б.В., Подрядчикова, М.В. (2024). Национальный корпус русского языка 2.0: новые возможности и перспективы развития. Вопросы языкознания, 2, 7—34. 
    Savchuk, S.O., Arhangelsky, T.A., Bonch-Osmolovskaya, A.A., Donina, O.V., Kuznetsova, Yu.N., Lyashevskaya, O.N., Orekhov, B.V., Podryadchikova, M.V. (2024). The National Corpus of the Russian Language 2.0: New Possibilities and Development Prospects. Linguistics Issues, 2, 7—34. (In Russ.).
15. Список Синтаксических Отношений. (2023). URL: https://ruscorpora.ru/media/uploads/2023/12/29/rajadw.pdf (дата обращения: 10.04.2025).
    List of Syntactic Relations (2023). (In Russ.). URL: https://ruscorpora.ru/media/uploads/2023/12/29/rajadw.pdf (viewed: 10.04.2025).
16. Almayev, N.A., Murasheva, O.V., Petrovich, D.L. (2024). Rusalov STQ-150 and Rusalov-Trofimova STQ-77 in Relation to the EEG Spectral Power. Natural Systems of Mind, 4(1), 6—16. https://doi.org/10.38098/nsom_2024_04_01_01 (viewed: 19.03.2026).
17. Kovriguina, L., Shilin, I., Shipilo, A., Putintseva, A. (2017). Russian Tagging and Dependency Parsing Models for Stanford CoreNLP Natural Language Toolkit. In: Knowledge Engineering and Semantic Web — 8th International Conference, KESW 2017, Szczecin, Poland, November 8—10, 2017, Proceedings (pp. 101—111). https://doi.org/1007/978-3-319-69548-8_8
18. Liu, F., Lee, J.S. (2023). Hybrid models for sentence readability assessment. In: Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications, BEA, 2023 (pp. 448—454).
19. Natasha/Slovnet (Электронный ресурс): репозиторий. URL: https://github.com/natasha/slovnet (viewed: 05.2024).
20. Ortega, L. (2003). Syntactic complexity measures and their relationship to l2 proficiency: A research synthesis of college-level l2 writing. Applied Linguistics, 24(4), 492—518.
21. Pan, W, Li, X, Chen, X, Xu, R. (2024). Textual form features for text readability assessment. Natural Language Processing, 1—42. https://doi.org/10.1017/nlp.2024.50
22. Soegaard, M., Dam, R.F. (Eds.). (2015). The Encyclopedia of Human-Computer Interaction. Aarhus, Dansk: The Interaction Design Foundation. URL: https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed (viewed: 19.03.2026).