Perception and assessment of an interactive AI assistant in a digital educational environment: a psychological and pedagogical analysis

Context and relevance. Traditional passive online learning formats in higher education often fail to sustain students' academic engagement. This raises the challenge of finding effective tools for personalizing education while minimizing cognitive and socio-psychological risks. Conversational AI assistants hold significant potential to address this issue; however, their impact on the subjective educational experience remains insufficiently explored. The rationale for this study stems from the need for a psychological and pedagogical analysis of how interactivity, personalization, and media-based stylization determine learners' engagement and satisfaction. Objective. The objective of this study is to provide a comprehensive evaluation of the impact of the AI lecturer DeepTalk on students’ academic engagement and their subjective perception of the educational course. Hypothesis. The use of an interactive AI assistant implementing principles of personalization and dialogue is expected to lead to higher levels of perceived engagement and overall course satisfaction compared to a traditional non-interactive online format. Methods and materials. A mixed-methods design was employed. Quantitative data were gathered through questionnaires, while qualitative data were derived from semi-structured interviews. The sample comprised 40 students (M = 20,9, SD = 2,43; 62,5% female) from Tomsk State University, who were assigned to either an experimental group (N = 20) learning with an AI lecturer or a control group (N = 20) following a conventional online format. Results. Subjective assessments revealed a largely positive perception of the AI-assisted course. The majority of the experimental group (85%) gave it high ratings, praising its novel format, logical structure, and practical utility. Interactivity, personalization, and gamification were identified as key drivers of this positive experience. However, significant barriers were also noted, primarily technical issues (e.g., AI response delays) and the assistant's limited anthropomorphic qualities, such as a monotonous voice. A key finding is that most participants believe AI cannot fully replace a human teacher in creative or discussion-heavy subjects. Additionally, 20% of respondents raised concerns about data privacy. Conclusions. These findings can inform the development of more effective and human-centered AI courses. While the results highlight AI's potential as a powerful supplementary tool, they also emphasize the critical need for robust technical implementation, user-controlled flexibility, and the irreplaceable role of the human educator in complex learning environments.

Introduction

Modern higher education operates in the context of accelerating digital transformation, which presents the complex challenge of maintaining and increasing students' academic engagement. Traditional online learning formats, which have become widespread, often fail to meet this challenge, leading to decreased motivation and superficial assimilation of material (Lin, Huang, Lu, 2023). Against this background, artificial intelligence (AI) technologies, particularly interactive conversational assistants, are positioned as one of the most promising solutions capable of radically changing the educational paradigm (Chan, Tsi, 2024). The potential of AI lies in its ability to implement the concept of deep personalization, adapting the content, pace, and style of information delivery to the unique cognitive and affective needs of each learner (Dzhanegizova et al., 2024; Kumar, Raman, 2022; Timea, Veres, 2023).

Realizing this potential is accompanied by a fundamental psycho-pedagogical contradiction. On the one hand, the introduction of AI opens the path to achieving one of the key tasks of didactics: creating an individual tutor for each student capable of providing instant feedback and maintaining constant cognitive activity (Vieriu, Petrea, 2025). On the other hand, replacing human communication with algorithmic interaction carries socio-psychological risks. These include the potential degradation of critical thinking skills, the formation of learned helplessness, cognitive dependence on the AI system (Davydova, Shlykova, 2024; Lukichev, Chekmarev, 2024), as well as broader threats to socio-political communication systems and corporate structures (Scott, Carter, Coiera, 2021). Thus, the research problem lies in finding a balance between the technological capabilities of AI to enhance engagement and the need to minimize concomitant risks to the holistic development of the student's personality.

The relevance of our research is determined by the accelerating pace of AI solution implementation in educational practice, which significantly outpaces their scientific understanding. The modern scientific discourse on this topic can be conditionally divided into several directions. The first direction is devoted to the analysis of students' general attitudes and expectations regarding AI (Gnambs et al., 2025). The works of Kumar and Raman (2022) and Dzhanegizova et al. (2024) convincingly demonstrate that students associate AI with hopes for greater flexibility and personalization of the learning process. The second direction focuses on the theoretical analysis of risks, including the ethical consequences of using generative chatbots (Williams, 2024). Studies by Davydova and Shlykova (2024), as well as Lukichev and Chekmarev (2024), deeply explore threats to academic integrity, independence, and students' analytical abilities, as evidenced by their own concerns expressed in surveys (Tikhonova, Ilduganova, 2024).

The present study complements previous works by focusing on a detailed psycho-pedagogical analysis of students' subjective experience. We compare the real experience of interacting with an innovative technology with hypothetical expectations and the evaluation of a traditional format. This approach allows for a deeper understanding of how well students' expectations of AI align with reality, and helps identify the key factors determining the success or failure of implementing such systems (Kim, Lee, 2024).

The theoretical framework of the study consists of two complementary concepts. Michael Moore's transactional distance theory allows us to consider the AI assistant as a tool for potentially reducing the psychological and communicative distance between the learner and the learning environment. We assume that a high degree of dialogue and adaptive course structure, implemented by the DeepTalk technology, contribute to reducing this distance and increasing the sense of engagement. Benjamin Bloom's mastery learning concept emphasizes the importance of step-by-step material acquisition with regular diagnostics and immediate corrective feedback.

The main objective of the study is to conduct a comprehensive empirical evaluation of the impact of the interactive AI assistant DeepTalk on student engagement and subjective perception of the educational process, as well as to identify the key psycho-pedagogical factors determining this experience.

Based on the theoretical approach, the main research hypothesis was formulated: the use of an interactive AI assistant implementing the principles of personalization and dialogic interaction will lead to a higher level of engagement and overall course satisfaction compared to a traditional, non-interactive online format.

Additional specific hypotheses were as follows:

1. Elements of personalization and media stylization will be evaluated by students as significant factors increasing interest in the learning material.
2. Despite technological advantages, learners will express doubts about the AI assistant's ability to completely replace a human teacher, especially in aspects requiring empathy and flexibility.
3. Concerns about the confidentiality of personal data will become one of the key barriers limiting full trust in a personalized AI system.

To comprehensively test the proposed hypotheses and address the research questions, a study design was required that would allow for a comparison between students' real-world experience with AI and their habitual educational routine. This necessitated a mixed-methods data collection strategy, combining the measurement of baseline satisfaction with a qualitative analysis of the deep nuances of the new user experience.

Materials and methods

To empirically test the formulated hypotheses, a pilot (exploratory) study was conducted using the DeepTalk technology. It was designed with two groups for comparative analysis of the perception of innovative and traditional learning formats.

The study involved 40 undergraduate and graduate students (M = 20,9, SD = 2,43; 62,5% female) majoring in humanities (psychology, philology) at National Research Tomsk State University, who were interested in taking the course "Fundamentals of Prompt Engineering". The educational content in both groups was identical; only the format of material delivery varied (video lecture vs. interactive dialogue). The participants were divided into an experimental group (E-group, N = 20), which took the course with an interactive dialogue format using the DeepTalk AI assistant (see Fig. 2 for the experimental course interface example), adaptive personalization of educational content and animated presentations and a control group (C-group, N = 20), which studied the material in the format of a standard video lecture without the possibility of voice interaction with the AI lecturer (see Fig. 1 for the control course interface example). The distribution of participants was non-randomized; the differentiation criterion was a preliminary screening: the experimental group included learners who confirmed their familiarity with the plot of the series “Squid Game”. The rest formed the control group. This approach was necessary to ensure students' understanding of the course stylization context. The choice of this media product was due to its high virality in the student environment, ensuring the validity of the perception of gamified metaphors.

A mixed strategy for collecting psychosocial data was applied, chosen due to the exploratory nature of the work and the fundamental difference in the nature of the educational experience between the groups. The use of different instruments (interviews for the E-group and questionnaires for the C-group) is a conscious methodological decision:

1. For the experimental group, interacting with innovative technology, a qualitative method was chosen – semi-structured interviews. Interaction with the AI assistant represented a fundamentally new, complex user experience that could not be validly measured by a standard questionnaire without losing critically important contextual nuances of perception. The interview guide included blocks of questions aimed at identifying the subjective assessment of overall satisfaction (direct question: “Rate your overall impression of the course on a scale from 1 to 10”), perception of the anthropomorphic characteristics of the AI lecturer, attitude towards stylization and personalization, and the level of subjective engagement. Qualitative data were obtained from transcripts of 20 interviews with a total duration of 677 minutes and processed using thematic coding.
2. For the control group, the educational experience was routine and standardized, which allowed the use of a questionnaire. The toolkit included an assessment of satisfaction with the traditional lecture on a 5-point Likert scale, open questions about format deficits (“How often did you get distracted during the lecture?”) and a block of projective questions to identify hypothetical expectations regarding the introduction of AI (“How interesting was it to follow the course of the lecture?”).

Special attention was paid to the operationalization of the concept of “engagement”, which was defined not through external metrics, but through students' self-reports:

1. In the experimental group (with interviews), engagement was recorded by the presence of markers of an active state in the transcripts – mentions of “immersion”, “interest in the dialogue”, and the absence of mentions of “boredom” or “desire to get distracted”.
2. In the control group (questionnaire), engagement was assessed through a reverse question: “How often did you get distracted during the lecture?” and a direct assessment: “How interesting was it to follow the course of the lecture?”.

This approach allows for comparing the qualitative depth of immersion in the E-group with the assessment of attention retention in the C-group. Thus, the comparison of groups in this design is conducted not through direct statistical comparison of metrics (which is impossible due to the difference in scales), but through cross-contextual analysis of semantic patterns: comparing the actually experienced innovation (E-group) with the evaluation of current educational routine and attitudes towards AI (C-group). This approach allows focusing on the qualitative differences in the nature of the educational experience.

Results

Students in the experimental group who interacted with the DeepTalk technology demonstrated a predominantly positive attitude towards the course. The vast majority – 17 participants (85%) – rated their experience at 8–10 points on a 10-point scale (during the interview, participants in this block were asked: “Rate your overall impression of the course on a scale from 1 to 10?”, “Why did you give this rating?”, “What exactly did you like about the lecture?”, etc.).

Analysis of the interview transcripts allowed us to identify the factors determining this assessment. The key drivers of engagement were interactivity and personalization, noted by 14 students (70%). The accessibility and logical structure of the material were highlighted by 15 people (75%). One respondent described their experience as follows: “It seemed to me that since it was a direct dialogue, it was very pleasant that it was adjusted to the series...”.

At the same time, a number of significant barriers were identified. According to 8 people (40%), deficiencies in "anthropomorphic quality" included voice monotony and insufficient dialogue flexibility (participants were asked: “How did you perceive the AI lecturer?”, “Were there moments when you felt you were communicating with a human teacher rather than artificial intelligence?”, etc.). Technical limitations of the platform (delays, glitches) were noted by 7 (35%) people.

Media stylization caused an ambivalent reaction: for 11 people (55%) it was a successful gaming element, but for 4 participants (20%) it was a distracting factor. An interesting psychological phenomenon was the perception of feedback from AI as more objective (questions in this block were: “Was the feedback from the AI lecturer useful to you?”, etc.). However, 12 participants (60%) agreed that AI is not yet capable of completely replacing a human teacher in complex topics (participants were asked: “In your opinion, can an AI lecturer in this format replace a human teacher?”, etc.).

Detailed results of the analysis for the experimental group are presented in Table 1.

Table 1

Comprehensive analysis of student perceptions of the experimental AI-assisted course (N = 20)

Note: n – number of respondents who selected the category (out of total ).

Students in the control group, evaluating the traditional online course, gave it an overall positive rating (mean score 4,1 out of 5). As advantages of the format, 14 people (70%) highlighted the academic structuring of the material and the absence of distracting factors inherent in gamification.

Analysis of open-ended questions revealed the key deficits of the traditional format. Seven participants (35%) noted low interactivity as the main barrier to engagement. The lack of personalization (universal content) was also named as a motivation-reducing factor by 7 people (35%).

Remarkably, the hypothetical expectations of this group regarding a course with an AI assistant mirror the shortcomings of their real experience. More than half of the respondents – 11 people (55%) – assumed that AI would increase their motivation precisely due to interactivity and personalization. It is worth noting that 9 students (45%) expressed concern that the lack of emotion in AI would hinder full-fledged dialogue, and 6 students (30%) stated their unwillingness to share personal data with AI systems. Generalized results for the control group are presented in Table 2.

Table 2

Analysis of student perceptions of the control course and their hypothetical expectations from an AI assistant (N = 20)

Note: n – number of respondents who selected the category (out of total ).

Comparing the data from both groups allows us to conclude that the deficits inherent in traditional online courses form a clear student demand for the functions that modern AI assistants, particularly the DeepTalk technology, can develop. Analysis of subjective reports showed that in the experimental group, the presence of these functions is associated with a higher level of declared engagement compared to the control group, where their absence was regarded by students as a demotivating factor.

Regardless of the format, learners expect high-quality visual support, the ability to control the pace of learning, and a large number of relevant practical examples. The need for a more “lively” and emotionally colored voice of the AI lecturer once again emphasizes that even in a digital environment, the demand for “humanity” in communication remains extremely high.

Conclusions

In conclusion, the empirical verification fully confirmed our primary hypothesis. Furthermore, the specific hypotheses regarding the role of gamification, the boundaries of AI capabilities, and the significance of ethical barriers were also supported.

The findings of this study hold potential for addressing broader, interdisciplinary challenges. For instance, insights into the importance of AI “voice” quality and platform stability can be applied to the design of human-computer interaction (HCI) systems in other domains.

Overall, our research confirms that artificial intelligence serves as a powerful auxiliary tool in education. However, its successful integration requires not merely technological innovation, but a profound understanding of the cognitive and social aspects of the learning interaction, alongside the preservation of the teacher's central role as a moderator of complex educational processes.

Future research prospects lie in transitioning from capturing initial perceptions to investigating long-term effects. It is crucial to examine how the engagement effect is maintained as students become accustomed to the AI lecturer (i.e., the “novelty effect” versus sustained motivation). Additionally, a promising avenue involves the development and testing of hybrid pedagogical models, wherein AI handles routine tasks while mentoring and emotional support remain the domain of the human teacher. Future studies also plan to expand the sample to verify the identified patterns among students in technical disciplines.

Limitations. Given the limited sample size (n = 40), the findings should be interpreted as preliminary psycho-pedagogical results regarding the impact of the interactive AI assistant DeepTalk.

The authors acknowledge that the method of group formation constitutes a methodological limitation, as it allows for potential baseline differences between the groups in terms of cognitive or motivational parameters. However, within the framework of a pilot study, this approach was deemed appropriate for the valid assessment of specific reactions to stylization and interactivity.

1. Букина, Т.В. (2025). Искусственный интеллект в образовании: современное состояние и перспективы развития. Общество: социология, психология, педагогика, 1, 76—83. https://doi.org/10.24158/spp.2025.1.9
   Bukina, T.V. (2025). Artificial intelligence in education: current state and development prospects. Society: Sociology, Psychology, Pedagogy, 1, 76—83. (In Russ.). https://doi.org/10.24158/spp.2025.1.9
2. Давыдова, Г.И., Шлыкова, Н.В. (2024). Риски и вызовы при внедрении искусственного интеллекта в систему высшего образования. Вестник практической психологии образования, 21(3), 62—69. https://doi.org/10.17759/bppe.2024210308
   Davydova, G.I., Shlykova, N.V. (2024). Risks and challenges of artificial intelligence implementation in the higher education system. Bulletin of Practical Psychology in Education, 21(3), 62—69. (In Russ.).
3. Джанегизова, А.С., Нурсейит, А.М., Выборова, К.С. (2024). Искусственный интеллект в образовании: анализ динамики, восприятия и перспектив интеграции. Qainar Journal of Social Science, 2(4), 34—49. https://doi.org/10.58732/2958-7212-2023-4-34-49
   Dzhanegizova, A.S., Nurseit, A.M., Vuborova, K.S. (2024). Artificial intelligence in education: analysis of dynamics, perception, and integration prospects. Qainar Journal of Social Science, 2(4), 34—49. (In Russ.).
4. Лукичев, П.М., Чекмарев, О.П. (2024). Риски применения искусственного интеллекта в системе высшего образования. Вопросы инновационной экономики, 14(2), 463—-482. https://doi.org/10.18334/vinec.14.2.120731
   Lukichev, P.M., Chekmarev, O.P. (2024). Risks of using artificial intelligence in the higher education system. Russian Journal of Innovation Economics, 14(2), 463—482. (In Russ.).
5. Тихонова, Н.В., Ильдуганова, Г.М. (2024). «Меня пугает то, с какой скоростью развивается искусственный интеллект»: восприятие студентами искусственного интеллекта в обучении иностранным языкам. Высшее образование в России, 33(4), 63—83. https://doi.org/10.31992/0869-3617-2024-33-4-63-83
   Tikhonova, N.V., Ilduganova, G.M. (2024). «I am frightened by the speed at which artificial intelligence is developing»: students’ perception of artificial intelligence in teaching foreign languages. Higher Education in Russia, 33(4), 63—83. (In Russ.).
6. Cao, C.C., Ding, Z., Lin, J., Hopfgartner, F. (2023). AI Chatbots as multi-role pedagogical agents: Transforming engagement in CS education [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2308.03992
7. Chan, C.K.Y., Tsi, L.H. (2024). Will generative AI replace teachers in higher education? A study of teacher and student perceptions. Studies in Educational Evaluation, 83, 101395. https://doi.org/10.1016/j.stueduc.2024.101395
8. Gnambs, T., Stein, J.P., Appel, M., Griese, F., Zinn, S. (2025). An economical measure of attitudes towards artificial intelligence in work, healthcare, and education (ATTARI-WHE). Computers in Human Behavior: Artificial Humans, 3, 100106. https://doi.org/10.1016/j.chbah.2024.100106
9. Jose, B., Cherian, J., Jaya, P.J., Kuriakose, L., Leema, P.R. (2024). The ghost effect: how gamification can hinder genuine learning. Frontiers in Education, 9, 1474733. https://doi.org/10.3389/feduc.2024.1474733
10. Kim, S.W., Lee, Y. (2024). Investigation into the influence of sociocultural factors on attitudes toward artificial intelligence. Education and Information Technologies, 29(8), 9907— https://doi.org/10.1007/s10639-023-12172-y
11. Kumar, V.R., Raman, R. (2022). Student Perceptions on Artificial Intelligence (AI) in higher education. In 2022 IEEE Integrated STEM Education Conference (ISEC), (pp. 450—454). IEEE. https://doi.org/10.1109/ISEC54952.2022.10025165
12. Lin, C.C., Huang, A.Y., Lu, O.H. (2023). Artificial intelligence in intelligent tutoring systems toward sustainable education: a systematic review. Smart Learning Environments, 10(1), 41. https://doi.org/10.1186/s40561-023-00260-y
13. Rani, N., Majumder, S., Bhardwaj, I., Garcia, P.G.F. (2025). Can AI support student engagement in classroom activities in higher education? [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2506.18941
14. Scott, I.A., Carter, S.M., Coiera, E. (2021). Exploring stakeholder attitudes towards AI in clinical practice. BMJ Health & Care Informatics, 28(1), e100450. https://doi.org/10.1136/bmjhci-2021-100450
15. Shum, N.Y.E., Lau, H.P.B. (2024). Perils, power and promises: Latent profile analysis on the attitudes towards artificial intelligence (AI) among middle-aged and older adults in Hong Kong. Computers in Human Behavior: Artificial Humans, 2(2), 100091. https://doi.org/10.1016/j.chbah.2024.100091
16. Timea, K.A., Veres, E. (2023). Students' perception of artificial intelligence in higher education. International Scientific Journal on Social Science, 56(3). https://doi.org/10.35603/sws.iscss.2023/s08.38
17. Vieriu, A.M., Petrea, G. (2025). The impact of artificial intelligence (AI) on students academic development. Education Sciences, 15(3), 343. https://doi.org/10.3390/educsci15030343
18. Wang, D., Bian, C., Chen, G. (2024). Using explainable AI to unravel classroom dialogue analysis: Effects of explanations on teachers' trust, technology acceptance and cognitive load. British Journal of Educational Technology, 55(6), 2530— https://doi.org/10.1111/bjet.13466
19. Williams, R.T. (2024). The ethical implications of using generative chatbots in higher education. Frontiers in Education, 8, 1331607. https://doi.org/10.3389/feduc.2023.1331607
20. Yuan, L., Liu, X. (2025). The effect of artificial intelligence tools on EFL learners' engagement, enjoyment, and motivation. Computers in Human Behavior, 162, 108474. https://doi.org/10.1016/j.chb.2024.108474
21. Zhang, T. (2025). Constructing and evaluating the effects of an immersive teaching mode for art education based on machine learning. Journal of Computational Methods in Sciences and Engineering, 25(2), 355—369. https://doi.org/10.1177/14727978251322681