Skip to Main Content

Sentiment Analysis for Automatic Assessment of Learners' Experience (on the Basis of Reviews on Online Courses in Russian and English)

103

M.A. Kirina, L.D. Telnina

PDF (ENG)
PDF (RUS)

Abstract

The paper describes an experiment aimed at comparing the effectiveness of sentiment analysis tools for evaluating user experience based on public reviews on online courses on Stepik. The results of automatic extraction of sentiment scores for the corresponding reviews both in Russian and in English are considered. With regards to reviews on online courses in Russian, the use of the KartaSlovSent dictionary of emotive lexis and the model pre-trained on the RuSentiment dataset implemented in the dostoevsky library is discussed. Considering the reviews in English, popular NLP libraries, such as TextBlob and VADER, are tested. It is analyzed how the scores, on the scale from 1 to 5, which are given by users after the completion of online course correlate with the sentiment scores obtained on the basis of learners’ feedback. The lexical characteristics of reviews describing positive and negative experiences from online education are discussed. It is suggested that the combination of tools will allow developing a program for the text mining of user experiences from online courses in order to improve the educational products in question.

General Information

Keywords: sentiment analysis, user experience modelling, online education, natural language processing, semantic compression of text, computational linguistics

Article type: scientific article

Funding. The publication was prepared within the framework of the Academic Fund Program at the National Research University Higher School of Economics (HSE) in 2022 (grant # 21-04-053 ‘Artificial Intelligence Methods in Literature and Language Studies’).

Acknowledgements. The authors are grateful to Moskvina A.D. for assistance in data collection.

For citation: Kirina M.A., Telnina L.D. Sentiment Analysis for Automatic Assessment of Learners' Experience (on the Basis of Reviews on Online Courses in Russian and English). Digital Humanities and Technology in Education (DHTE 2022): Collection of Articles of the III All-Russian Scientific and Practical Conference with International Participation. November 17-18, 2022 / V.V. Rubtsov, M.G. Sorokova, N.P. Radchikova (Eds). Moscow: Publishing house MSUPE, 2022., pp. 355–374.

References

  1. Avtomaticheskaya obrabotka tekstov na estestvennom yazyke i analiz dannykh : ucheb. posobie [Automatic processing of natural language texts and data analysis] / Bol'shakova E.I., Vorontsov K.V., Efremova N.E., Klyshinskii E.S., Lukashevich N.V., Sapin A.S. Moscow: NIU VShE Publ., 2017. 269 p. (In Russ.).
  2. Al-Razgan M. et al. Using diffusion of innovation theory and sentiment analysis to analyze attitudes toward driving adoption by Saudi women. Technology in Society, 2021. Vol. 65. p. 101558. doi:10.1016/j.techsoc.2021.101558
  3. Hutto C., Gilbert E. VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text. Proceedings of the International AAAI Conference on Web and Social Media, 2014. Vol. 8, no. 1, pp. 216–225.
  4. Kim E., Klinger R. A Survey on Sentiment and Emotion Analysis for Computational Literary Studies, 2021. p. 38. doi:10.48550/arXiv.1808.03137
  5. Kulagin D.I. Publicly available sentiment dictionary for the Russian language KartaSlovSent, 2021. pp. 1106–1119. doi:10.28995/2075-7182-2021-20-1106-1119
  6. Liang Y., Liu Y., Loh H. T. Exploring Online Reviews For User Experience Modeling, 2013. p. 10.
  7. Loria S. textblob Documentation, 2020. p. 73.
  8. Ngoc T. V., Thi M. N., Thi H. N. Sentiment Analysis of Students’ Reviews on Online Courses: A Transfer Learning Method, 2021. p. 9.
  9. Phan M. H., Ogunbona P. O. Modelling Context and Syntactical Features for Aspect-based Sentiment Analysis Online: Association for Computational Linguistics, 2020. pp. 3211–3220. doi: 10.18653/v1/2020.acl-main.293
  10. Pope L. Comparing VADER and Text Blob to Human Sentiment. Medium [Online resource]. URL: https://towardsdatascience.com/comparing-vader-and-text-blob-to-human-sentiment-77068cf73982 (Accessed 02.09.2022)
  11. Reyes-Menendez A., Saura J. R., Filipe F. Marketing challenges in the #MeToo era: gaining business insights using an exploratory sentiment analysis. Heliyon, 2020. Vol. 6, iss. 3. doi: 10.1016/j.heliyon.2020.e03626
  12. Smetanin S. The Applications of Sentiment Analysis for Russian Language Texts: Current Challenges and Future Perspectives. IEEE Access. 2020, Vol. 8. pp. 110693–110719.
  13. Wang X. et al. A Depression Detection Model Based on Sentiment Analysis in Micro-blog Social Network. In J. Li et al. (ed.), Berlin, Heidelberg: Springer Berlin Heidelberg, 2013. pp. 201–213.
  14. White B. Sentiment Analysis: VADER or TextBlob? Medium [Online resource]. URL: https://towardsdatascience.com/sentiment-analysis-vader-or-textblob-ff25514ac540 (Accessed 02.09.2022)

Information About the Authors

Margarita A. Kirina, Master’s Student, Visiting Lecturer at Department of Philology, National Research University Higher School of Economics, St.Petersburg, Russia, ORCID: https://orcid.org/0000-0002-7381-676X, e-mail: mkirina2412@gmail.com

Ludmila D. Telnina, Bachelor’s Student, Department of Philology, National Research University Higher School of Economics, St.Petersburg, Russia, ORCID: https://orcid.org/0000-0003-2725-1902, e-mail: ldtelnina@edu.hse.ru

Metrics

Views

Total: 121
Previous month: 9
Current month: 0

Downloads

Total: 103
Previous month: 4
Current month: 4