Text Detoxification System in Dialogue Conversations

55

Abstract

The work is aimed at improving the cultural level of correspondence in dialog systems. The key feature of the work is its focus on real–time use and ensuring sustainable detoxification, taking into account the specifics of dialog communication (typos, noise symbols, transliteration, etc.). The solution offers the use of a neural network approach and software processing to obtain embeds of tokens and the subsequent solution of the classification problem. Unlike traditional message filters, the task is to preserve the meaning of the source text by clearing it of toxic content. The operability of the system can be checked on the basis of the Telegram messenger, in which the model is presented in the form of a bot. The system itself is deployed on the basis of Serverless technology from a cloud provider, which allows it to adapt to peak loads and at the same time be easy to maintain.

General Information

Keywords: detoxification of text, neural networks, serverless

Journal rubric: Data Analysis

Article type: scientific article

DOI: https://doi.org/10.17759/mda.2023130102

Received: 17.01.2023

Accepted:

For citation: Suvorov M.D., Vinogradov V.I. Text Detoxification System in Dialogue Conversations. Modelirovanie i analiz dannikh = Modelling and Data Analysis, 2023. Vol. 13, no. 1, pp. 19–24. DOI: 10.17759/mda.2023130102. (In Russ., аbstr. in Engl.)

References

  1. Rubtsova Yu.V. Automatic construction and analysis of the corpus of short texts (microblogging posts) for the task of developing and training a tone classifier //Knowledge engineering and semantic web technologies. - 2012. – Vol. 1. – pp. 109-116.

Information About the Authors

Mariam D. Suvorov, student, Moscow Aviation Institute (National Research University) (MAI), Moscow, Russia, ORCID: https://orcid.org/0000-0002-8376-0448, e-mail: msuvorov7@gmail.com

Vladimir I. Vinogradov, PhD in Physics and Matematics, Associate Professor, Moscow Aviation Institute (National Research University), Moscow, Russia, ORCID: https://orcid.org/0000-0003-3773-9653, e-mail: vvinogradov@inbox.ru

Metrics

Views

Total: 120
Previous month: 18
Current month: 5

Downloads

Total: 55
Previous month: 3
Current month: 1