Skip to Main Content

Modelling and Data Analysis
2024. Vol. 14, no. 2, 7–22
https://doi.org/10.17759/mda.2024140201
ISSN: 2219-3758 / 2311-9454 (online)

Experience in Using the Transformer Network Architecture to Approximate Agent’s Policy in Reinforcement Learning

 
Audio is AI-generated
144

,

PDF (RUS)
Online translation in english
GS
Information
in Google Scholar
Metrics

Abstract

This paper discusses the basics of the deep reinforcement learning algorithm and the use of neural networks to approximate the agent’s policy. The comparison of using a fully connected neural network and a transformer network in the reinforcement learning algorithm is considered.

General Information

Keywords: artificial intelligence, machine learning, deep reinforcement learning, Markov decision processes, transformer, optimization

Journal rubric: Data Analysis

OpenAlex citations: 0

OpenAlex trends: Advanced Research in Systems and Signal Processing

Information about the work in OpenAlex

Number of citations: 0

Topics

Advanced Research in Systems and Signal Processing

This cluster of papers focuses on the integration of cyber, physical, and social systems, with an emphasis on decision making, urban computing, autodyne sensors, machine learning, data mining, transportation systems, information management, parallel computing, and infrastructure development.

Number of works: 31317  |  Total number of citations: 164598

Topic detailsв OpenAlex

Work details in OpenAlex

Article type: scientific article

DOI: https://doi.org/10.17759/mda.2024140201

Received 03.06.2024

Accepted

Published

For citation: Novikov, N.P., Vinogradov, V.I. (2024). Experience in Using the Transformer Network Architecture to Approximate Agent’s Policy in Reinforcement Learning. Modelling and Data Analysis, 14(2), 7–22. (In Russ.). https://doi.org/10.17759/mda.2024140201

© Novikov N.P., Vinogradov V.I., 2024

License: CC BY-NC 4.0

References

  1. An outline of reinforcement learning // Arxiv URL: https://arxiv.org/pdf/2201.09746.pdf (circulation date: 02.01.2024).
  2. Proximal Policy Optimization Algorithms // Arxiv URL: https://arxiv.org/pdf/1707.06347.pdf (circulation date: 24.12.2023).
  3. Attention Is All You Need // Arxiv URL: https://arxiv.org/abs/1706.03762 (circulation date: 16.12.2023).
  4. Gymnasium URL: https://gymnasium.farama.org/ (circulation date: 10.12.2023).
  5. Stable Baselines Documentation // URL: https://buildmedia.readthedocs.org/media/pdf/stable-baselines/master/stable-baselines.pdf (circulation date: 08.12.2023).
  6. High-dimensional continuous control using generalized advantage estimation // Arxiv URL: https://arxiv.org/pdf/1506.02438.pdf (circulation date: 07.12.2023).

Information About the Authors

Nikita P. Novikov, master's student, Institute of Computer Science and Applied Mathematics, Moscow Aviation Institute (National Research University) (MAI), Moscow, Russian Federation, e-mail: rtyderson@gmail.com

Vladimir I. Vinogradov, Candidate of Science (Physics and Matematics), Associate Professor, Department of Mathematical Cybernetics, Moscow Aviation Institute (National Research University), Moscow, Russian Federation, ORCID: https://orcid.org/0000-0003-3773-9653, e-mail: vvinogradov@inbox.ru

Metrics

 Web Views

Whole time: 452
Previous month: 33
Current month: 26

 PDF Downloads

Whole time: 144
Previous month: 6
Current month: 22

 Total

Whole time: 596
Previous month: 39
Current month: 48