Russian Psychological Issues
JournalsTopicsAuthorsEditor's Choice For AuthorsAbout PsyJournals.ruContact Us

Modelling and Data Analysis

Publisher: Moscow State University of Psychology and Education

ISSN (printed version): 2219-3758

ISSN (online): 2311-9454


License: CC BY-NC 4.0

Started in 2011

Published 4 times a year

Free of fees
Open Access Journal


Gradient Optimization Methods in Machine Learning for the Identification of Dynamic Systems Parameters 260

Panteleev A.V.
Doctor of Physics and Matematics, Professor, Head of the Department of Mathematical Cybernetics, Faculty of Information Technologies and Applied Mathematics, Moscow aviation Institute (national research University), Moscow, Russia

Lobanov A.V.
Master’s Degree student of the Faculty of Information, Technology and Applied Mathematics of Moscow Aviation Institute (National Research University), Moscow, Russia

The article considers one of the possible ways to solve the problem of estimating the unknown parameters of dynamic models described by differential-algebraic equations. Parameters are estimated based on the results of observations of the behavior of the mathematical model. Their values are found as a result of minimizing the criterion that describes the total quadratic deviation of the state vector coordinates from the exact values obtained at measurements at different points in time. The parallelepiped type constraints are imposed on the parameter values. To solve the optimization problem, it is proposed to use gradient optimization methods used in machine learning procedures: the stochastic gradient descent method, the classical moment method, the Nesterov accelerated gradient method, the adaptive gradient method, root mean square propagation method, the adaptive moment estimation method, the adaptive estimation method modification, Nesterov–accelerated adaptive moment estimation method. An example of identification of the parameters of a linear mathematical model describing a change in the characteristics of a chemical process is shown, which demonstrates the comparative effectiveness of the optimization methods of the selected group. The methods used to search for an extremum do not guarantee finding a result – a global extremum, but allow you to get a solution of good enough quality for an acceptable time. The results of calculations by all the listed optimization methods are presented. Recommendations on the selection of method parameters are given. The obtained numerical results demonstrated the effectiveness of the proposed approach. The found approximate values of the estimated parameters slightly differ from the best known solutions obtained by other methods.

Keywords: conditional optimization, machine learning, gradient methods of machine learning, parameter estimation

Column: Optimization Methods


For Reference

  1. Ivchenko G.I., Medvedev Yu. I. Vvedenie v matematicheskuyu statistiku [Introduction to Mathematical Statistics]. Moscow: Publ. Librocom, 2014.
  2. Panteleev A.V., Letova Т.А. Metody optimizacii. Prakticheskij kurs [Optimization Methods. Practical Course]. Мoscow: Logos, 2011.
  3. Panteleev A.V., Skavinskaya D.V. Metaehvristicheskie algoritmy globalnoj optimizacii [Global optimization metaheuristic algorithms]. Мoscow: Publ. Vuzovskaya kniga, 2019.
  4. Panteleev A.V., Kryuchkov A. Yu. Metaehvristicheskie metody optimizacii v zadachah ocenki parametrov dinamicheskih sistem [Metaheuristic optimization methods for parameters estimation of dynamic systems] // Civil Aviation High Technologies. 2017; 20(2): 37–45.
  5. Ruder S. An Overview of Gradient Descent Optimization Algorithms arXiv:1609.04747v2 [cs.LG] 15 Jun 2017.
  6. Floudas C.A., Pardalos P.M., Adjimann C.S., Esposito W.R., Gumus Z.H., Harding S.T., Schweiger C.A. Handbook of test problems in local and global optimization, 1999. Vol. 67. Springer US. 442 p.
  7. Tjoa I.–B., Biegler L.T. Simultaneous solution and optimization strategies for parameter estimation of differential–algebraic equation systems. Industrial & Engineering Chemistry Research, 1991, Vol. 30, No. 2, pp. 376–385.
comments powered by Disqus

© 2007–2020 Portal of Russian Psychological Publications. All rights reserved in Russian

Publisher: Moscow State University of Psychology and Education

Catalogue of academic journals in psychology & education MSUPE

Creative Commons License

RSS Psyjournals at facebook Psyjournals at Twitter Psyjournals at Youtube ??????.???????