Gradient Optimization Methods in Machine Learning for the Identification of Dynamic Systems Parameters



The article considers one of the possible ways to solve the problem of estimating the unknown parameters of dynamic models described by differential-algebraic equations. Parameters are estimated based on the results of observations of the behavior of the mathematical model. Their values are found as a result of minimizing the criterion that describes the total quadratic deviation of the state vector coordinates from the exact values obtained at measurements at different points in time. The parallelepiped type constraints are imposed on the parameter values. To solve the optimization problem, it is proposed to use gradient optimization methods used in machine learning procedures: the stochastic gradient descent method, the classical moment method, the Nesterov accelerated gradient method, the adaptive gradient method, root mean square propagation method, the adaptive moment estimation method, the adaptive estimation method modification, Nesterov–accelerated adaptive moment estimation method. An example of identification of the parameters of a linear mathematical model describing a change in the characteristics of a chemical process is shown, which demonstrates the comparative effectiveness of the optimization methods of the selected group. The methods used to search for an extremum do not guarantee finding a result – a global extremum, but allow you to get a solution of good enough quality for an acceptable time. The results of calculations by all the listed optimization methods are presented. Recommendations on the selection of method parameters are given. The obtained numerical results demonstrated the effectiveness of the proposed approach. The found approximate values of the estimated parameters slightly differ from the best known solutions obtained by other methods.

General Information

Keywords: conditional optimization, machine learning, gradient methods of machine learning, parameter estimation

Journal rubric: Optimization Methods

Article type: scientific article


For citation: Panteleev A.V., Lobanov A.V. Gradient Optimization Methods in Machine Learning for the Identification of Dynamic Systems Parameters. Modelirovanie i analiz dannikh = Modelling and Data Analysis, 2019. Vol. 9, no. 4, pp. 88–99. DOI: 10.17759/mda.2019090407. (In Russ., аbstr. in Engl.)


  1. Ivchenko G.I., Medvedev Yu. I. Vvedenie v matematicheskuyu statistiku [Introduction to Mathematical Statistics]. Moscow: Publ. Librocom, 2014.
  2. Panteleev A.V., Letova Т.А. Metody optimizacii. Prakticheskij kurs [Optimization Methods. Practical Course]. Мoscow: Logos, 2011.
  3. Panteleev A.V., Skavinskaya D.V. Metaehvristicheskie algoritmy globalnoj optimizacii [Global optimization metaheuristic algorithms]. Мoscow: Publ. Vuzovskaya kniga, 2019.
  4. Panteleev A.V., Kryuchkov A. Yu. Metaehvristicheskie metody optimizacii v zadachah ocenki parametrov dinamicheskih sistem [Metaheuristic optimization methods for parameters estimation of dynamic systems] // Civil Aviation High Technologies. 2017; 20(2): 37–45.
  5. Ruder S. An Overview of Gradient Descent Optimization Algorithms arXiv:1609.04747v2 [cs.LG] 15 Jun 2017.
  6. Floudas C.A., Pardalos P.M., Adjimann C.S., Esposito W.R., Gumus Z.H., Harding S.T., Schweiger C.A. Handbook of test problems in local and global optimization, 1999. Vol. 67. Springer US. 442 p.
  7. Tjoa I.–B., Biegler L.T. Simultaneous solution and optimization strategies for parameter estimation of differential–algebraic equation systems. Industrial & Engineering Chemistry Research, 1991, Vol. 30, No. 2, pp. 376–385.

Information About the Authors

Andrey V. Panteleev, Doctor of Physics and Matematics, Professor, Head of the Department of Mathematical Cybernetics, Institute of Information Technologies and Applied Mathematics, Moscow Aviation Institute (National Research University), Moscow, Russia, ORCID:, e-mail:

Aleksandr V. Lobanov, Master’s Degree student of the Faculty of Information, Technology and Applied Mathematics of Moscow Aviation Institute (National Research University), Moscow, Russia, e-mail:



Total: 863
Previous month: 9
Current month: 1


Total: 1230
Previous month: 9
Current month: 0