Analysis of the efficiency of a multi-step adaptive optimization algorithm with prediction

Context and relevance. Global extremum search problems arise in the solution of a wide range of applied optimization problems, including the tuning of technical system parameters and the study of complex multi-extremal functions. For such problems, the application of classical deterministic methods is often difficult due to the presence of local extreme, the complex structure of the objective function, and the lack of complete information about its properties. In this regard, the development of software tools implementing adaptive and metaheuristic optimization algorithms is of great relevance. Objective. The aim of this work is to develop and study software for implementing a multi-step adaptive optimization algorithm with forecasting, designed to find the global extremum of the objective function within a given set of feasible solutions. Hypothesis. It is hypothesized that using a predicted solution position, a history of successful steps, adaptive step size adjustment, and the generation of new initial solutions via the Lévy distribution will improve the stability of the search and reduce the probability of prematurely reaching a local extremum. Methods and materials. This paper considers a multi-step adaptive algorithm based on sequential search in the neighborhood of the predicted solution, the use of memory of successful positions, and the generation of new initial solutions using the Lévy distribution. The developed software allows users to specify the objective function, the feasible solution region, and the algorithm parameters; perform computational experiments; and visualize the search process and the results obtained. Benchmark test functions of varying complexity were used to verify the program’s performance. Results. The computational experiments conducted showed that the implemented algorithm allows finding solutions close to the exact values of the global extremum for both simple and multi-extremal test functions. The results obtained confirm the functionality of the developed software package and the possibility of its application for analysis of the impact of parameter selection on the quality of the search. Conclusions. The developed software can be used for the numerical study of global optimization problems, the analysis of the behavior of a multi-step adaptive algorithm, and the selection of its parameters. The use of prediction, adaptive step size adjustment, and the generation of new initial points makes it possible to improve the stability of the search and reduce the probability of premature convergence to a local extremum.

1. Бортаковский, А. С. (2016) Теория управления в примерах и задачах: Учебное пособие / А. С. Бортаковский, А. В. Пантелеев. – 2-е издание, стереотипное. – Москва : Общество с ограниченной ответственностью «Научно-издательский центр ИНФРА-М». – (Бакалавриат). – ISBN 978-5-16-011862-8. – EDN VYIWMR.
   Bortakovskii, A. S., Panteleev, A. V. (2016) Control theory in examples and problems. Moscow: INFRA-M. EDN VYIWMR (In Russ.)
2. Гасников, А.В. (2018) Современные численные методы оптимизации. Метод универсального градиентного спуска. – М.: МФТИ.
   Gasnikov, A. V. (2018) Modern Numerical Methods for Optimization: The Universal Gradient Descent Method. Moscow: MIPT. (In Russ.).
3. Карпенко, А.П. (2021) Современные алгоритмы поисковой оптимизации. Алгоритмы, вдохновленные природой. – М.: МГТУ им. Н. Э. Баумана.
   Karpenko, A. P. (2021) Modern Search Optimization Algorithms: Algorithms Inspired by Nature. Moscow: Bauman Moscow State Technical University. (In Russ.)
4. Нестеров, Ю. Е. (2010) Методы выпуклой оптимизации. – М.: МЦНМО.
   Nesterov, Yu. E. (2010) Methods of Convex Optimization. – Moscow: MCNMO. (In Russ.).
5. Пантелеев, А. В. (2023) Метаэвристические стратегии и алгоритмы глобальной оптимизации / А. В. Пантелеев, Д. В. Скавинская. – Москва : Факториал. – ISBN 978-5-98688-322-9. – EDN XUERAR.
   Panteleev, A. V., Skavinskaya, D. V. (2023) Metaheuristic Strategies and Algorithms for Global Optimization. Moscow: (In Russ.).
6. Пантелеев, А. В. (2025) Методы оптимизации в машинном обучении / А. В. Пантелеев. – Москва: Издательство «Доброе слово и Ко». – ISBN 978-5-605-36885-4. – EDN QDZHUZ.
   Panteleev, A. V. (2025) Optimization Methods in Machine Learning. Moscow: Dobroe Slovo i Ko. EDN QDZHUZ. (In Russ.)
7. Пантелеев, А.В., Каранэ, М. М. С. (2024) Мультиагентные и биоинспирированные методы оптимизации технических систем. – М.: Доброе слово и Ко. EDN: NJFRAO.
   Panteleev, A. V., Karane, M. M. S. (2024) Multi-agent and Bio-inspired Methods for the Optimization of Technical Systems. Moscow: Dobroe Slovo i Ko. EDN: NJFRAO. (In Russ.)
8. Поляк, Б. Т. (1983) Введение в оптимизацию. – М.: Наука.
   Polyak, B. T. (1983) Introduction to Optimization. Moscow: Nauka. (In Russ.).
9. Alorf, A. (2023) A survey of recently developed metaheuristics and their comparative analysis. Engineering Applications of Artificial Intelligence, 117. Part A. Art. 105622. DOI: 10.1016/j.engappai.2022.105622.
10. Bazaraa, M. S., Sherali, H. D., Shetty, C. M. (2006) Nonlinear Programming: Theory and Algorithms. – 3rd ed. – Hoboken, NJ, USA: John Wiley & Sons.
11. Deb, K.; Goyal, M. (1996) A combined genetic adaptive search (GeneAS) for engineering design. Computer Science and informatics, 26, 30–45.
12. Del Ser, J., Osaba, E., Molina, D., Yang, X.-S., Salcedo-Sanz, S., Camacho, D., Das, S., Suganthan, P. N., Coello Coello, C. A., Herrera, F. (2019) Bio-inspired computation: Where we stand and what’s next. Swarm and Evolutionary Computation, 48, 220–250. DOI: 10.1016/j.swevo.2019.04.008.
13. Encyclopedia of Optimization (2009) / Eds. C. A. Floudas, P. M. Pardalos. – New York, NY, USA: Springer.
14. Gandomi, A. H., Yang, X.-S.; Alavi, A. H. (2013) Cuckoo search algorithm: A metaheuristic approach to solve structural optimization problems. Engineering with Computers,29(1), 17–35. DOI: 10.1007/s00366-011-0241-y.
15. Garg, H. (2019) A hybrid GSA-GA algorithm for constrained optimization problems. Information Sciences, 478, 499–523. DOI: 1016/j.ins.2018.11.041.
16. Gutowski, M. (2001) Lévy flights as an underlying mechanism for global optimization algorithms. – Cornell University. arXiv:math-ph/0106003.
17. Handbook of Metaheuristics (2019) / Eds. M. Gendreau, J-Y. Potvin. – New York, NY, USA: Springer. DOI: 10.1007/978-3-319-91086-4.
18. Hansen, E., Walster, G. W. (2004) Global Optimization Using Interval Analysis. – New York, NY, USA: Marcel Dekker.
19. Kannan, B. K.; Kramer, S. N. (1994) An augmented Lagrange multiplier based method for mixed integer discrete continuous optimization and its applications to mechanical design. Mech. Des. Trans., 116, 318–320.
20. Locatelli, M., Schoen, F. (2021) (Global) Optimization: Historical notes and recent developments. EURO Journal on Computational Optimization, 9. Art. 100012. DOI: 1016/j.ejco.2021.100012.
21. Luus, R. (2000) Iterative Dynamic Programming. – London, UK: Chapman & Hall/CRC. DOI: 10.1201/9781420036022.
22. Nekoo, S.R., Acosta, J.A., Ollero, A. (2022) A search algorithm for constrained engineering optimization and tuning the gains of controllers. Expert Systems with Applications. 206. 117866. DOI:10.1016/j.eswa.2022.117866.
23. Niculina Dragoi, E., Dafinescu, V. (2021) Review of metaheuristics inspired from the animal kingdom. Mathematics, 9(18). Art. 2335. DOI: 10.3390/math9182335.
24. Sangren, E. (1990) Nonlinear integer and discrete programming in mechanical design optimization. Mech.Des.-T.ASME, 112(2), 223–229.
25. Sergeyev, Y. D., Kvasov, D. E. (2017) Deterministic Global Optimization: An Introduction to the Diagonal Approach. – New York, NY, USA: Springer.
26. Tomar, V., Bansal, M., Singh, P. (2024) Metaheuristic algorithms for optimization: A brief review. Engineering Proceedings, 59 (1). Art. 238. DOI: 10.3390/engproc2023059238.
27. Tzanetos, A., Fister, I., Dounias, G. (2020) A comprehensive database of nature-inspired algorithms. Data in Brief, 31. Art. 105792. DOI: 10.1016/j.dib.2020.105792.
28. Wolpert, D. H., Macready, W. G. (1997) No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67–82.
29. Yang, X. S., Chien, S. F., Ting, T. O. (2015) Bio-inspired Computation and Optimization. – New York, NY, USA: Morgan Kaufmann. DOI: 10.1016/B978-0-12-801538-4.00001-X.