Embedding Physics-Informed Neural Networks into Numerical Schemes for Modeling Dynamical Systems

Context and relevance. Physics-Informed Neural Networks (PINNs) are considered a promising tool for mathematical modeling of dynamical systems described by differential equations. However, classical PINN approaches require repeated computation of high-order derivatives, which leads to significant computational costs and limits their applicability in modeling tasks. Objective. To develop and experimentally validate an approach to mathematical modeling of dynamical systems based on embedding physics-informed neural networks into classical numerical schemes. Hypothesis. Embedding numerical schemes into the architecture of physics-informed neural networks can improve the computational efficiency of dynamical system modeling while maintaining accuracy comparable to classical PINN approaches. Methods and materials. A neural network architecture integrating a generalized θ-scheme (trapezoidal method) directly into the PINN architecture is proposed. A compact parametric network with 13 trainable parameters adaptively selects the balance between explicit and implicit numerical schemes at different points of the space–time domain. The experimental study was conducted on four types of differential equations characteristic of various dynamical processes: the heat equation, wave equation, reaction–diffusion equation, and Burgers equation. Results. In the experiments, under a fixed training time budget, the proposed 13-parameter model achieves comparable or higher accuracy than a 67-parameter classical PINN, while exhibiting significantly lower variance across runs. Conclusions. Integrating numerical schemes into the architecture of physics-informed neural networks improves the efficiency of constructing dynamical system models and provides interpretability of the architecture through the underlying numerical methods. The proposed approach can be considered a basis for building compact models in surrogate modeling tasks.

1. Antonov, V.I., Tarkhov, D.A., Vasilyev, A.N. (2018). Unified approach to constructing the neural network models of real objects. Part 1. Mathematical Methods in the Applied Sciences, 41(18), 9244—9251. https://doi.org/10.1002/mma.5205
2. Cuomo, S., Di Cola, V.S., Giampaolo, F., Rozza, G., Raissi, M., Piccialli, F. (2022). Scientific machine learning through physics-informed neural networks: Where we are and what's next. Journal of Scientific Computing, 92(3), Article 88. https://doi.org/10.1007/s10915-022-01939-z
3. Hairer, E., Nørsett, S.P., Wanner, G. (1993). Solving ordinary differential equations I: Nonstiff problems (2nd rev. ed.). Berlin: Springer. https://doi.org/10.1007/978-3-540-78862-1
4. Jagtap, A.D., Kawaguchi, K., Karniadakis, G.E. (2020). Adaptive activation functions accelerate convergence in deep and physics-informed neural networks. Journal of Computational Physics, 404, Article 109136. https://doi.org/10.1016/j.jcp.2019.109136
5. Jiang, P., Zhou, Q., Shao, X. (2020). Surrogate model-based engineering design and optimization. Singapore: Springer. https://doi.org/10.1007/978-981-15-0731-1
6. Karniadakis, G.E., Kevrekidis, I.G., Lu, L., Perdikaris, P., Wang, S., Yang, L. (2021). Physics-informed machine learning. Nature Reviews Physics, 3(6), 422—440. https://doi.org/10.1038/s42254-021-00314-5
7. Lazovskaya, T.V., Tarkhov, D.A. (2016). Multilayer neural network models based on grid methods. IOP Conference Series: Materials Science and Engineering, 158, Article 012061. https://doi.org/10.1088/1757-899X/158/1/012061
8. Lazovskaya, T.V., Tarkhov, D.A. (2026). Numerics as neural networks: A compact low-fidelity layer for multi-fidelity modelling. In: Advances in neural computation, machine learning, and cognitive research IX (pp. 381—391). Cham: Springer. https://doi.org/10.1007/978-3-032-07690-8_31
9. Lazovskaya, T.V., Tarkhov, D.A., Vasilyev, A.N. (2018). Multi-layer solution of heat equation. In: B. Kryzhanovsky, W. Dunin-Barkowski, V. Redko (Eds.), Advances in neural computation, machine learning, and cognitive research (Studies in Computational Intelligence, Vol. 736, pp. 17—22). Cham: Springer. https://doi.org/10.1007/978-3-319-66604-4_3
10. Manzoni, L., Papetti, D.M., Cazzaniga, P., Spolaor, S., Mauri, G., Besozzi, D., Nobile, M.S. (2020). Surfing on fitness landscapes: A boost on optimization by Fourier surrogate modeling. Entropy, 22(3), Article 285. https://doi.org/10.3390/e22030285
11. Peherstorfer, B. (2019). Multifidelity Monte Carlo estimation with adaptive low-fidelity models. SIAM/ASA Journal on Uncertainty Quantification, 7(2), 579—603. https://doi.org/10.1137/17M1159208
12. Penwarden, M., Zhe, S., Narayan, A., Kirby, R.M. (2022). Multifidelity modeling for physics-informed neural networks (PINNs). Journal of Computational Physics, 451, Article 110844. https://doi.org/10.1016/j.jcp.2021.110844
13. Raissi, M., Perdikaris, P., Karniadakis, G.E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378, 686—707. https://doi.org/10.1016/j.jcp.2018.10.045
14. Ranftl, S., von der Linden, W. (2021). Bayesian surrogate analysis and uncertainty propagation. Physical Sciences Forum, 3(1), Article 6. https://doi.org/10.3390/psf2021003006
15. Tarkhov, D.A., Lazovskaya, T.V., Malykhina, G.F. (2023). Constructing physics-informed neural networks with architecture based on analytical modification of numerical methods by solving the problem of modelling processes in a chemical reactor. Sensors, 23(2), Article 663. https://doi.org/10.3390/s23020663
16. Tarkhov, D.A., Vasilyev, A.N. (2019). Semi-empirical neural network modeling and digital twins development. Cambridge, MA: Academic Press.
17. Vasilyev, A.N., Tarkhov, D.A. (2014). Mathematical models of complex systems on the basis of artificial neural networks. Nonlinear Phenomena in Complex Systems, 17(3), 327—335.
18. Wang, S., Teng, Y., Perdikaris, P. (2021). Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5), A3055—A3081. https://doi.org/10.1137/20M1318043
19. Zhou, M., Mei, G., Xu, N. (2023). Enhancing computational accuracy in surrogate modeling for elastic–plastic problems by coupling S-FEM and physics-informed deep learning. Mathematics, 11(9), Article 2016. https://doi.org/10.3390/math11092016