NEW PROGRESS IN INTELLIGENT SOLUTION OF NEURAL OPERATORS AND PHYSICS-INFORMED-BASED METHODS
-
摘要:深度学习通过多层神经网络对数据进行学习, 不仅能揭示潜藏信息, 还能很好地解决复杂非线性问题. 偏微分方程(PDE)是描述自然界中许多物理现象的基本数学模型. 两者的碰撞与融合, 产生了基于深度学习的PDE智能求解方法, 它具有高效、灵活和通用等优点. 文章聚焦PDE智能求解方法, 以是否求解单一问题为判定依据, 把求解方法分为两类: 神经算子方法和类物理信息神经网络(PINN)方法, 其中神经算子方法用于求解一类具有相同数学特征的PDE问题, 类PINN方法用于求解单一问题. 对于神经算子方法, 从数据驱动和物理约束两个方面展开介绍, 分析研究现状并指出现有方法的不足. 对于类PINN方法, 首先介绍了基础PINN的3种改进方法(基于数据优化、基于模型优化和基于领域知识优化), 然后详细介绍了基于物理驱动的两类解决方案: 基于传统PDE离散方程的智能求解方案和无网格的非离散求解方案. 最后总结技术路线, 探讨现有研究存在的不足, 给出可行的研究方案. 最后, 简要介绍智能求解程序发展现状, 并对未来研究方向给出建议.Abstract:Deep learning, characterized by its multi-layer neural networks, has demonstrated its capability not only to uncover hidden information in data but also to effectively address complex nonlinear problems. As a fundamental mathematical model, partial differential equations (PDEs) find wide application in describing various physical phenomena in the natural world. The amalgamation of deep learning and PDEs has given rise to the emergence of intelligent PDE solving methods based on deep learning. These methods possess several advantageous traits, including high efficiency, flexibility, and universality, which make them valuable in practical applications. This paper focuses on intelligent PDE solving methods, categorizing the solving approaches into two types based on whether they handle single or multiple problems: neural operator methods and physics-informed neural network (PINN) methods. Neural operator methods are employed to solve a class of PDE problems with the same mathematical characteristics, while PINN-based methods are used to solve single problems. The first category encompasses neural operator methods, which are utilized for solving a group of PDE problems that share similar mathematical characteristics. These methods leverage data-driven approaches and physical-constraint approaches to formulate their solutions. An in-depth analysis is conducted to examine the current research status, along with the identification of existing drawbacks in these approaches. As for PINN-based methods, this paper introduces the relevant research progress from three derivative approaches of PINN (data-optimization, model-optimization, and domain-knowledge-optimization). Finally, this paper provides a comprehensive overview of the technical roadmap for PDE intelligent solving methods. It critically evaluates the existing research limitations and proposes feasible research plans to overcome these challenges. Additionally, the paper briefly introduces the current state of intelligent solving program development and offers suggestions for future research directions. By amalgamating deep learning and PDEs, these intelligent solving methods have the potential to revolutionize various scientific and engineering domains, enabling more accurate and efficient problem-solving in complex nonlinear systems.
-
图 1偏微分方程智能求解方法的分类归纳图
Figure 1.Classification and induction diagram of intelligent solution methods for PDEs
表 1神经算子方法及特点
Table 1.Neural operator methods and characteristics
Category Method Feature neural operators data-driven DeepONet[19] nonlinear, multiple input and output functions FNO[20] nonlinear, high dimensional or periodic PDEs Deeponet-grid-uq[23] nonlinear, uncertainty quantification B-DeepONet[24] parametric PDEs, noise data U-FNO[25] nonlinear, multiphase flow MP-PDE solver[26] nonlinear, resolution-independent physical-constraint Pi-deeponet[27] parametric evolution equations, long-term prediction LordNet[28] nonlinear, Navier-Stokes equation, no labels 
下载:
导出CSV
表 2类PINN方法及特点
Table 2.PINN-based methods and characteristics
Category Method Feature PINN-based method input-data-based method DFS-Net[39] nonlinear, sampling weight strategy ADNN[40] nonlinear, high-dimensional PDEs time segmentation[41-42] temporal-causal PDEs model-based method CAN-PINN[36] nonlinear, sparse sampling ND-PINN[43] linear, accelerate solution learning rate annealing[44] nonlinear, multi-scale, turbulent problems DNNsolve[45] nonlinear, periodic function TgNN-LD[46] nonlinear, noise data HOrderDNN[47] high frequency PDE knowledge-based method iPINN[48] nonlinear, reaction-diffusion PDE MAD[49] parametric PDEs metalearning mehtod[49] parametric PDEs 下载:
导出CSV
表 3基于离散和非离散的物理驱动求解方案
Table 3.Physics-driven solution based on discrete and non-discrete
Category Method Feature discrete model a fast solver[56] system of linear equations DeLISA[57] system of nonlinear equations PICNN[58-59] system of nonlinear equations FCGNN[60] system of linear equations ADNN[61] system of 2-order nonlinear equations a nonlinear solver[62] system of nonlinear equations non-discrete model LSTM-AM[63] nonlinear, non-convex flux functions GW-PINN[64] nonlinear, groundwater flow equations phyCRNet[65] nonlinear, periodic oscillation PDEs physical AS-nets[66] nonlinear, seepage equation 下载:
导出CSV
-
[1] Yan B, Harp DR, Chen B, et al. A physics-constrained deep learning model for simulating multiphase flow in 3D heterogeneous porous media.Fuel, 2022, 313: 122693doi:10.1016/j.fuel.2021.122693 [2] 查文舒, 李道伦, 沈路航等. 基于神经网络的偏微分方程求解方法研究综述. 力学学报, 2022, 54(3): 543-556 (Zha Wenshu, Li Daolun, Shen Luhang, et al. Review of neural network-based methods for solving partial differential equations.Chinese Journal of Theoretical and Applied Mechanics, 2022, 54(3): 543-556. (in Chinese)doi:10.6052/0459-1879-21-617 Zha Wenshu, Li Daolun, Shen Luhang, et al . Review of neural network-based methods for solving partial differential equations. Chinese Journal of Theoretical and Applied Mechanics,2022 ,54 (3 ):543 -556 . (in Chinese)doi:10.6052/0459-1879-21-617[3] 宋洪庆, 都书一, 王九龙等. 数智流体力学的发展及油气渗流领域应用. 力学学报, 2023, 55(3): 765-791 (Song Hongqing, Du Shuyi, Wang Jiulong, et al. Development of digital intelligence fluid dynamics and applications in the oil & gas seepage fields.Chinese Journal of Theoretical and Applied Mechanics, 2023, 55(3): 765-791. (in Chinese)doi:10.6052/0459-1879-22-484 Song Hongqing, Du Shuyi, Wang Jiulong, et al . Development of digital intelligence fluid dynamics and applications in the oil & gas seepage fields. Chinese Journal of Theoretical and Applied Mechanics,2023 ,55 (3 ):765 -791 . (in Chinese)doi:10.6052/0459-1879-22-484[4] Stephany R, Earls C. PDE-READ: Human-readable partial differential equation discovery using deep learning.Neural Networks, 2022, 154: 360-382doi:10.1016/j.neunet.2022.07.008 [5] Gao H, Zahr MJ, Wang JX. Physics-informed graph neural Galerkin networks: A unified framework for solving PDE-governed forward and inverse problems.Computer Methods in Applied Mechanics and Engineering, 2022, 390: 114502doi:10.1016/j.cma.2021.114502 [6] Chen Y, Lu L, Karniadakis GE, et al. Physics-informed neural networks for inverse problems in nano-optics and metamaterials.Optics Express, 2020, 28(8): 11618-11633doi:10.1364/OE.384875 [7] Kochkov D, Smith JA, Alieva A, et al. Machine learning–accelerated computational fluid dynamics.Proceedings of the National Academy of Sciences, 2021, 118(21): e2101784118doi:10.1073/pnas.2101784118 [8] Sirignano J, Spiliopoulos K. DGM: A deep learning algorithm for solving partial differential equations.Journal of Computational Physics, 2018, 375: 1339-1364doi:10.1016/j.jcp.2018.08.029 [9] Han J, Nica M, Stinchcombe AR. A derivative-free method for solving elliptic partial differential equations with deep neural networks.Journal of Computational Physics, 2020, 419: 109672doi:10.1016/j.jcp.2020.109672 [10] Cybenko G. Approximation by superpositions of a sigmoidal function.Mathematics of Control, Signals and Systems, 1989, 2(4): 303-314doi:10.1007/BF02551274 [11] Pinkus A. Approximation theory of the MLP model in neural networks.Acta Numerica, 1999, 8: 143-195doi:10.1017/S0962492900002919 [12] Hanin B. Universal function approximation by deep neural nets with bounded width and relu activations.Mathematics, 2019, 7(10): 992doi:10.3390/math7100992 [13] Lusch B, Kutz JN, Brunton SL. Deep learning for universal linear embeddings of nonlinear dynamics.Nature Communications, 2018, 9(1): 4950doi:10.1038/s41467-018-07210-0 [14] Lu L, Meng X, Mao Z, et al. DeepXDE: A deep learning library for solving differential equations.SIAM Review, 2021, 63(1): 208-228doi:10.1137/19M1274067 [15] Zhu Y, Zabaras N, Koutsourelakis PS, et al. Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data.Journal of Computational Physics, 2019, 394: 56-81doi:10.1016/j.jcp.2019.05.024 [16] Raissi M, Wang Z, Triantafyllou MS, et al. Deep learning of vortex-induced vibrations.Journal of Fluid Mechanics, 2019, 861: 119-137doi:10.1017/jfm.2018.872 [17] Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Computational Physics, 2019, 378: 686-707doi:10.1016/j.jcp.2018.10.045 [18] Bi K, Xie L, Zhang H, et al. Accurate medium-range global weather forecasting with 3D neural networks.Nature, 2023, 1-6 [19] Lu L, Jin PZ, Pang GF, et al. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators.Nature Machine Intelligence, 2021, 3(3): 218-229doi:10.1038/s42256-021-00302-5 [20] Li Z, Kovachki N, Azizzadenesheli K, et al. Fourier neural operator for parametric partial differential equations. 2020, arXiv: 2010.08895 [21] Xu C, Cao BT, Yuan Y, et al. Transfer learning based physics-informed neural networks for solving inverse problems in engineering structures under different loading scenarios.Computer Methods in Applied Mechanics and Engineering, 2023, 405: 115852doi:10.1016/j.cma.2022.115852 [22] Penwarden M, Zhe S, Narayan A, et al. A metalearning approach for physics-informed neural networks (PINNs): Application to parameterized PDEs.Journal of Computational Physics, 2023, 477: 111912doi:10.1016/j.jcp.2023.111912 [23] Moya C, Zhang SQ, Lin G, et al. Deeponet-grid-uq: A trustworthy deep operator framework for predicting the power grid’s post-fault trajectories.Neurocomputing, 2023, 535: 166-182doi:10.1016/j.neucom.2023.03.015 [24] Lin G, Moya C, Zhang ZC. B-DeepONet: An enhanced Bayesian DeepONet for solving noisy parametric PDEs using accelerated replica exchange SGLD.Journal of Computational Physics, 473, (2023): 111713 [25] Wen G, Li Z, Azizzadenesheli K, et al. U-FNO—An enhanced Fourier neural operator-based deep-learning model for multiphase flow.Advances in Water Resources, 2022, 163: 104180doi:10.1016/j.advwatres.2022.104180 [26] Brandstetter J, Worrall D, Welling M. Message passing neural PDE solvers. 2022, arXiv: 2202.03376 [27] Wang S, Perdikaris P. Long-time integration of parametric evolution equations with physics-informed deeponets.Journal of Computational Physics, 2023, 475: 111855doi:10.1016/j.jcp.2022.111855 [28] Shi W, Huang X, Gao X, et al. Lordnet: Learning to solve parametric partial differential equations without simulated data. 2022, arXiv: 2206.09418 [29] Zhang K, Zuo YD, Zhao HJ, et al. Fourier neural operator for solving subsurface oil/water two-phase flow partial differential equation.SPE Journal, 2022, 27(3): 1815-1830doi:10.2118/209223-PA [30] Deng Z, Liu H, Shi B, et al. Temporal predictions of periodic flows using a mesh transformation and deep learning-based strategy.Aerospace Science and Technology, 2023, 134: 108081doi:10.1016/j.ast.2022.108081 [31] Kaiser E, Kutz JN, Brunton SL. Sparse identification of nonlinear dynamics for model predictive control in the low-data limit.Proceedings of the Royal Society A, 2018, 474(2219): 20180335doi:10.1098/rspa.2018.0335 [32] Zhong M, Yan Z. Data-driven forward and inverse problems for chaotic and hyperchaotic dynamic systems based on two machine learning architectures.Physica D:Nonlinear Phenomena, 2023, 446: 133656doi:10.1016/j.physd.2023.133656 [33] Bar-Sinai Y, Hoyer S, Hickey J, et al. Learning data-driven discretizations for partial differential equations.Proceedings of the National Academy of Sciences, 2019, 116(31): 15344-15349doi:10.1073/pnas.1814058116 [34] Boussif O, Bengio Y, Benabbou L, et al. MAgnet: Mesh agnostic neural PDE solver.Advances in Neural Information Processing Systems, 2022, 35: 31972-31985 [35] Koric S, Abueidda DW. Data-driven and physics-informed deep learning operators for solution of heat conduction equation with parametric heat source.International Journal of Heat and Mass Transfer, 2023, 203: 123809doi:10.1016/j.ijheatmasstransfer.2022.123809 [36] Chiu PH, Wong JC, Ooi C, et al. CAN-PINN: A fast physics-informed neural network based on coupled-automatic–numerical differentiation method.Computer Methods in Applied Mechanics and Engineering, 2022, 395: 114909 [37] Fang ZW. A high-efficient hybrid physics-informed neural networks based on convolutional neural network.IEEE Transactions on Neural Networks and Learning Systems, 2021, 33(10): 5514-5526 [38] Xie CY, Du SY, Wang JL, et al. Intelligent modeling with physics-informed machine learning for petroleum engineering problems.Advances in Geo-Energy Research, 2023, 8(2): 71-75doi:10.46690/ager.2023.05.01 [39] Chen X, Chen R, Wan Q, et al. An improved data-free surrogate model for solving partial differential equations using deep neural networks.Scientific Reports, 2021, 11(1): 19507doi:10.1038/s41598-021-99037-x [40] Zeng SJ, Zhang Z, Zou QS. Adaptive deep neural networks methods for high-dimensional partial differential equations.Journal of Computational Physics, 2022, 463: 111232 [41] Krishnapriyan A, Gholami A, Zhe S, et al. Characterizing possible failure modes in physics-informed neural networks.Advances in Neural Information Processing Systems, 2021, 34: 26548-26560 [42] Wang S, Sankaran S, Perdikaris P. Respecting causality is all you need for training physics-informed neural networks. 2022, arXiv: 2203.07404 [43] Lim KL, Dutta R, Rotaru M. Physics informed neural network using finite difference method//2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2022: 1828-1833 [44] Wang S, Teng Y, Perdikaris P. Understanding and mitigating gradient flow pathologies in physics-informed neural networks.SIAM Journal on Scientific Computing, 2021, 43(5): A3055-A3081doi:10.1137/20M1318043 [45] Guidetti V, Muia F, Welling Y, et al. DNNsolve: An efficient NN-based PDE solver. 2021, arXiv: 2103.08662 [46] Rong M, Zhang DX, Wang NZ. A Lagrangian dual-based theory-guided deep neural network.Complex & Intelligent Systems, 2022, 8(6): 4849-4862 [47] Chang ZP, Li K, Zou XF, et al. High order deep neural network for solving high frequency partial differential equations.Communications In Computational Physics, 2022, 31: 370-397doi:10.4208/cicp.OA-2021-0092 [48] Dekhovich A, Sluiter MHS, Tax DMJ, et al. iPINNs: Incremental learning for Physics-informed neural networks. 2023, arXiv: 2304.04854 [49] Huang X, Ye Z, Liu H, et al. Meta-auto-decoder for solving parametric partial differential equations.Advances in Neural Information Processing Systems, 2022, 35: 23426-23438 [50] Liu W, Liu Y, Li H. Time difference physics-informed neural network for fractional water wave models.Results in Applied Mathematics, 2023, 17: 100347doi:10.1016/j.rinam.2022.100347 [51] Jin X, Cai S, Li H, et al. NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations.Journal of Computational Physics, 2021, 426: 109951doi:10.1016/j.jcp.2020.109951 [52] Wang NZ, Zhang DX, Chang HB, et al. Deep learning of subsurface flow via theory-guided neural network.Journal of Hydrology, 2020, 584: 124700doi:10.1016/j.jhydrol.2020.124700 [53] Rahaman N, Baratin A, Arpit D, et al. On the spectral bias of neural networks//International Conference on Machine Learning. PMLR, 2019: 5301-5310 [54] Hong Q, Siegel JW, Tan Q, et al. On the activation function dependence of the spectral bias of neural networks. 2022, arXiv: 2208.04924 [55] Penwarden M, Zhe SD, Narayan A, et al. A metalearning approach for Physics-Informed Neural Networks (PINNs): Application to parameterized PDEs.Journal of Computational Physics, 2023, 477: 111912doi:10.1016/j.jcp.2023.111912 [56] 蒋子超, 江俊扬, 姚清河等. 基于神经网络的差分方程快速求解方法. 力学学报, 2021, 53(7): 1912-1921 (Jiang Zichao, Jiang Junyang, Yao Qinghe, et al. A fast solver based on deep neural network for difference equation.Chinese Journal of Theoretical and Applied Mechanics, 2021, 53(7): 1912-1921 (in Chinese)doi:10.6052/0459-1879-21-040 Jiang Zichao, Jiang Junyang, Yao Qinghe, et al . A fast solver based on deep neural network for difference equation. Chinese Journal of Theoretical and Applied Mechanics,2021 ,53 (7 ):1912 -1921 (in Chinese)doi:10.6052/0459-1879-21-040[57] Li Y, Zhou ZJ, Ying SH. DeLISA: Deep learning based iteration scheme approximation for solving PDEs.Journal of Computational Physics, 2022, 451: 110884doi:10.1016/j.jcp.2021.110884 [58] Zhang Z. A physics-informed deep convolutional neural network for simulating and predicting transient Darcy flows in heterogeneous reservoirs without labeled data.Journal of Petroleum Science and Engineering, 2022, 211: 110179doi:10.1016/j.petrol.2022.110179 [59] Zhang Z, Yan X, Liu PY, et al. A physics-informed convolutional neural network for the simulation and prediction of two-phase Darcy flows in heterogeneous porous media.Journal of Computational Physics, 2023, 477: 111919doi:10.1016/j.jcp.2023.111919 [60] Xiao L, Li KL, Tan ZG, et al. nonlinear gradient neural network for solving system of linear equations.Information Processing Letters, 2019, 142: 35-40doi:10.1016/j.ipl.2018.10.004 [61] Ebadi M, Zabihifar SH, Bezyan Y, et al. A nonlinear solver based on an adaptive neural network, introduction and application to porous media flow.Journal of Natural Gas Science and Engineering, 2021, 87: 103749doi:10.1016/j.jngse.2020.103749 [62] Li DL, Lv SJ, Zha WS, et al. A nonlinear solver based on residual network for seepage equation.Engineering Applications of Artificial Intelligence, 2023, 126: 106850doi:10.1016/j.engappai.2023.106850 [63] Shan LQ, Liu CQ, Liu YC, et al. Physics-informed machine learning for solving partial differential equations in porous media.Advances in Geo-Energy Research, 2023, 8(1): 37-44doi:10.46690/ager.2023.04.04 [64] Zhang XP, Zhu Y, Wang J, et al. GW-PINN: A deep learning algorithm for solving groundwater flow equations.Advances in Water Resources, 2022, 165: 104243doi:10.1016/j.advwatres.2022.104243 [65] Ren P, Rao C, Liu Y, et al. PhyCRNet: Physics-informed convolutional-recurrent network for solving spatiotemporal PDEs.Computer Methods in Applied Mechanics and Engineering, 2022, 389: 114399doi:10.1016/j.cma.2021.114399 [66] Shen LH, Li DL, Zha WS, et al. Physical asymptotic-solution nets: physics-driven neural networks solve seepage equations as traditional numerical solution behaves.Physics of Fluids, 2023, 35(2 [67] Xiao L, He YJ, Liao BL. A parameter-changing zeroing neural network for solving linear equations with superior fixed-time convergence.Expert Systems with Applications, 2022, 208: 118086doi:10.1016/j.eswa.2022.118086 [68] Qu JG, Cai WH, Zhao YJ. Learning time-dependent PDEs with a linear and nonlinear separate convolutional neural network.Journal of Computational Physics, 2022, 453: 110928doi:10.1016/j.jcp.2021.110928 [69] Lagaris IE, Likas A, Fotiadis DI. Artificial neural networks for solving ordinary and partial differential equations.IEEE Transactions on Neural Networks, 1998, 9(5): 987-1000doi:10.1109/72.712178 [70] Mattheakis M, Sondak D, Dogra AS, et al. Hamiltonian neural networks for solving equations of motion.Physical Review E, 2022, 105(6): 065305doi:10.1103/PhysRevE.105.065305 [71] Liu XL, Zha WS, Li DL, et al. Automatic well test interpretation method for circular reservoirs with changing wellbore storage by using one-dimensional convolutional neural network.Journal of Energy Resources Technology, 2023, 145(3): 033201 [72] 李道伦, 刘旭亮, 查文舒等. 基于卷积神经网络的径向复合油藏自动试井解释方法. 石油勘探与开发, ISSN 1876-3804, 2020, 47(3): 583-591 (Li Daolun, Liu Xuliang, Zha Wenshu, et al. Automatic well test interpretation method for radial composite reservoir based on convolutional neural network.Petroleum Exploration and Development, 2020, 47(3): 583-591 (in Chinese) Li Daolun, Liu Xuliang, Zha Wenshu, et al . Automatic well test interpretation method for radial composite reservoir based on convolutional neural network. Petroleum Exploration and Development,2020 ,47 (3 ):583 -591 (in Chinese)[73] Shen LH, Li DL, Zha WS, et al. Surrogate modeling for porous flow using deep neural networks.Journal of Petroleum Science and Engineering, 2022, 213: 110460doi:10.1016/j.petrol.2022.110460 [74] Liu Z, Hao YX, Li DL, et al. Multiparameter inversion of reservoirs based on deep learning.SPE Journal, 2023, SPE-217437-PA [75] Li DL, Zhang L, Wang JY, et al. Composition-transient analysis in shale-gas reservoirs with consideration of multicomponent adsorption.SPE Journal, 2016, 21(2): 648-664doi:10.2118/178435-PA -
下载:
点击查看大图
图(7)/
表(3)
计量
- 文章访问数:87
- HTML全文浏览量:46
- PDF下载量:32
- 被引次数:0
