论文标题
关于神经网络和偏微分方程的兼容性，用于物理知识学习
On the Compatibility between Neural Networks and Partial Differential Equations for Physics-informed Learning
论文作者
论文摘要
我们阐明了陷阱和物理信息神经网络（PINNS）的机会。我们证明，仅具有relu（整流线性单元）或类似Relu的Lipschitz激活功能的多层感知器（MLP）始终会导致消失的Hessian。这样的网络施加的约束与任何第二或高阶部分微分方程（PDE）矛盾。因此，基于RELU的MLP不能形成允许的函数空间以实现其溶液的近似。受此陷阱的启发，我们证明，当其输出层的重量位于某个超平面上时，MLP的线性PDE可以严格满足，而MLP可以严格满足其$ c^n $激活功能的功能，如称为耗尽的超平台。配备了外层hyperplane的MLP变为“物理性强化”，不再需要PDE本身的损失函数（而是针对初始和边界条件的损失函数）。这样的超平面不仅存在于MLP，而且对于由完全连接的隐藏层尾声的任何网络体系结构都存在。据我们所知，这应该是第一个在PDE上执行点正确性的Pinn架构。我们显示了二阶线性PDE的外层hyperplane的封闭式表达，可以将其推广到高阶非线性PDE。
We shed light on a pitfall and an opportunity in physics-informed neural networks (PINNs). We prove that a multilayer perceptron (MLP) only with ReLU (Rectified Linear Unit) or ReLU-like Lipschitz activation functions will always lead to a vanished Hessian. Such a network-imposed constraint contradicts any second- or higher-order partial differential equations (PDEs). Therefore, a ReLU-based MLP cannot form a permissible function space for the approximation of their solutions. Inspired by this pitfall, we prove that a linear PDE up to the $n$-th order can be strictly satisfied by an MLP with $C^n$ activation functions when the weights of its output layer lie on a certain hyperplane, as called the out-layer-hyperplane. An MLP equipped with the out-layer-hyperplane becomes "physics-enforced", no longer requiring a loss function for the PDE itself (but only those for the initial and boundary conditions). Such a hyperplane exists not only for MLPs but for any network architecture tailed by a fully-connected hidden layer. To our knowledge, this should be the first PINN architecture that enforces point-wise correctness of PDEs. We show a closed-form expression of the out-layer-hyperplane for second-order linear PDEs, which can be generalised to higher-order nonlinear PDEs.
