Structure Preserving PINN for Solving Time Dependent PDEs with Periodic Boundary
Baoli Hao Ulisses Braga-Neto Chun Liu Lifan Wang Ming Zhong

An Analysis of Recent Advances in Deepfake Image Detection in an
 Evolving Threat Landscape
    Sifat Muhammad Abdullah, 
Aravind Cheruvu, Shravya Kanchi,
 Taejoong Chung, Peng Gao, Murtuza Jadliwala1, Bimal Viswanath

Virginia Tech, 1UT San Antonio



{sifat, acheruvu, shravya, tijay, penggao, vbimal}@vt.edu, 1murtuza.jadliwala@utsa.edu

    April 29, 2024
========================================================================================================================================================================================================================================================

 
We present a structure preserving PINN for solving a series of time dependent PDEs with periodic boundary.  Our method  can incorporate the periodic boundary condition as the natural output of any deep neural net, hence significantly improving the training accuracy of baseline PINN.  Together with mini-batching and other PINN variants (SA-PINN, RBA-PINN, etc.), our structure preserving PINN can even handle stiff PDEs for modeling a wide range of convection-diffusion and reaction-diffusion processes.  We demonstrate the effectiveness of our PINNs on various PDEs from Allen Cahn,  Gray Scott to nonlinear Schrödinger.  









§ INTRODUCTION


Time-dependent partial differential equations (PDEs) hold a ubiquitous presence across numerous scientific and engineering models, underpinning various physical phenomena <cit.>. Particularly, when coupled with periodic boundary conditions, these PDEs become instrumental in modeling phenomena on infinite physical domains. However, the task of solving such PDEs often poses significant challenges. In response to these challenges, we propose an innovative series of machine learning-based solvers tailored for handling time-dependent PDEs, with a specific emphasis on forward evolution of the data. The utilization of machine learning methodologies has demonstrated notable success in various scientific endeavors, ranging from the intricate process of protein folding <cit.>, new drug discoveries <cit.>, to expediting matrix multiplication <cit.>. Traditional data-driven machine learning approaches rely heavily on vast amounts of training data, sometimes yielding models that lack interpretability due to their detachment from the physical understanding of the problem domain. Moreover, in numerous scientific domains, acquiring high-fidelity data proves to be both expensive and time-consuming.  The emerging field of physics-informed machine learning presents a promising avenue to address these challenges. By integrating constraints derived from physical laws, physics-informed machine learning enables the prediction of complex system behaviors using sparse data <cit.>. However, the successful training of physics-informed machine learning models remains a formidable task <cit.>. Fundamental challenges include the difficulty in propagating information away from data, particularly in scenarios involving stiff PDEs.

In this paper, we introduce a novel algorithm designed to preserve the structural integrity of physics-informed neural networks (PINNs). Our approach involves embedding information regarding initial and boundary conditions directly into the neural network architecture, thereby reducing the reliance on spectral bases in training various types of losses <cit.>. This architectural modification not only accelerates the training process but also enhances the accuracy of solutions derived from PINNs.

Our key insight lies in recognizing that collocation-based machine learning solvers, utilized for training PINNs, can be viewed as a specialized form of regularized regression. In this framework, the initial and boundary conditions, along with observed data points, serve as the basis for training, while collocation points act as regularization data. However, the multi-objective nature of the training loss often poses challenges in finding global minimizers. Furthermore, the solution manifolds of most time-dependent PDEs exhibit sensitivity to initial and boundary conditions, where slight variations may yield vastly different outcomes. By integrating initial and boundary condition data into the neural network structure—thus preserving the underlying problem structure—we simplify the PINN training process, particularly for stiff time-dependent PDEs. Moreover, our approach can be augmented with existing training enhancement techniques, such as self-adaptive weights <cit.>, causal PINNs <cit.>, and time-marching PINNs <cit.>, Residual-based Attention PINN <cit.>, co-training PINN <cit.>, further refining prediction accuracy.



  §.§.§ Related Work


The foundational PINN algorithm, while demonstrating remarkable efficacy across numerous applications, occasionally exhibits shortcomings in accurately approximating solutions or achieving convergence, particularly when applied to “stiff” PDEs, where a small time scale is required in order for traditional numerical algorithms to stay stable. Extensive research indicates that such challenges stem from an imbalance within the PINN loss function between its data-fitting and residual components <cit.>. Notably, gradient descent tends to prioritize minimizing the residual loss over the data-fitting component, impeding convergence to the correct solution. This issue is particularly pronounced in forward problems, where all available data are confined to initial and boundary conditions, with scant information within the PDE domain. For instance, in time-evolution scenarios, neural network training struggles to propagate information from initial conditions to subsequent time steps, as corroborated by several researchers <cit.>.

In addressing these challenges, <cit.> proposed a method to propagate information forward in time by segmenting the time axis into smaller intervals, sequentially training PINNs on each segment, starting from the interval closest to the initial condition. However, this approach is time-intensive due to the necessity of training multiple PINNs. Furthermore, both <cit.> and <cit.> emphasized the importance of weighting initial condition data and residual points near t = 0 to effectively propagate information forward, as evidenced in nonlinear Allen-Cahn PDE benchmarks. Notably, <cit.> demonstrated that their self-adaptive PINNs autonomously learn this weighting during training, a phenomenon observed in benchmarks featuring complex initial conditions, such as the Wave equation.  Similarly, <cit.> proposed a “sequence-to-sequence” approach, wherein the PINN sequentially learns each time step in a time-marching scheme, further addressing the propagation challenge. <cit.> identified a lack of “causality” in standard PINN training and proposed a corrective weighting scheme that gradually prioritizes data from early to later times, leveraging information from the PDE residue itself, akin to the self-adaptive weighting mechanism proposed by <cit.>. Moreover, recent developments, such as the pseudo-label approach introduced in <cit.>, bear resemblance to self-training methodologies in PINNs, further diversifying the arsenal of techniques available for enhancing convergence and accuracy in PINN-based solutions.

Finally, a characteristic-informed neural networks (CINNs) framework was introduced in <cit.> for training PINNs for solving transport equations, where the PDE information is built-in to the structure of the neural networks.  Similar approach was proposed in <cit.> for a Dirichlet BC, our method is essential similar to that approach but with a more careful design of stability in mind.



  §.§.§ Contributions of this Work


In this paper, we proposed a structure-preserving PINN algorithm by building the IC/BC information into the structure of neural networks, in order to reduce the training difficulties for PINNs for solving various stiff time-dependent PDEs.


  * We develop structure preserving PINN algorithms, and investigate their stability, training speed, and prediction accuracy. The structure preserving PINN is similar to the method proposed recently in <cit.>.  However in that paper, PDE is built into the structure; whereas ours build the IC/BC into the PINNs.

  * We investigate how structure preserving ameliorates the issue of propagating information forward in time, which is a common failure mode of PINNs.

  * We propose the first structure preserving PINNs which incorporate both IC and BC data in a neural networks.




§ METHODOLOGY


We consider the following setup for a family of time dependent PDEs.  Let u be an unknown scalar function defined on [0, T]×Ω with Ω⊂ℛ^d.  Here the physical domain Ω comes with a Lipschitz boundary ∂Ω.  Let 𝒫 be a spatial partial differential operator, i.e. for the heat equation u_t - λ u_xx = 0, 𝒫 = λ∂^2/∂ x^2.  Moreover 𝒫 can be nonlinear.  Let ℬ be another partial differential operator on the boundary, i.e. for Dirichlet boundary condition u(t, x) = g(t, x) for (t, x) ∈ [0, T]×∂Ω, ℬ = ℐ (the identity).  Then we say that u is a solution of an PDE, if u satisfies the following 

    u_t - 𝒫(u)(t, )    = 0,    (t, ) ∈ (0, T]×Ω, 
    
    u(0, )    = u_0(),   ∈Ω̅= Ω∪∂Ω, 
    ℬ(u)(t, )    = g(t, ),    (t, ) ∈ [0, T]×∂Ω,

Here the two functions u_0, g are user-input and are assumed to satisfy desired regularity so that the existence and uniqueness of solutions for such a PDE is guaranteed.  We also assume that the compactibility condition, i.e., g(0, ) = u_0() for ∈∂Ω, is satisfied in order to prevent any ill-conditioning of the PDE problem.

Types of Boundary Conditions: one can consider many kinds of boundary conditions as follows

    ℬ(u)(t, )    = u(t, ),   Dirichlet type, 
    ℬ(u)(t, )    = ∂ u/∂(t, ),   Neumann type,  is the outward normal vector to ∂Ω, 
    ℬ(u)(t, ;a, b)    = au(t, ) + b∂ u/∂(t, ),   Robin type.

The periodic boundary condition gives d different equations, and they are given as follows,

    ℬ(u)(t, ) = u(t, ) - u(t,  + P_i) = 0,    i = 1, ⋯, d.

Here P is the period and _i is the i^th standard basis vector in ^d, i.e.

    _i = [ 0 ⋯ 0 1 0 ⋯ 0 ]^⊤_1 is the in the i^th position..

One can even consider a mixture of two or all of the above BC conditions, i.e. where Ω = Γ_1 ∪⋯∪Γ_K and each Γ_i defines a different kind of BC.  In this paper, we will mainly focus on the periodic type of boundary conditions.

Classical numerical techniques such as the Finite Difference Method (which approximates derivatives on a point mesh), Finite Element Method (utilizing basis approximation on a triangular mesh with weak formulation), and Spectral Method (employing Fourier basis with FFT on a point mesh) are widely utilized in scientific computations. However, recent advancements in scientific machine learning integrate the principles of physics, particularly the understanding of PDEs, into the training process of machine learning models. This integration enables the development of PINNs to solve for u as follows:  one tries to find an approximate solution from a set of deep neural networks (of the same depth, same number of neurons on each hidden layer, and same activation functions on each layer) ℋ_NN, which is a minimizer of the following loss functional

    Loss(u_nn)    = 1/N_CL∑_i = 1^N_CL(∂ u/∂ t - 𝒫(u_nn))(t_i^CL, _i^CL)^2 + 1/N_IC∑_i = 1^N_ICu_nn(0, _i^IC) - u_0(_i^IC)^2 
          + ∑_i = 1^d1/N_BC∑_j = 1^N_BCu_nn(t_j^BC, _j^BC) - u_nn(t_j^BC, _j^BC + P_i)^2
    ,   ∀ u_nn∈ℋ_NN

Here {(t^CL, ^CL)_i}_i = 1^N_CL∈ (0, T]×Ω are called collocation points, {(0, ^IC)_i}_i = 1^N_IC∈{0}×Ω̅ are the initial condition points, and {(t^BC, ^BC)_i}_i = 1^N_BC∈ [0, T]×∂Ω are the boundary condition points.  The minimizer, denoted as u_NN, will an be approximate solution to (<ref>).  



 §.§ Hard Constrains


The major motivation for us to consider hard constrains transformation for PINNs is the possible ill-conditioning of the PDE solution map's dependence on IC and BC.  Traditional numerical methods do not need to encounter this kind of difficulty, as the IC data is exactly satisfied in their solutions.  PINNs, on the other hand, use L^2-loss to fit the IC/BC and PDE residual data, causing the solutions to be highly sensitive to the goodness of fit of the IC in the training (as pointed out in <cit.>).  By moving the IC/BC data into the structure of PINNs, it reduces the ill-conditioning of the solutions.  Therefore, We consider the following transformation

    ũ(t, ) = ψ(t, ) + ϕ(t, )u_nn(t, ).

Here the two functions ψ and ϕ are both smooth function with the following properties

    ϕ(0, ) = 0   for ∈Ω̅  and  ϕ(t, ) = 0,   for ∈∂Ω;

and

    ψ(0, ) = u_0()   for ∈Ω̅  and  ψ(t, ) = 0,   for ∈∂Ω.

For the Neumann and Robin type, a more complicated ψ and ϕ have to be designed.  For the periodic BC, we simply require ψ and ϕ to be P-periodic.  If we use such transformation, then the training for u_nn is changed, due to the fact that IC/BC can be built into the PDE loss, i.e.

    Loss(u_nn) = 1/N_CL∑_i = 1^N_CL(ũ_t - 𝒫(ũ))(t_i^CL, _i^CL)^2,   for ũ = ψ +ϕ u_nn.

We have reduced the kinds of losses in the training; however the information about IC and BC has then moved into the PDE residual loss through ψ and ϕ.  The difficulty in reducing the new PDE residual loss has been increased.  Hence, other training enhancement techniques can be used to greatly improve training.

Periodic Neural Networks: even though we require ψ and ϕ to be periodic, we still need to build the information of periodicity into u_nn, hence we consider the following composition

    u_nn(t, x) = f_nn(v(t, x)),   f_nn is a neural network, and x ∈,

and 

    v(t, x) = [           t           1  cos(2π/Px)  sin(2π/Px) cos(2π/P2x) sin(2π/P2x)           ⋯ cos(2π/Pmx) sin(2π/Pmx) ]^⊤,

where m > 0 is a positive integer (another hyper-parameter).  With this composition, u_nn will be automatically P-periodic <cit.>.  For higher dimensional physical , similar transformation can be used. (write the 2D here).



 §.§ Mini-batching and Other Techniques


As we have mentioned in the previous section, although we have reduced the kinds of losses to one PDE residual loss, the information about IC and BC have been transferred into the new PDE residual loss.  So we consider mini-batching in the training for decreasing the PDE residual loss.  As pointed out in <cit.>, the mini-batching technique is similar to time-marching resampling.  But with mini-batching, one do not consider an explicit time-grid.  Similar training enhancement can be added, such as SA PINN <cit.>, RBA PINN <cit.>, xPINN <cit.>.



§ NUMERICAL EXPERIMENTS


In this section, We will evaluate our proposed PINN scheme in the previous section on multiple prototypical time dependent PDEs with periodic boundary. 

First of all, we use a Latin hypercube sampling (LHS) strategy to sample the collocation points in the whole domain including the boundary. We next take a simple transformation as follows

    ũ(t, ) = u_0() * exp(-t)+ t*u_nn(t, ),    (t, ) ∈ [0, T]×Ω̅,

where ψ(t, ) = u_0() and ϕ(t, ) = t. 
Through the construction of the transformed u-network and the specification of appropriate loss function, we train our neural network using the generated points. Next, we utilize the trained model to approximate the solution to the partial differential equations (PDEs).

To ascertain the accuracy of our results, we use Chebfun, a spectral-style system implemented in MATLAB for handling functions in an object-oriented manner. This allows us to obtain solutions with high accuracy against which we can rigorously compare the performance of our scheme. Given the data points {x_i,t_i}_i=1^N with N the total number of points, we take  relative L_2 norm of the "exact" value u(x_i,t_i) at those points and the network output 𝒰(x_i,t_i) at those points to evaluate the accuracy of our trained model:
    ℰ = √(∑_i=1^N |𝒰(x_i,t_i)-u(x_i,t_i)|^2)/√(∑_i=1^N |u(x_i,t_i)|^2).



 §.§ Solving the Viscous Burger's Equation

The viscous Burger's Equation is a fundamental partial differential equation and convection-diffusion equation used in fluid dynamics and nonlinear waves <cit.>.  

For a given field u(x,t), and viscosity ν, the general form of viscous Burgers' equation is:

    u_t +uu_x = ν u_xx ,    (x,t) ∈ [-1,1] × [0,∞) ,
        u(x,0) = u_0(x) ,    -1 < x < 1 ,         
       u(1,t) = u(-1,t) = 0 ,    t>0 .

The parameters we used to train are in Table <ref>:


It trains with 50k Adam steps with learning rate at 5 × 10^-3 first, and then uses the L-BFGS-B optimizer to fine-tune the neural network. 
The results are shown in Figure <ref> and Figure <ref>:





 §.§ Solving the Allen-Cahn Equation


Allen-Cahn type PDE is a classical phase-field model, has  been widely used to investigate the phenomena of phase separation <cit.>.
The Allen-Cahn equation has numerous practical applications across various fields, such as material science <cit.>, biological systems <cit.>, electro-chemical systems <cit.> etc.
We first tested the one-dimensional Allen-Cahn Equation with periodic boundary conditions, as follows:

    u_t - γ_1 u_xx +γ_2 u^3-γ_2 u = 0,     (t, x) ∈ (0, T)×(a, b), 
       u(0, x) = u_0(x),    x ∈ [a, b], 
       u(t, a) = u(t, b),    t ∈ [0, T], 
       u_x(t,a) = u_x(t,b),    t ∈ [0, T],

where γ_1,γ_2 > 0, T > 0, a < b are prescribed constants. As γ_2 increases, the transition interface of the solutions is sharper, which makes it harder to solve the AC equation numerically. 



  §.§.§ Case I

Therefore, we demonstrate the effectiveness of our  scheme by testing on AC PDE with a large γ_2:

    u_0(x) = x^2cos(π x),    T = 1,    a = -1,    b = 1,   γ_1 = 0.001,   γ_2 =5.

We train a PINN solution using the same parameters in Table <ref>. 

Figure <ref> and <ref> show the results: 







Table <ref> compares the errors among the three approaches: baseline PINN, re-sampling technique proposed by <cit.> and our approach. Unfortunately, using the standard PINN approach alone, we were not able to learn the accurate solution for the Allen-Cahn equation, and capture the dynamics of equation.  The relative error is almost equal to 1. Compared with the technique proposed by <cit.>, the error is much less and the accuracy is improved. 



  §.§.§ Case II

Next, we explore how our enhanced PINN approach performs in handling scenarios with sharper moving interfaces. Specifically, we modify the initial conditions of the Allen-Cahn equation to introduce more oscillations, and vary the parameters to observe the method's effectiveness across different problem settings. Instead of Equation <ref> in case I, we use the following equations: 
    u_t - γ_1 u_xx +γ_2 u^3-γ_2 u = 0,     (t, x) ∈ (0, 1)×(-1, 1), 
       u(0, x) = x^2sin(2π x),    x ∈ [-1, 1], 
       u(t, -1) = u(t, 1),    t ∈ [0, 1], 
       u_x(t,-1) = u_x(t,1),    t ∈ [0, 1],

where γ_1 = 0.001,γ_2=4.

Figure <ref> and <ref> show the results: 





 §.§ Solving the Cahn-Hilliard Equation

Related to Allen-Cahn Equation, Cahn-Hilliard Equation also describes the process of phase separation, by which the two components of a binary fluid spontaneously separate and form domains pure in each component <cit.>. It is defined as the following equation: 

    u_t = ϵ_1(-u_xx  - ϵ_2u_xxxx + (u^3)_xx),    (t, x) ∈ (0, T]×(-L, L),
       u(0, x) = u_0(x),     x ∈ [-L, L], 
       u(t, -L) = u(t, L),     t ∈ [0, T],

where ϵ_1=10^-2, ϵ_2=10^-4, T=1, L=1. We specify the initial condition as u_0(x) = -cos(2π x). This PDE  has higher order derivatives and is known to be harder to solve than the Allen-Cahn equation. We train the neural network by using 7 layers, 128 neurons and 16,384 collection points with 50k steps. The results are shown in the following Figures <ref> and Figure <ref>: 





 §.§ Solving the Kuramoto-Sivashinsky equation

 Kuramoto–Sivashinsky equation is a fourth-order nonlinear partial differential equation, known for its chaotic behavior <cit.>, as follows:
 
    u_t =  -u_xx - u_xxxx - uu_x,     (t, x) ∈ (0, T]×(a, b), 
       u(0, x) = u_0(x),    x ∈ [a, b], 
       u(t, a) = u(t, b),    t ∈ [0, T],

where T=20, a=0, b=32π, and u_0(x) = cosx/16(1 + sinx-1/16). The parameters are the same with Table <ref>. The results are shown in Figure <ref> and <ref>:








 §.§ Solving the Gray-Scott Equation

Reaction and diffusion of chemical species can produce a variety of patterns <cit.>, reminiscent of those often seen in nature. The Gray-Scott type system is one of classical mathematical models for chemical reactions <cit.>. The general irreversible GS equations describe such reactions: 
    U + 2V    ⟶ 3V, 
    
             V    ⟶ P.

This system is defined by two equations that describe the dynamics of two reacting substances:

    u_t = ϵ_1 u_xx + b(1 - u) - uv^2,    (t, x) ∈ (0, T]×(-L, L),
       v_t = ϵ_2  v_xx -(b+k)v + uv^2, 
       u(0, x) = u_0(x),    v(0, x) = v_0(x),   ∀ x ∈ [-L, L], 
       u(t, -L) = u(t, L),    v(t, -L) = v(t, L),   ∀ t ∈ [0, T],j

where T=20,L=50, ϵ_1=1,ϵ_2=0.01 are diffusion rates, b=0.02 is the "feeding rate" that adds U, k=0.0562 is the  "killing rate" that removes V. We set our initial conditions as: 
    u_0(x) =1- sin(π (x - 50)/100)^4/2,   v_0(x) =sin(π (x - 50)/100)^4/4.

The parameters that we used are the same with before. Figures <ref> and <ref> show the results:


Figure <ref> and Figure<ref> show the results of exact solutions of the 1D GS Equations that we obtain using Chebfun, with the corresponding network predictions and the absolute error differences:









 §.§ Solving the Belousov-Zhabotinsky equation

A Belousov–Zhabotinsky reaction is one of a class of reactions that serve as a classical example of non-equilibrium thermodynamics. It typically involves the oxidation of organic compounds by bromine in an acidic medium <cit.>. The system is as follows:

    u_t = ϵ_1u_xx+u + v - uv - u^2 ,
       v_t = ϵ_2 v_xx+w - v - uv,
       w_t = ϵ_1 w_xx+u - w,    (t, x) ∈ (0, T]×[-L, L]
       u(0, x) = u_0(x),    v(0, x) = v_0(x),    w(0, x) = w_0(x),   ∀ x ∈ [-L, L] 
       u(t, -L) = u(t, L),     v(t, -L) = v(t, L),    w(t, -L) = w(t, L),   ∀ t ∈ [0, T],

where T=3,L=1, ϵ_1=10^-5,ϵ_2=2× 10^-5 are diffusion rates. We let the initial conditions be:
    u_0(x) = exp(-100(x+0.5)^2),   v_0(x) =exp(-100x^2),   w_0(x)=exp(-100(x-0.5)^2).

The results are: 



The comparison between the exact solutions of (u,v,w) equations with corresponding neural network predictions by our approach on the whole (x,t) domain are shown in Figures <ref>, <ref> and <ref>:







 §.§ Solving the Nonlinear Schroedinger equation

In theoretical physics, nonlinear schroedinger equation is a nonlinear PDE, applicable to classical and quantum mechanics <cit.>.  The dimensionless equation of the classical field is 
    u_t = i u_xx + i|u|^2u,    (t,x) ∈ [0, 2]×[-π, π],  
         u(0, x) = u_0(x),     x ∈ [-π, π], 
       u(t, -π) = u(t, π),     t ∈ [0, 2],

where we let 
    u_0(x) =2/2 - √(2)cos(x) - 1.


Using the same parameters and training step with before, we obtain the following results:











§ CONCLUSION


We introduce a structure-preserving PINN where the information about IC and BC are built-in to the structure of the neural networks.  It greatly reduces the training difficulties for seven different kinds of stiff 1-dimensional time-dependent PDEs with periodic boundary condition, ranging scalar output, vector output, and to complex number output.  Future work on how to extend such methods to 2- or 3-dimensional PDEs are on the work, and investigation on the optimal choice of ψ and ϕ, i.e. the transformation is ongoing.  



§ ACKNOWLEDGEMENTS


BH and MZ thank Prof. Yannis Kevrekids (JHU) for the discussion of well-conditioning of PDE solution maps, and Prof. George Karniadakis (Brown) for his suggestion on training enhancement methods for PINNs.



§ FUNDING STATEMENT


MZ is supported by NSF-AoF grant #2225507.




johd




[
Allen
  Cahn

Allen
  Cahn

1979
]
allen1979microscopic

allen1979microscopic

Allen, SM.
  Cahn, JW.

1979.
A microscopic theory for antiphase boundary motion and its application to antiphase domain coarsening A microscopic theory for antiphase boundary motion and its application to antiphase domain coarsening.
Acta metallurgica2761085–1095.


[
Anagnostopoulos
, Toscano
, Stergiopulos
  Karniadakis

Anagnostopoulos
 .
2024
]
ANAGNOSTOPOULOS2024116805

ANAGNOSTOPOULOS2024116805

Anagnostopoulos, SJ.
, Toscano, JD.
, Stergiopulos, N.
  Karniadakis, GE.

2024.
Residual-based attention in physics-informed neural networks Residual-based attention in physics-informed neural networks.
Computer Methods in Applied Mechanics and Engineering421116805.
 <https://www.sciencedirect.com/science/article/pii/S0045782524000616> 
https://doi.org/10.1016/j.cma.2024.116805


[
Bazant

Bazant

2017
]
bazant2017thermodynamic

bazant2017thermodynamic

Bazant, MZ.

2017.
Thermodynamic stability of driven open systems and control of phase separation by electro-autocatalysis Thermodynamic stability of driven open systems and control of phase separation by electro-autocatalysis.
Faraday discussions199423–463.


[
Braga-Neto

Braga-Neto

2023
]
braganeto2023characteristicsinformed

braganeto2023characteristicsinformed

Braga-Neto, U.

2023.
Characteristics-Informed Neural Networks for Forward and Inverse Hyperbolic Problems. Characteristics-informed neural networks for forward and inverse hyperbolic problems.


[
Burgers

Burgers

1948
]
burgers1948mathematical

burgers1948mathematical

Burgers, JM.

1948.
A mathematical model illustrating the theory of turbulence A mathematical model illustrating the theory of turbulence.
Advances in applied mechanics1171–199.


[
Cahn
  Hilliard

Cahn
  Hilliard

1958
]
cahn1958free

cahn1958free

Cahn, JW.
  Hilliard, JE.

1958.
Free energy of a nonuniform system. I. Interfacial free energy Free energy of a nonuniform system. i. interfacial free energy.
The Journal of chemical physics282258–267.


[
Cai
, Wang
, Lu
, Zaki
  Karniadakis

Cai
 .
2021
]
cai2021deepm

cai2021deepm

Cai, S.
, Wang, Z.
, Lu, L.
, Zaki, TA.
  Karniadakis, GE.

2021.
DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks DeepM&Mnet: Inferring the electroconvection multiphysics fields based on operator approximation by neural networks.
Journal of Computational Physics436110296.


[
Carracedo-Reboredo
 .
Carracedo-Reboredo
 .
2021
]
CARRACEDOREBOREDO20214538

CARRACEDOREBOREDO20214538

Carracedo-Reboredo, P.
, Liñares-Blanco, J.
, Rodríguez-Fernández, N.
, Cedrón, F.
, Novoa, FJ.
, Carballal, A.
Fernandez-Lozano, C.

2021.
A review on machine learning approaches and trends in drug discovery A review on machine learning approaches and trends in drug discovery.
Computational and Structural Biotechnology Journal194538-4558.
 <https://www.sciencedirect.com/science/article/pii/S2001037021003421> 
https://doi.org/10.1016/j.csbj.2021.08.011


[
Cassani
, Monteverde
  Piumetti

Cassani
 .
2021
]
cassani2021belousov

cassani2021belousov

Cassani, A.
, Monteverde, A.
  Piumetti, M.

2021.
Belousov-Zhabotinsky type reactions: The non-linear behavior of chemical systems Belousov-zhabotinsky type reactions: The non-linear behavior of chemical systems.
Journal of mathematical chemistry59792–826.


[
X. Chen
 .
X. Chen
 .
2022
]
chen2022using

chen2022using

Chen, X.
, Jeffery, DJ.
, Zhong, M.
, McClenny, L.
, Braga-Neto, U.
  Wang, L.

2022.
Using Physics Informed Neural Networks for Supernova Radiative Transfer Simulation. Using physics informed neural networks for supernova radiative transfer simulation.


[
Y. Chen
, Hosseini
, Owhadi
  Stuart

Y. Chen
 .
2021
]
chen2021solving

chen2021solving

Chen, Y.
, Hosseini, B.
, Owhadi, H.
  Stuart, AM.

2021.
Solving and learning nonlinear PDEs with Gaussian processes Solving and learning nonlinear PDEs with Gaussian processes.
Journal of Computational Physics447110668.


[
Coutinho
 .
Coutinho
 .
2023
]
coutinho2022physics

coutinho2022physics

Coutinho, EJR.
, Dall'Aqua, M.
, McClenny, L.
, Zhong, M.
, Braga-Neto, U.
  Gildin, E.

2023.
Physics-informed neural networks with adaptive localized artificial viscosity Physics-informed neural networks with adaptive localized artificial viscosity.
Journal of Computational Physics489112265.
 <https://www.sciencedirect.com/science/article/pii/S0021999123003601> 
https://doi.org/10.1016/j.jcp.2023.112265


[
De Kepper
, Castets
, Dulos
  Boissonade

De Kepper
 .
1991
]
de1991turing

de1991turing

De Kepper, P.
, Castets, V.
, Dulos, E.
  Boissonade, J.

1991.
Turing-type chemical patterns in the chlorite-iodide-malonic acid reaction Turing-type chemical patterns in the chlorite-iodide-malonic acid reaction.
Physica D: Nonlinear Phenomena491-2161–169.


[
Dong
  Ni

Dong
  Ni

2021
]
dong2021method

dong2021method

Dong, S.
  Ni, N.

2021.
A method for representing periodic functions and enforcing exactly periodic boundary conditions with deep neural networks A method for representing periodic functions and enforcing exactly periodic boundary conditions with deep neural networks.
Journal of Computational Physics435110242.


[
Fawzi
 .
Fawzi
 .
2022
]
MMML2022

MMML2022

Fawzi, A.
, Balog, M.
, Huang, A.
, Hubert, T.
, Romera-Paredes, B.
, Barekatain, M.
Kohli, P.

2022.
Discovering faster matrix multiplication algorithms with reinforcement learning Discovering faster matrix multiplication algorithms with reinforcement learning.
Nature61047 - 53.


[
Fibich

Fibich

2015
]
fibich2015nonlinear

fibich2015nonlinear

Fibich, G.

2015.
The nonlinear Schrödinger equation The nonlinear schrödinger equation ( 192).
Springer.


[
Gray
  Scott

Gray
  Scott

1983
]
gray1983autocatalytic

gray1983autocatalytic

Gray, P.
  Scott, S.

1983.
Autocatalytic reactions in the isothermal, continuous stirred tank reactor: isolas and other forms of multistability Autocatalytic reactions in the isothermal, continuous stirred tank reactor: isolas and other forms of multistability.
Chemical Engineering Science38129–43.


[
Gray
  Scott

Gray
  Scott

1984
]
gray1984autocatalytic

gray1984autocatalytic

Gray, P.
  Scott, SK.

1984.
Autocatalytic reactions in the isothermal, continuous stirred tank reactor: Oscillations and instabilities in the system A+ 2B→ 3B; B→ C Autocatalytic reactions in the isothermal, continuous stirred tank reactor: Oscillations and instabilities in the system a+ 2b→ 3b; b→ c.
Chemical Engineering Science3961087–1097.


[
Haitsiukevich
  Ilin

Haitsiukevich
  Ilin

2022
]
haitsiukevich2022improved

haitsiukevich2022improved

Haitsiukevich, K.
  Ilin, A.

2022.
Improved Training of Physics-Informed Neural Networks with Model Ensembles Improved training of physics-informed neural networks with model ensembles.
arXiv preprint arXiv:2204.05108.


[
Horstmann
, Danner
  Bessler

Horstmann
 .
2013
]
horstmann2013precipitation

horstmann2013precipitation

Horstmann, B.
, Danner, T.
  Bessler, WG.

2013.
Precipitation in aqueous lithium–oxygen batteries: a model-based analysis Precipitation in aqueous lithium–oxygen batteries: a model-based analysis.
Energy & Environmental Science641299–1314.


[
Hyman
, Weber
  Jülicher

Hyman
 .
2014
]
hyman2014liquid

hyman2014liquid

Hyman, AA.
, Weber, CA.
  Jülicher, F.

2014.
Liquid-liquid phase separation in biology Liquid-liquid phase separation in biology.
Annual review of cell and developmental biology3039–58.


[
Jagtap
  Karniadakis

Jagtap
  Karniadakis

2020
]
jagtap2020extended

jagtap2020extended

Jagtap, AD.
  Karniadakis, GE.

2020.
Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations.
Communications in Computational Physics2852002–2041.


[
Jin
, Cai
, Li
  Karniadakis

Jin
 .
2021
]
jin2021nsfnets

jin2021nsfnets

Jin, X.
, Cai, S.
, Li, H.
  Karniadakis, GE.

2021.
NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations.
Journal of Computational Physics426109951.


[
Jumper
 .
Jumper
 .
2021
]
PF2019

PF2019

Jumper, J.
, R., E.
, Pritzel, A.
, Green, T.
, Figurnov, M.
, Ronneberger, O.
Hassabis, D.

2021.
Highly accurate protein structure prediction with AlphaFold Highly accurate protein structure prediction with alphafold.
Nature596583 - 589.


[
Kato

Kato

1987
]
kato1987nonlinear

kato1987nonlinear

Kato, T.

1987.
On nonlinear Schrödinger equations On nonlinear schrödinger equations.
 Annales de l'IHP Physique théorique Annales de l'ihp physique théorique ( 46,  113–129).


[
Kevrekidis
, Rasmussen
  Bishop

Kevrekidis
 .
2001
]
kevrekidis2001discrete

kevrekidis2001discrete

Kevrekidis, P.
, Rasmussen, K.
  Bishop, A.

2001.
The discrete nonlinear Schrödinger equation: a survey of recent results The discrete nonlinear schrödinger equation: a survey of recent results.
International Journal of Modern Physics B15212833–2900.


[
Kim
 .
Kim
 .
2016
]
kim2016basic

kim2016basic

Kim, J.
, Lee, S.
, Choi, Y.
, Lee, SM.
, Jeong, D.
 .
2016.
Basic principles and practical applications of the Cahn–Hilliard equation Basic principles and practical applications of the cahn–hilliard equation.
Mathematical Problems in Engineering2016.


[
Krishnapriyan
, Gholami
, Zhe
, Kirby
  Mahoney

Krishnapriyan
 .
2021
]
krishnapriyan2021characterizing

krishnapriyan2021characterizing

Krishnapriyan, A.
, Gholami, A.
, Zhe, S.
, Kirby, R.
  Mahoney, MW.

2021.
Characterizing possible failure modes in physics-informed neural networks Characterizing possible failure modes in physics-informed neural networks.
Advances in Neural Information Processing Systems3426548–26560.


[
Kudryashov

Kudryashov

1990
]
kudryashov1990exact

kudryashov1990exact

Kudryashov, NA.

1990.
Exact solutions of the generalized Kuramoto-Sivashinsky equation Exact solutions of the generalized kuramoto-sivashinsky equation.
Physics Letters A1475-6287–291.


[
Lagaris
, Likas
  Fotiadis

Lagaris
 .
1998
]
Lagaris_1998

Lagaris_1998

Lagaris, I.
, Likas, A.
  Fotiadis, D.

1998.
Artificial neural networks for solving ordinary and partial differential equations Artificial neural networks for solving ordinary and partial differential equations.
IEEE Transactions on Neural Networks95987–1000.
 <http://dx.doi.org/10.1109/72.712178> 
10.1109/72.712178


[
Lee
 .
Lee
 .
2014
]
lee2014physical

lee2014physical

Lee, D.
, Huh, JY.
, Jeong, D.
, Shin, J.
, Yun, A.
  Kim, J.

2014.
Physical, mathematical, and numerical derivations of the Cahn–Hilliard equation Physical, mathematical, and numerical derivations of the cahn–hilliard equation.
Computational Materials Science81216–225.


[
Liang
, Jiang
, Liu
, Wang
  Zhang

Liang
 .
2022
]
liang2022reversible

liang2022reversible

Liang, J.
, Jiang, N.
, Liu, C.
, Wang, Y.
  Zhang, TF.

2022.
On a reversible Gray-Scott type system from energetic variational approach and its irreversible limit On a reversible gray-scott type system from energetic variational approach and its irreversible limit.
Journal of Differential Equations309427–454.


[
Liu
  Wang

Liu
  Wang

2019
]
liu2019multi

liu2019multi

Liu, D.
  Wang, Y.

2019.
Multi-fidelity physics-constrained neural network and its application in materials modeling Multi-fidelity physics-constrained neural network and its application in materials modeling.
Journal of Mechanical Design14112.


[
Liu
  Wang

Liu
  Wang

2021
]
liu2021dual

liu2021dual

Liu, D.
  Wang, Y.

2021.
A Dual-Dimer method for training physics-constrained neural networks with minimax architecture A dual-dimer method for training physics-constrained neural networks with minimax architecture.
Neural Networks136112–125.


[
McClenny
  Braga-Neto

McClenny
  Braga-Neto

2020
]
mcclenny2020self

mcclenny2020self

McClenny, L.
  Braga-Neto, U.

2020.
Self-adaptive physics-informed neural networks using a soft attention mechanism Self-adaptive physics-informed neural networks using a soft attention mechanism.
arXiv preprint arXiv:2009.04544.


[
Michelson

Michelson

1986
]
michelson1986steady

michelson1986steady

Michelson, D.

1986.
Steady solutions of the Kuramoto-Sivashinsky equation Steady solutions of the kuramoto-sivashinsky equation.
Physica D: Nonlinear Phenomena19189–111.


[
Miranville

Miranville

2017
]
miranville2017cahn

miranville2017cahn

Miranville, A.

2017.
The Cahn–Hilliard equation and some of its variants The cahn–hilliard equation and some of its variants.
AIMS Mathematics23479–544.


[
Nishiura
  Ueyama

Nishiura
  Ueyama

1999
]
nishiura1999skeleton

nishiura1999skeleton

Nishiura, Y.
  Ueyama, D.

1999.
A skeleton structure of self-replicating dynamics A skeleton structure of self-replicating dynamics.
Physica D: Nonlinear Phenomena1301-273–104.


[
Rad
, Viardin
, Schmitz
  Apel

Rad
 .
2020
]
rad2020theory

rad2020theory

Rad, MT.
, Viardin, A.
, Schmitz, G.
  Apel, M.

2020.
Theory-training deep neural networks for an alloy solidification benchmark problem Theory-training deep neural networks for an alloy solidification benchmark problem.
Computational Materials Science180109687.


[
Raissi
  Karniadakis

Raissi
  Karniadakis

2018
]
raissi2018hidden

raissi2018hidden

Raissi, M.
  Karniadakis, GE.

2018.
Hidden physics models: Machine learning of nonlinear partial differential equations Hidden physics models: Machine learning of nonlinear partial differential equations.
Journal of Computational Physics357125–141.


[
Raissi
, Perdikaris
  Karniadakis

Raissi
 .
2019
]
raissi2019physics

raissi2019physics

Raissi, M.
, Perdikaris, P.
  Karniadakis, GE.

2019.
Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.
Journal of Computational Physics378686–707.


[
Shen
  Yang

Shen
  Yang

2010
]
shen2010numerical

shen2010numerical

Shen, J.
  Yang, X.

2010.
Numerical approximations of allen-cahn and cahn-hilliard equations Numerical approximations of allen-cahn and cahn-hilliard equations.
Discrete Contin. Dyn. Syst2841669–1691.


[
Shin
, Darbon
  Karniadakis

Shin
 .
2020
]
shin2020convergence

shin2020convergence

Shin, Y.
, Darbon, J.
  Karniadakis, GE.

2020.
On the convergence and generalization of physics informed neural networks On the convergence and generalization of physics informed neural networks.
arXiv preprint arXiv:2004.01806.


[
Takatori
  Brady

Takatori
  Brady

2015
]
takatori2015towards

takatori2015towards

Takatori, SC.
  Brady, JF.

2015.
Towards a thermodynamics of active matter Towards a thermodynamics of active matter.
Physical Review E913032117.


[
Tian
, Mao
, Brown
, Rutledge
  Hatton

Tian
 .
2015
]
tian2015electrochemically

tian2015electrochemically

Tian, W.
, Mao, X.
, Brown, P.
, Rutledge, GC.
  Hatton, TA.

2015.
Electrochemically nanostructured polyvinylferrocene/polypyrrole hybrids with synergy for energy storage Electrochemically nanostructured polyvinylferrocene/polypyrrole hybrids with synergy for energy storage.
Advanced Functional Materials25304803–4813.


[
R. Wang
, Zhong
, Xu
, Sánchez-Cortés
  de Cominges Guerra

R. Wang
 .
2023
]
wang2023pinnsbased

wang2023pinnsbased

Wang, R.
, Zhong, M.
, Xu, K.
, Sánchez-Cortés, LG.
  de Cominges Guerra, I.

2023.
PINNs-Based Uncertainty Quantification for Transient Stability Analysis. Pinns-based uncertainty quantification for transient stability analysis.


[
S. Wang
, Sankaran
  Perdikaris

S. Wang
 .
2022
]
wang2022respecting

wang2022respecting

Wang, S.
, Sankaran, S.
  Perdikaris, P.

2022.
Respecting causality is all you need for training physics-informed neural networks Respecting causality is all you need for training physics-informed neural networks.
arXiv preprint arXiv:2203.07404.


[
S. Wang
, Teng
  Perdikaris

S. Wang
, Teng
  Perdikaris

2020
]
wang2020understanding

wang2020understanding

Wang, S.
, Teng, Y.
  Perdikaris, P.

2020.
Understanding and mitigating gradient pathologies in physics-informed neural networks Understanding and mitigating gradient pathologies in physics-informed neural networks.
arXiv preprint arXiv:2001.04536.


[
S. Wang
, Yu
  Perdikaris

S. Wang
, Yu
  Perdikaris

2020
]
wang2020and

wang2020and

Wang, S.
, Yu, X.
  Perdikaris, P.

2020.
When and why PINNs fail to train: A neural tangent kernel perspective When and why PINNs fail to train: A neural tangent kernel perspective.
arXiv preprint arXiv:2007.14527.


[
Whitham

Whitham

2011
]
whitham2011linear

whitham2011linear

Whitham, GB.

2011.
Linear and nonlinear waves Linear and nonlinear waves.
John Wiley & Sons.


[
Wight
  Zhao

Wight
  Zhao

2020
]
wight2020solving

wight2020solving

Wight, CL.
  Zhao, J.

2020.
Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networks Solving allen-cahn and cahn-hilliard equations using the adaptive physics informed neural networks.
arXiv preprint arXiv:2007.04542.


[
Yang
, Barajas-Solano
, Tartakovsky
  Tartakovsky

Yang
 .
2019
]
yang2019physics

yang2019physics

Yang, X.
, Barajas-Solano, D.
, Tartakovsky, G.
  Tartakovsky, AM.

2019.
Physics-informed CoKriging: A Gaussian-process-regression-based multifidelity method for data-model convergence Physics-informed cokriging: A Gaussian-process-regression-based multifidelity method for data-model convergence.
Journal of Computational Physics395410–431.


[
Zhabotinsky

Zhabotinsky

2007
]
zhabotinsky2007belousov

zhabotinsky2007belousov

Zhabotinsky, AM.

2007.
Belousov-zhabotinsky reaction Belousov-zhabotinsky reaction.
Scholarpedia291435.


[
Zhong
, Liu
, Arroyave
  Braga-Neto

Zhong
 .
2024
]
zhong2024label

zhong2024label

Zhong, M.
, Liu, D.
, Arroyave, R.
  Braga-Neto, U.

2024.
Label Propagation Training Schemes for Physics-Informed Neural Networks and Gaussian Processes. Label propagation training schemes for physics-informed neural networks and gaussian processes.


[
Zhu
, Liu
  Yan

Zhu
 .
2021
]
zhu2021machine

zhu2021machine

Zhu, Q.
, Liu, Z.
  Yan, J.

2021.
Machine learning for metal additive manufacturing: predicting temperature and melt pool fluid dynamics using physics-informed neural networks Machine learning for metal additive manufacturing: predicting temperature and melt pool fluid dynamics using physics-informed neural networks.
Computational Mechanics672619–635.








