Physics-informed deep learning for incompressible laminar flows

Chengping Raoa, Hao Sunb,c, Yang Liua,∗

aDepartment of Mechanical and Industrial Engineering, Northeastern University, Boston, MA 02115, USAbDepartment of Civil and Environmental Engineering, Northeastern University, Boston, MA 02115, USA

cDepartment of Civil and Environmental Engineering, MIT, Cambridge, MA 02139, USA

Abstract

Physics-informed deep learning has drawn tremendous interest in recent years to solve computational physics problems,whose basic concept is to embed physical laws to constrain/inform neural networks, with the need of less data for traininga reliable model. This can be achieved by incorporating the residual of physics equations into the loss function. Throughminimizing the loss function, the network could approximate the solution. In this paper, we propose a mixed-variablescheme of physics-informed neural network (PINN) for fluid dynamics and apply it to simulate steady and transientlaminar flows at low Reynolds numbers. A parametric study indicates that the mixed-variable scheme can improve thePINN trainability and the solution accuracy. The predicted velocity and pressure fields by the proposed PINN approachare also compared with the reference numerical solutions. Simulation results demonstrate great potential of the proposedPINN for fluid flow simulation with a high accuracy.

Keywords: Physics-informed neural networks (PINN), deep learning, fluid dynamics, incompressible laminar flow

1. Introduction

Deep learning (DL) has attracted tremendous at-tentions in recent years in the field of computationalmechanics due to its powerful capability in nonlinearmodeling of complex spatiotemporal systems. Ac-cording to a technical report [1] by U.S. Departmentof Energy, a DL-based approach should be featuredwith the domain-aware, interpretable and robust tobe a general approach for solving the science and en-gineering problems. Recent studies of leveraging DLto model physical system include, just to name a few,[2–6]. These applications can be categorized into twotypes based on how the DL model is constructed: ineither data-driven or physics-informed manner. In adata-driven framework, the DL model is constructedas a black-box to learn a surrogate mapping fromthe formatted input x ∈ Rm to the output y ∈ Rn.The exceptional approximation ability of deep neu-ral network (DNN) makes possible to learn the map-ping even when the dimensionality m and n are veryhigh. The training dataset {x,y}, typically very rich,can be obtained by conducting high-fidelity simula-

∗Corresponding author. Tel: +1 617-373-8560Email addresses: h.sun@northeastern.edu (Hao Sun),

yang1.liu@northeastern.edu (Yang Liu)

tions using exact solvers (e.g., see [3, 4, 7]). Never-theless, obtaining a rich and sufficient dataset fromsimulations for training a reliable DL model is com-putationally expensive and requires careful case de-sign. To address this fundamental challenge, physics-informed DL explicitly embed the physical laws (e.g.,the governing partial differential equations (PDEs),initial/boundary conditions, etc.) into the DNN, con-straining the network’s trainable parameters withina feasible solution space. The objective of exploitingphysical laws in DNN is assumed to (1) reduce thelarge dependency of the model on available datasetin terms of both quality and quantity, and (2) im-prove the robustness and interpretability of the DLmodel. In this regard, DNN essentially has the capac-ity of approximating the latent solutions for PDEs[8, 9], with distinct benefits summarized as follows:(1) the superior interpolation ability of DNN, (2) theapproximated solution has a close form with its in-finite derivative continuous, and (3) state-of-the-arthardware advances make the numerical implementa-tion and parallelization extremely convenient.

More recently, Raissi et al. [5, 6] introduced ageneral framework of PINN and demonstrated its ca-pacity in modeling complex physical systems such assolving/identifying PDEs. A huge difference fromsome of the previous studies is that, in addition to the

Preprint submitted to Theoretical and Applied Mechanics Letters April 23, 2020

arX

iv:2

002.

1055

8v2

[ph

ysic

s.fl

u-dy

n] 2

2 A

pr 2

020

𝑡

𝑦 ⋮ ⋮

⋮

𝝈

𝜓

⋮𝑥

𝜕

𝜕𝑡

1

𝜕

𝜕𝑦

𝜕

𝜕𝑥

𝐯 (𝑥, 𝑦, 𝑡)

𝝈 (𝑥, 𝑦, 𝑡)

𝝈

𝛁 ⋅ 𝝈

𝐯

𝜕𝐯

𝜕𝑡

𝛁 ⋅ 𝐯

Automatic Differentiation 1. Governing equations

𝜕𝐯

𝜕𝑡+ 𝐯 ⋅ 𝛁 𝐯 =

1

𝜌𝛁 ⋅ 𝝈 + 𝒈

𝝈 = −𝑝𝐈 + 𝜇(𝛁𝐯 + 𝛁𝐯T)

DNN (unknown parameters {𝐰, 𝐛})Physical Laws

Data Loss Function: 𝐽𝑑(𝐰, 𝐛) Physics Loss Function: 𝐽𝑝(𝐰, 𝐛)

{𝐰∗, 𝐛∗} = argmin 𝐽𝑑 𝐰, 𝐛 + 𝛼 ⋅ 𝐽𝑝 𝐰, 𝐛{𝐰, 𝐛}

𝛁 ⋅ 𝐯 = 𝟎

𝛁𝐯

𝐯 ⋅ 𝛁 𝐯

𝜕

𝜕𝑦

−𝜕

𝜕𝑥

2. Boundary conditions

𝐯 = 𝒍 at Dirichlet boundary

3. Initial condition

𝝈 ⋅ 𝐧 = 𝒉 at Neumann boundary

𝐯 𝑥, 𝑦, 𝟎 = 𝐯0

𝑝(𝑥, 𝑦, 𝑡)𝑝

Figure 1: Architecture of the physics-informed neural network for fluid dynamics. Note that α is a user-defined weightingcoefficient. w and b are weights and biases for the DNN. The constraint of initial and boundary conditions can be converted asresiduals adding to the loss function based on Lagrangian multipliers. The data loss Jd is present only when data is available.

physical laws, the PINN can also exploit the availablemeasurement data, making it possible to discover thesystems whose physics are not fully understood. Inparticular, seminal contributions of using PINN tomodel fluid flows have been made recently. For ex-ample, Kissas et al. [10] employed PINN to predictthe arterial blood pressure based on the MRI dataand the conservation laws. Sun et al. [11] proposed aPINN approach for surrogate modeling of fluid flowswithout any simulation data. Zhu et al. [12] pro-posed a physics-constrained convolutional encoder-decoder network and a generative model for modelingof stochastic fluid flows.

In this paper, we formulate a mixed-variable PINNscheme for simulation of viscous incompressible lam-inar flows without any measurement data. The re-maining this paper is organized as follows. In Section2, we will introduce the methodology of the mixed-variable PINN and the mathematical formulation forfluid dynamics. Subsequently in Section 3, the steadyand transient laminar flows passing a circular cylin-der will be modeled using the proposed PINN schemewithout any simulation data. A comparison study ismade to demonstrate the improved solution accuracyand network trainability by the proposed scheme. Sec-tion 4 summarizes the conclusions.

2. Methodology

Let us consider the incompressible Newtonian flowgoverned by the following NavierStokes equations:

∇ · v = 0 (1)

∂v

∂t+ (v · ∇)v = −1

ρ∇p+

µ

ρ∇2v + g (2)

where ∇ is the Nabla operator, v = (u, v) is thevelocity vector, p is the pressure, µ is the viscosityof the fluid, ρ is the density of fluid and g is thegravitational acceleration. When leveraging PINN tosolve the aforementioned PDEs, minimizing a com-plex residual loss resulted from Eq. (2) is intractabledue to its complex form with multiple latent vari-ables (e.g., v and p) and high-order derivatives (e.g.,∇2). In order to design an easily trainable PINN, weconvert the NavierStokes equation in Eq. (2) to thefollowing continuum and constitutive formulations:

∂v

∂t+ (v · ∇)v =

1

ρ∇ · σ + g (3)

σ = −pI + µ(∇v +∇vT) (4)

where σ is the Cauchy stress tensor and p = −tr(σ)/2.The benefits of using the continuum-mechanics-basedformulation are two-fold: (1) reducing the order ofderivatives when a mixed-variable scheme in PINN isused, and (2) improved trainability of DNN as foundin the comparison of numerical results.

The proposed mixed-variable scheme is used inthis paper to solve the aforementioned PDEs (see Eqs.(1), (3) and (4)) that govern the laminar flow dynam-ics. The salient feature of PINN is that the physicalfields are approximated globally by a DNN. In par-ticular, the DNN maps the spatiotemporal variables{t,x}T to the mix-variable solution {ψ, p, σ}, wherethe stream function ψ is employed rather than the

2

velocity v to ensure the divergence free condition ofthe flow. In this way, the continuity equation will besatisfied automatically. For a two-dimensional prob-lem, the velocity components can be computed by[u, v, 0] = ∇× [0, 0, ψ]. Note that v = [u, v] is takenas the latent variable. The automatic differentiationis used to obtain the partial derivatives of the DNNoutput regarding the time and space (e.g., t, x andy). The loss function is composed of the data loss Jd(if measurements are available) and the physics lossJp. The physics loss Jp is the summation of the gov-erning equation loss Jg and the initial and boundarycondition loss Ji/bc, given by

Jg =1

Ng

Ng∑i=1

||rg(xi, ti)||2 (5)

Ji/bc =1

NI

NI∑i=1

||rI(xi, 0)||2

+1

Nnb

Nnb∑i=1

||rnb(xi, ti)||2

+1

Ndb

Ndb∑i=1

||rdb(xi, ti)||2

(6)

where r(·) denotes the residual, || · || denotes the `2norm andN(·) denotes the number of collocation points(subscripts g for governing equation, I for initial con-dition, nb for Neumann boundary, and db for Dirichletboundary). The total physics loss Jp is defined as

Jp = Jg + βJi/bc (7)

where β > 0 is a user-defined weighting coefficientfor initial and boundary condition loss. Noteworthy,having the measurement data makes the fluid flowmodeling data-driven, which is however not a prereq-uisite. The architecture of the proposed PINN forfluid dynamics simulation is presented in Fig. 1. Inthis paper, no measurement data from simulations orphysical experiments is used for training the PINN.

3. Results

In this section, we employ the proposed PINNto model the steady and transient flows passing acircular cylinder. A parabolic velocity profile is ap-plied on the inlet while the zero pressure conditionis applied on the outlet, as shown in Fig. 2. Non-slip conditions are enforced on the wall and cylin-der boundaries. The gravity is ignored in both two

L=1.1 m

H=0.41 m

0.1 m

0.15 m

0.15 mOutlet

Wall

Wall

Inlet

x

y

Figure 2: Diagram of the computation model.

cases. The proposed PINN is implemented on theTensorFlow [13] and the source codes can be foundin https://github.com/Raocp/PINN-laminar-flow.

3.1. Steady flow

For the steady case, the dynamic viscosity anddensity of the fluid is 2× 10−2kg/(m · s) and 1kg/m3

respectively. The normal velocity profile is defined as

u(0, y) = 4Umax(H− y)y/H2 (8)

with Umax equal to 1.0 m/s which results in a smallReynolds number so that the flow is dominated bylaminar flow. A total number of Ng = 50000 col-location points, which includes Ndb = 1200 Dirichletboundary (cylinder, wall, inlet) points and Nnb = 200Neumann boundary (outlet) points, are generated us-ing Latin hypercube sampling (LHS) for the trainingthe network. It should be noted that the collocationpoints are refined near the cylinder to better capturethe details of the flow. A grid search strategy is usedto find an optimal combination of depth and widthfor the network. The relative `2 error defined by

ε =

√∑Mi=1 ||f ipred − f iref||2√∑M

i=1 ||f iref||2(9)

is used as the metric for comparison, where f is thephysical quantity of interest, and M is the total num-ber of reference points. The Adam [14] and Limited-memory BFGS (L-BFGS) optimizer [15] is employedto train the DNN due to their good convergence speeddemonstrated in the tests. We also implement thetraditional scheme for fluid dynamics employed in[5, 6] where the stream function and pressure {ψ, p}act as the output variables. From the relative `2errors of the velocity field (see Table 1), it can beseen that the network of 8× 40 achieves the best re-sult among all the configurations. The mixed-variablePINN improves the accuracy of numerical results overthe traditional PINN.

3

(a) (b) (c)

Figure 3: Velocity and pressure fields of the steady flow passing a circular cylinder: (a) Reference solution from ANSYS Fluent;(b) Mixed-variable scheme solution with 8×40 network; (c) Traditional scheme solution with 8×40 network. The hyperparametersand collocation points for training these two PINNs are kept same.

Table 1: Relative `2 errors (unit: 10−2) of the velocity field fordifferent DNN configurations with β = 2 (left: the traditionalscheme; right: the proposed mixed-variable scheme).

WidthDepth

6 7 8

40 76.1/4.2 69.1/4.7 73.8/1.850 74.0/2.6 81.1/2.4 76.9/2.360 74.5/2.3 75.2/2.4 75.4/1.9

0 232

2 (rad)

0

1

2

3

4

Pres

sure

(Pa)

PINN (mixed)PINN (traditional)Reference𝜃

x

y

Figure 4: Distribution of pressure on cylinder. Network of8×40 and β = 2 are used.

The predicted velocity and pressure fields by thePINNs with mixed-variable and traditional schemeare shown in Fig. 3(b) and (c) respectively. Thereference solution is obtained from the ANSYS Flu-ent 18.1 package (finite volume-based) [16] (see Fig.3(a)). It can be observed that the PINN with the tra-ditional scheme fails to model the flow. In particular,the traditional scheme fails to enforce the non-slip

10 3

10 1

101

103

105

Loss

Mixed ( = 1)Mixed ( = 2)Mixed ( = 5)Mixed ( = 10)Traditional ( = 1)Traditional ( = 2)Traditional ( = 5)Traditional ( = 10)

0 1 2 3 4 5 6 7 8Iteration

×104

Figure 5: Comparison of convergence curves with respect tocoefficient β. Network of 8 × 40 is used in all the cases. 10000iterations trained with Adam optimizer followed by L-BFGSoptimizer.

y (m)

0.0 0.1 0.2 0.3 0.4t (s

)0.0

0.10.2

0.30.4

0.5

u(m

/s)

0.00.20.40.60.81.0

Figure 6: Transient normal velocity profile.

condition on the lower and upper boundaries. How-ever, the steady velocity and pressure fields are wellreproduced by the PINN with mixed-variable scheme.

4

(a) t=0.3 s (b) t=0.4 s (c) t=0.5 s

Figure 7: Snapshots of the PINN-predicted transient flow fields passing a circular cylinder

0.0 0.1 0.2 0.3 0.4 0.5Time (s)

0.5

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.1 0.2 0.3 0.4 0.5Time (s)

0.5

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.1 0.2 0.3 0.4 0.5Time (s)

0.50.00.51.01.52.02.53.03.5

Pres

sure

(Pa)

(a) P1 (b) P2 (c) P3

PINN Reference

Figure 8: Pressure time histories on P1, P2 and P3 probes.

It is worth mentioning that the pressure distributionon the cylinder surface is typically of interest for com-puting the resultant drag and lift forces. Therefore,we compare the pressure distributions obtained bytwo types of PINN and ANSYS Fluent as shown inFig. 4. The overall agreement between the mixed-variable PINN and ANSYS Fluent is very good.

We also compare the performance of these twoschemes under various β which controls the weight ofthe boundary condition loss. As shown in Fig. 5, theconvergence of the traditional scheme is significantlyaffected by β, though the final loss can be reducedby increasing the value of β up to 10. However, themixed-variable scheme yields consistent results forvarious β. The improvement by the mixed-variablescheme is thanks to the reduced order of derivatives(see Section 2) required to construct the loss func-tion, in comparison with the traditional scheme [5, 6],which makes the optimization problem easier.

3.2. Transient flow

The transient flow with the same computation do-main depicted in Fig. 2 is considered in this case. The

dynamic viscosity of the flow is µ = 5×10−3kg/(m · s)while the density is ρ = 1kg/m3. The time dura-tion for the modeling is 0.5 s. Three virtual pressureprobes P1(0.15, 0.2) m, P2(0.2, 0.25) m and P3(0.25,0.2) m are installed on the surface of the cylinder.The flow is initially still while a time-varying parabolicinlet velocity profile is applied subsequently, which isdefined as

u(0, y) = 4

[sin

(2πt

T+

3π

2

)+ 1

]Umax(H− y)y/H2

(10)where Umax equals to 0.5 m/s and the period T is 1.0s. The remaining boundary conditions are the sameas those in the previous example. The inflow velocityas a function of t and y is visualized in Fig. 6. Thewidth and depth of the network are selected to be 50and 7 respectively while the coefficient β is set to be2. A total number of Ng = 120000 collocation points,which include Ndb = 9600 points on cylinder, walland inlet boundaries, Nnb = 3200 points on outlet,and NI = 3500 points at initial time, are used totrain the network.

5

Three snapshots of the predicted flow fields arepresented in Fig. 7 which shows the evolution of theflow as the inlet velocity increases over time. Thereference flows obtained by ANSYS Fluent are notshown here since the PINN-predicted result matchesvery well with them. The pressure time historieson three probes obtained from the proposed PINNare compared with those from ANSYS Fluent, as de-picted in Fig. 8. It can be seen that the proposedPINN approach can well predict the pressure timehistories in a transient flow.

4. Conclusions

We propose a mixed-variable PINN scheme formodeling fluid flows, with particular applications toincompressible laminar flows. The salient featuresof the proposed scheme include (1) employing thegeneral continuum equations together with the mate-rial constitutive law rather than the derived Navier-Stokes equations, and (2) using stream function toensure the divergence free condition of the flow in amixed-variable setting. The comparison study indi-cates the benefits (high accuracy and good trainabil-ity) of the proposed mixed-variable scheme. In boththe steady and transient flow cases, the result pro-duced by the PINN shows a good agreement with thereference numerical solutions.

It is notable that the applications in this paper arelimited to the laminar flows at low Reynolds num-bers, although the approach is in theory applicableto turbulent flows at large Reynolds numbers. How-ever, it requires discretizing the computation domainwith much finer collocation points which will lead tocomputer memory issues and drastically increase thecomputational cost. Our future work aims to addressthis challenge by developing a “divide-and-conquer”training scheme in the context of transfer learning,that is to divide the time domain into multiple stepsand re-train the network partially while fixing theweights and the biases from the previous step [17].

References

[1] S. Lee, N. Baker, Basic research needs for scientific ma-chine learning: Core technologies for artificial intelligence,Tech. rep., USDOE Office of Science (SC)(United States)(2018).

[2] R. Zhang, Z. Chen, S. Chen, J. Zheng, O. Bykztrk,H. Sun, Deep long short-term memory networks for non-linear structural seismic response prediction, Computers& Structures 220 (2019) 55 – 68.

[3] C. Yang, X. Yang, X. Xiao, Data-driven projectionmethod in fluid simulation, Computer Animation and Vir-tual Worlds 27 (3-4) (2016) 415–424.

[4] J. Tompson, K. Schlachter, P. Sprechmann, K. Perlin, Ac-celerating eulerian fluid simulation with convolutional net-works, in: Proceedings of the 34th International Confer-ence on Machine Learning-Volume 70, JMLR. org, 2017,pp. 3424–3433.

[5] M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework forsolving forward and inverse problems involving nonlinearpartial differential equations, Journal of ComputationalPhysics 378 (2019) 686–707.

[6] M. Raissi, Z. Wang, M. S. Triantafyllou, G. E. Karni-adakis, Deep learning of vortex-induced vibrations, Jour-nal of Fluid Mechanics 861 (2019) 119–137.

[7] N. Geneva, N. Zabaras, Quantifying model form un-certainty in reynolds-averaged turbulence models withbayesian deep neural networks, Journal of ComputationalPhysics 383 (2019) 125–147.

[8] I. E. Lagaris, A. Likas, D. I. Fotiadis, Artificial neuralnetworks for solving ordinary and partial differential equa-tions, IEEE transactions on neural networks 9 (5) (1998)987–1000.

[9] I. E. Lagaris, A. C. Likas, D. G. Papageorgiou, Neural-network methods for boundary value problems with irreg-ular boundaries, IEEE Transactions on Neural Networks11 (5) (2000) 1041–1049.

[10] G. Kissas, Y. Yang, E. Hwuang, W. R. Witschey, J. A.Detre, P. Perdikaris, Machine learning in cardiovascularflows modeling: Predicting arterial blood pressure fromnon-invasive 4d flow mri data using physics-informed neu-ral networks, Computer Methods in Applied Mechanicsand Engineering 358 (2020) 112623.

[11] L. Sun, H. Gao, S. Pan, J.-X. Wang, Surrogate modelingfor fluid flows based on physics-constrained deep learningwithout simulation data, Computer Methods in AppliedMechanics and Engineering (2019) 112732.

[12] Y. Zhu, N. Zabaras, P.-S. Koutsourelakis, P. Perdikaris,Physics-constrained deep learning for high-dimensionalsurrogate modeling and uncertainty quantification with-out labeled data, Journal of Computational Physics 394(2019) 56–81.

[13] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis,J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard,et al., Tensorflow: A system for large-scale machine learn-ing, in: 12th USENIX Symposium on Operating SystemsDesign and Implementation, 2016, pp. 265–283.

[14] D. P. Kingma, J. Ba, Adam: A method for stochasticoptimization, arXiv preprint arXiv:1412.6980.

[15] R. Byrd, P. Lu, J. Nocedal, C. Zhu, A limited memory al-gorithm for bound constrained optimization, SIAM Jour-nal on Scientific Computing 16 (5) (1995) 1190–1208.

[16] ANSYS Fluent 18.1, ANSYS FLUENT Theory Guide,ANSYS Inc., PA, USA, 2018.

[17] S. Goswami, C. Anitescu, S. Chakraborty, T. Rabczuk,Transfer learning enhanced physics informed neural net-work for phase-field modeling of fracture, Theoretical andApplied Fracture Mechanics 106 (2020) 102447.

6