Some Mathematical and Numerical Issues in Multiscale Modeling

Weinan E

Princeton University

Supported by ONR, DOE and NSF.

Collaborators: Carlos Garcia(UCSB), Xiantao Li (PSU), JianfengLu (Princeton), Pingbing Ming (CAS)

Multiscale modeling has had a lot of successes

Quantum mechanics-Molecular mechanics (QM-MM)methods (1975, Warshel and Levitt)

Car-Parrinello molecular dynamics (1985, avoid empiricalpotentials, compute force fields directly from electronicstructure information)

Kinetic schemes in gas dynamics

Quasicontinuum method (1996, Tadmor, Ortiz and Phillips)

......

There are still many fundamental issues

Focus on:Difficulties with coupled atomistic (electronic structure)-continuum methods,from the viewpoint of numerical analysis:

Boundary conditions

Consistency

Stability

Illustrated using simple examples.

Major difficulty: Microsopic models, such as molecular dynamicsand electronic structure models, are very poorly understood.

Boundary conditions

Consistency

Stability

Linear and sublinear scaling algorithms

Molecular dynamics

mj

d2xj

dt2= Fj = −

∂V

∂xj

Electronic structure models

Wave functions for the electrons (Kohn-Sham densityfunctional theory, tight-binding models, etc)

Electron density (Orbital-free density functional theory)

Boundary conditions for MD and QM models

Super-cell: making things periodic.

Effect of boundary condition more pronounced due to the fact thatwe can only simulate rather small systems.

What would it be like if all finite element calculations are restrictedto periodic boundary conditions?

Boundary conditions: Phonon reflection from MD

simulation of solids

Dirichlet type of boundary condition is used.

(Holian et al., Phys. Rev. B, 1998).

Boundary condition for MD simulation of solids

Main objective: Use a small system to mimic the behavior of muchlarger system.

Obvious analogy with ABC (absorbing boundary condition for waveequation)

Key differences:

Small k expansion not enough, phonons exist at all k.

Finite temperature: Also need to take phonons from theenvironment.

Problem formulationEliminate heat bath variables

The right perspective: Mori-Zwanzig formalismA simple linear example

dp

dt= A11p + A12q,

dq

dt= A21p + A22q.

Objective: Obtain a closed model for p.

q(t) = eA22tq(0) +

∫ t

0eA22(t−τ)A21p(τ)dτ.

dp

dt= A11p + A12

∫ t

0eA22(t−τ)A21p(τ)dτ + A12e

A22tq(0)

= A11p +

∫ t

0K (t − τ)p(τ)dτ + f (t),

K (t) = A12eA22tA21 is the memory kernel

f (t) = A12eA22tq(0) is the noise term, if we think of q(0) as a

random variable.

General Mori-Zwanzig formalism1. Projection operator (conditional expectation),

Pg = E (g |retained variables),Q = I − P.

2. For any function of the retained variables: ϕ,

d

dtϕ(t) = etLLϕ(0) = etLPLϕ(0) + etLQLϕ(0),

3. Dyson’s formula,

etL = etQL +

∫ t

0e(t−s)LPLesQLds.

4. Generalized Langevin equation:

ddtϕ(t) = etLPLϕ(0) +

∫ t

0e(t−s)LK (s)ds + R(t).

R(t) = etQLQLϕ(0), K (t) = PLR(t).

Recent literature: Chorin et al. on “Optimal prediction”.

Universal strategy for eliminating degrees of freedom.

By itself, it is almost useless. The key is to makeapproximations.

Mori-Zwanzig formalism for MD (Xiantao Li and E)

Partition of the system: u = (uI ,uj ), v = (vI , vj )The thermodynamic force: etLPLvI (0) = −∂W

∂uI.

The effective free energy:

W (uI ,T ) = −kBT lnZ , Z =

∫

e−

V (uI ,uj )

kBT duj ,

The memory term: −∫ t

0 Θ(τ)uI (t − τ)dτ.The generalized Langevin equation:

muI = −∇uIW −

∫ t

0Θ(τ)uI (t − τ)dτ + R(t) + fex(t).

Example:

uj = φ′(

uj+1 − uj

)

− φ′(

uj − uj−1

)

, j > 0uj = uj+1 − 2uj + uj−1, j ≤ 0.

Example:

u0 = φ′(u1−u0)−

∫ t

0θ(τ)u0(t−τ)dτ+R .

θ(t) =J2(2t)

t.

Local memory kernels

(E and Huang, 2001; Li and E, 2006–2007)

the memory kernel is independent of the temperature

at zero temperature, similar to absorbing BC for waveequations

Basic principles:

1. Efficiency: local kernels

2. Stability: positive-definite kernels

3. Consistency: fluctuation-dissipation theorem

Variational approach at T = 0Given the stencil (cost), find the best approximate kernels.Note the lack of a small parameter.

1. Express Θ(t) in the form of,

Θ(t) =

∫ +∞

−∞Γ(s)Γ(t + s)Tds.

Γ(t) is local:

Γij(t) = 0, if |ri − rj | > rc , or |t| > tc .

2. Objective functions

minΓ(t)

∫

e(

ω; Γ)

W (ω)dω.

3. Choose the functional: e.g. Total energy of reflected phonons.

Sample the random noise

The random noise R(t) is a stationary Gaussian process. Thefluctuation-dissipation theorem is satisfied:

⟨

R(t)R(0)T⟩

= kBTΘ(t).

Let W (t) be white noise with variance kBTI , then,

R(t) =∑

k

∫

Γ(s)W (t − s)ds.

Case studied 1: fracture simulation

Fixed boundary condition. Variational boundary condition.

Case studied 2: finite temperature in 3D BCC Iron

System temperature. Velocity autocorrelation.

Can be used as a “first-principle-based” thermostat.

Case studied 3: finite temperature crack simulation

At zero temperature

Case studied 3: finite temperature crack simulation

At finite temperature 500K

Boundary conditions

Consistency

Stability

Linear and sublinear scaling algorithms

Consistency between the MD and the continuum model

Outside region (eliminated region) is not just a heat bath, but apiece of material modeled by continuum equations (with atemperature field).

Work in progress by Xiantao Li and E

Will look at a special case:Quasicontinuum method (Tadmor et al. 1996)

Temperature = 0

No dynamics

Quasicontinuum method

An adaptive mesh and model refinement procedure Based on linear finite elements Representative atoms define the triangulation Near defects, the mesh becomes fully atomistic Local (continuum) and nonlocal (atomistic) regions

Consistency

Consistency in the bulk: For simple systems, the two modelsshould produce consistent results.

Consistency at the local-nonlocal interface.

Consistency between atomistic and continuum models

Nonlinear elasticity theory:

E (u) =

∫

ΩW (∇u)dx

W (·) = stored energy density.In linear elasticity, W = a quadratic function of ∇u.

E (y1, · · · , yN) =∑

i ,j

V2(yi , yj) +∑

i ,j ,k

V3(yi , yj , yk) + · · ·

Question: Can we relate W to the atomistic model?

The Cauchy-Born rule

Given A, a 3 × 3 matrix, W (A) =?

Deform the crystal uniformly: yj = xj + Axj = (I + A)xj

W (A) = energy density of deformed unit cell, computed accordingto the given atomistic or electronic structure model.

Validity of Cauchy-Born rule: ConsistencyOne dimension model: xk = kε.Assume: yk = xk + u(xk) and u is a smooth function.

V =1

2

∑

i 6=k

V0(yi − yk) (1)

≈1

2

∑

i

∑

k 6=i

V0(1 +du

dx(xi ))kε

=∑

i

W

(

du

dx(xi)

)

ε ≈

∫

W

(

du

dx(x)

)

dx ,

where

W (A) =1

2ε

∑

k

V0((1 + A)kε)

X. Blanc, C. Le Bris and P. L. Lions (2002) considered generalcase, including some QM models.

Validity of Cauchy-Born rule: Counterexample

Example: Lennard-Jones potential, next nearest neighborinteraction

Triangular lattice, Cauchy-Born rule is valid

Square lattice, Cauchy-Born gives negative shear modulus(unstable), can’t speak of elasticity theory.

Validity of Cauchy-Born rule: Stability

Continuum level (Born criteria) – Elastic stiffness tensor ispositive definite

Atomic level (Lindemann criteria) – Phonon spectra

(dispersion relation for the lattice waves) remain “positivedefinite”

Electronic level – Disperson relation for the charge-density

waves and spin waves

Phonon stability condition

Acoustic branch: dynamics of the Bravais lattice

Optical branch: relative motion of the internal degrees of freedom

1st Brillouin zone: Voronoi cell of the origin of the dual lattice

Consistency in the bulk

Under these conditions, the solutions to the Cauchy-Born

continuum model and the atomistic model are close.

E and Ming (2007), molecular mechanics models.

E and Lu (2008), several classes of quantum mechanicsmodels.

These conditions are sharp!!

Violation of the stability conditions signals onset of plasticdeformation or structural (or electronic) phase transformation.Related work of Ju Li, S. Yip et al. (Λ-criterion),R. Elliott et al.

Consistency at the loca-nonlocal interfaceThe issue of “ghost force” (e.g. in quasicontinuum methods)

−10 −8 −6 −4 −2 0 2 4 6 8 10−0.2

−0.15

−0.1

−0.05

0

0.05

0.1

0.15Ghost forces with 5th nearest neighbor interaction

Original QCQuasi−nonlocalLinear construction

Left side: using continuum model based on the Cauchy-Bornrule (effectively a nearest neighbor model).

Right side: Using full atomistic model, next nearest neighborinteraction.

Explicit solution for quadratic potential (E, Ming and Yang, 08)

0 5 10 15 20 25 30 35−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

N

D+(y

qc−

x)

1. The deformation gradient has O(1) error at the interface.

2. The influence of the ghost force decays exponential fast awayfrom the interface.

3. Away from an interfacial region of width O(ε| log(ε)|), theerror in the deformation gradient is of O(ε) (see also recentwork of Dobson and Luskin).

Removing the ghost force

Ghost force may induce numerical artifacts (e.g. plasticdeformation) at the interface.

Force-based approach (Tadmor et al., Miller, Dobson andLuskin)

Quasi-nonlocal atoms (Jacobson et al.)

Geometrically consistent scheme (E, Lu and Yang)

Classical numerical analysis viewpoint:

Truncation error = O(ε) in a weak sense

Stability conditions (similar to the ones discussed above)

Uniform O(ε) accuracy for smooth solutions.

Related work in the math literature:

Dobson and Luskin

Ortner and Suli

Blanc, Legoll and Le Bris

Improving atomistic/continuum coupling for solids:

MAAD: Abraham, Bernstein, Broughton and Kaxiras

Bridge domain: Xiao and Belytschko

Bridge scales: W. K. Liu et al.

AtC approach: Gunzburger et al., Lehoucq et al.

M. Robbins

......

Ghost force for the coupled OF-DFT/EAM method

(Choly, Lu, E, Kaxiras)

-0.03

-0.02

-0.01

0

0.01

-0.03

-0.02

-0.01

0

0.01

−3

∆ρ (A−3

) ∆ρ (A−3

)

1 nm

0.14A−3

A

(d)

0.22

(c)

(b)(a)

Loss of fluctuations

Example: Coupled KMC-continuum models of epitaxial crystalgrowth (Schulze, Smereka and E):

Around the step-edges, use KMC, since fluctuations areimportant

On the terraces, use continuum (e.g. diffusion) models

Mean position and variance of step edge

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0 200000 400000 600000 800000 1e+06

σe

D/F

+++++++++

+

33333333

3

3

+++++++++

+

+++++++++

+

Figure 6: The time and space averaged surface adatom density for the KMC simulations(diamonds) and the hybrid scheme (crosses) using cell-widths M = 20, 25, 40.

0

0.005

0.01

0.015

0.02

0 200000 400000 600000 800000 1e+06

σρ

D/F

++++++++++

33

33333

33

3

++++++++++ ++++++

+++

+

Figure 7: The standard deviation (in time) of the surface averaged adatom density as afunction of the ratio D/F . The solid curve (diamonds) are from the KMC simulations andthe remaining curves are for the hybrid scheme with cell-widths M = 20, 25, 40 sites percell.

21

Other examples: See work of Garcia, Bell and Donev, et al.

Boundary conditions

Consistency

Stability

Linear and sublinear scaling algorithms

Stability of coupled continuum/MD methods

Work of Weiqing Ren (NYU)

Example of fluids

General strategy: Domain decomposition (with overlap)

Coupling:

III

z=−az=−b

z=az=b

z=1

z=−1u=0

u=0

I

II

In continuum region (I, III): ρut − µuzz = 0

In particle region (II): md2xj

dt2 = f j

Four coupling schemes:

1. velocity(MD)-velocity(C),

2. velocity(MD)-flux(C),

3. flux(MD)-velocity(C),

4. flux(MD)-flux(C)

Particular features as a domain decomposition method

The MD (molecular dynamics) domain is very small.

Statistical error cannot be avoided

Numerical solutions for equilibrium states

0 0.5 1 1.5 2

x 104

0

0.02

0.04

t / T

||u|| 2

0 0.5 1 1.5 2

x 104

0

0.02

0.04

t / T

||u|| 2

0 0.5 1 1.5 2

x 104

0

0.02

0.04

t / T

||u|| 2

0 0.5 1 1.5 2

x 104

0

0.2

0.4

t / T

||u|| 2

Upper panel: velocity-velocity Upper panel: velocity-fluxLower panel: flux-velocity Lower panel: flux-flux

0

1

2

u

−100 −50 0 50 1000

1

2

z

u

0 2000 4000 6000 80000

0.1

0.2

e u

Steady-state calculation: Tc = ∞

Amplification factors g for the four schemes:

velocity-velocity, flux-velocity

un(z) =

n∑

i=1

gn−iξi1 − z

1 − a, 〈‖un‖2〉 ≤

(

1

3(1 − g2)

)1/2

σv

ξi : Statistical errors in velocity BC; σv =⟨

ξ2i⟩

g = a(1−b)b(1−a) for velocity-velocity; g = a

a−1 for flux-velocity

velocity-flux, flux-flux

un(z) =n

∑

i=1

gn−iξ(z − 1)

g = b−1b

for velocity-flux; (g > 1 → Diverge)

g = 1 for flux-flux → 〈‖un‖2〉 ≤ 3−1/2(1 − a)n1/2στ

Stability: Finite Tc

0 200 400 600 800 10000

0.5

1

Tc

k vv ,

k fv

10 30 50 70 900.8

0.9

1

1.1

Tc

k vf ,

k ff

Figure: The amplification factor g versus Tc/∆t for the four schemes:VV (squares), FV (diamonds), VF (triangles) and FF (circles).

Remarks:

There are many different variants of atomistic/continuumcoupling schemes.

Errors and artifacts are difficult to understand.

What I have described are examples of efforts to try to putthings on a solid foundation.

Boundary conditions

Consistency

Stability

Linear and sublinear scaling algorithms

Linear scaling algorithms

Cost ∼ number of degrees of freedom

Examples:

Multi-grid method

Fast multipole method

Linear scaling algorithms in electronic structure analysis

......

Sublinear scaling algorithms

Cost ≪ number of degrees of freedom (in the microscopic) model

Quasicontinuum method

AtC methods

Most multiscale methods are sublinear scaling algorithms.

Sub-linear scaling algorithm for electronic structure analysis

Garcıa-Cervera, Lu and E (2007)

Electron density distribution of Al inthe presence of vacancies

0 0.5 1 1.5 2 2.5

0.18

0.2

0.22

0.24

0.26

0.28

0.3

0.32

0.34

0.36

0.38

xρ

A slice of the electron density(modulated problem).

An analogy: Wave propagation and geometric optics

−1 −0.5 0 0.5 1 1.5 2

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

x

u

∂2t u = C (x)2∆u;

u(x , 0) = A0(x)e iφ0(x)

ǫ .

Solving the wave equation directly: O(ε−1) operations (linearscaling algorithm).

Geometric optics approximation:

Ansatz: u(x , t) = A(x , t) exp

(

iϕ(x , t)

ǫ

)

+ · · · · · ·

∂tϕ+ C (x)|∇ϕ| = 0;

∂tA + C (x)∇ϕ · ∇A

|∇ϕ|+

C (x)2∆ϕ− ∂2t ϕ

2C (x)|∇ϕ|A = 0.

Solving these equation require O(1) (independent of ε) operations.

In the general case, solve these limit equations away from caustics,and solve the original problem near caustics.This requires o(ε−1) operations, hence sub-linear scalingalgorithm.

Combine asymptotic analysis and numerical methods

Asymptotics for density functional theory

ψk = ψk(y), k = 1, · · · ,N = A set of N orthonormal orbitals:

Iε(ψk) = ε2N

∑

k=1

∫

R3

|∇ψk(y)|2 dy +

∫

R3

ǫxc(ρ)ρ(y) dy

+ε

2

∫∫

R3×R3

(ρ− m)(y)(ρ− m)(y′)

|y − y′|dy dy′.

ε = atomic length scale, e.g. the lattice constant.

ρ(y) =∑

k |ψk |2(y) is the electron density.

Input to the model: The atoms. m(y) =∑

yi∈εL∩Ω mai (y − yi ) =

(pseudo)-ionic potential (describing the atoms in the system).

yj = positions of the nuclei (ions).

maj = describe the types of atoms

Ansatz (first variable is Eulerian, second is Lagrangian):

ψα(y ,x

ε) =

1

ε3/2ψα,0(y ,

x

ε) +

1

ε1/2ψα,1(y ,

x

ε) + ε1/2ψα,2(y ,

x

ε) + · · ·

ρ(y ,x

ε) =

1

ε3ρ0(y ,

x

ε) +

1

ε2ρ1(y ,

x

ε) +

1

ε1ρ2(y ,

x

ε) + · · ·

φ(y ,x

ε) = φ0(y ,

x

ε) + εφ1(y ,

x

ε) + ε2φ2(y ,

x

ε) + · · · .

ψα(y , z) decays for large z

ρ(y , z) and φ(y , z) are periodic in z .

Electron density distribution of Al

Leading order: Electronic structure of a uniformly

deformed infinite latticeψα, where α numbers the valence electrons in a unit cell.∇1 = ∇y ,∇2 = ∇z .

− ∆x2ψα,0(y , z) + Vxc,0(ρ0)ψα,0(y , z) − φ0(y , z)ψα,0(y , z)

+∑

α′,zj∈L

λαα′j ,0(y)ψα′,0(y , z − zj) = 0;

− ∆x2φ0(y , z) = 4π(m0 − ρ0)(y , z);

∫

R3

ψα,0(y , z)ψα′,0(y , z − zj) dz = δαα′δ0j/ det(I + ∇u(x)).

∆x2 = ((I + ∇u(x))−T∇2)

2, the coefficients come from thecoordinate change, y = x + u(x).

This is a system of equations in z – the fast variables. y

enters only as parameters.

It is a periodic system, but the equation is formulated over thewhole space.

The continuum limit and the Cauchy-Born rule

E (u) =

∫

ΩWCB(∇u(x)) dx

Variational formulation

WCB(A) = infψ

W (A, ψα(·;A)) = infψ

det(I + A)

|Γ|IA(ψ)

IA =∑

α

∫

R3

|(I + A)−T∇ψα(z ;A)|2 dz +

∫

Γεxc,0(ρ(z ;A))ρ(z ;A) dz

+1

2

∫∫

Γ×Γ(ρ− mCB)(z ;A)G (z − z ′;A)(ρ− mCB)(z ′;A) dz dz ′.

Γ= unit cell, G is the periodic Green’s function.

Next order equations:

− ∆x2ψα,1(y , z) + Vxc,0(ρ0)ψα,1(y , z)

+ Vxc,1(ρ0, ρ1)ψα,0(y , z) − φ0ψα,1(y , z) − φ1ψα,0(y , z)

+∑

α′,zj∈L

(

λ(y)αα′j ,0∇1ψα′,0(y , z − zj)(I + ∇u) · zj

+ λ(y)αα′j ,0ψα′,1(y , z − zj) + λ

(y)αα′j ,1ψα′,0(y , z − zj)

)

= 0;

− ∆x2φ1(y , z) = 4π(m1(y , z) − ρ1(y , z));

∫

R3

ψα,0(y , z)ψα′,1(y , z − zj)

+ ψα,0(y , z)∇1ψα′,0(y , z − zj)(I + ∇u) · zj dz = 0.

Again, this is a set of equations in the fast variable.Differentiation in y only enters through the forcing term inthe constraint equation.

How do we make use of the asymptotics results to design sublinearscaling algorithms?

See talk by Carlos Garcia-Cervera

Summary: 1. Model error and solution error

Etot = EI + EII + Eint

Example: Coupled QM-continuum methods (E and Lu)

1. The whole domain is decomposed to the union of a continuumregion (Ωc) and QM (orbital-free DFT models) region (Ωqm) .

2. In the continuum region Ωc , use Cauchy-Born rule.

Two levels of Cauchy-Born rule:

Cauchy-Born energy density

Cauchy-Born electron density

IA(ρns ,Ω) = I (ρCB,Ωs) + I (ρns ,Ωns) + Eint(ρ).

IB(ρns ,Ω) =

∫

Ωs

WCB(∇u) dx + I (ρns ,Ωns) + Eint(ρ).

Eint(ρ) = ε

∫∫

y(Ωs)×y(Ωns )

(ρCB − m)(y)(ρns − m)(y ′)

|y − y ′|dy dy ′.

Lemma: (E and Lu, 2007) Assume that the displacement of theatoms is smooth, then

For model A, the error for the force on atoms is o(1).

For model B, the error for the force on atoms is O(1).

2. Linear scaling vs. sublinear scaling algorithms

Sublinear scaling algorithms usually rely on some results fromasymptotic analysis.

Concluding remarks

Understanding the fundamental numerical issues in multiscalemethods is a rather challenging task. More efforts are neededto better understand microscopic models, such as electronicstructure models, molecular dynamics and Monte Carlomethods.

The basic purpose of multiscale modeling is to get rid of badmodels. However, it is not a step forward to replace badmodels by bad numerics.

In the early days of numerical weather forcasting, Richardsonmade a lot of progress using an unstable numerical scheme,the Richardson scheme. But further progress was onlypossible after we understood basic issues about stability andaccuracy of numerical methods.