Signal processing and machine learning methods and algorithm ...
Modeling of Fluctuations in Algorithm Refinement Methods · Modeling of Fluctuations in Algorithm...
Transcript of Modeling of Fluctuations in Algorithm Refinement Methods · Modeling of Fluctuations in Algorithm...
Modeling of Fluctuations in AlgorithmRefinement Methods
John Bell
Lawrence Berkeley National Laboratory
DSMC09 ConferenceSanta Fe, New Mexico
Sept. 13-16, 2009
Collaborators: Aleksandar Donev, Eric Vanden-Eijnden,Alejandro Garcia, Jonathan Goodman, SarahWilliams
Bell, et. al., LBNL AMAR
Outline
Hybrid algorithmsMotivationApproach
Role of fluctuationsBurgers’ equation example
Landau-Lifshitz Navier Stokes equationsNumerical methods for stochastic PDE’sHybrid algorithms for fluidsExamplesConclusions
Bell, et. al., LBNL AMAR
Multi-scale models of fluid flow
Most computations of fluid flows use a continuum representation(density, pressure, etc.) for the fluid.
Dynamics described by set of PDEs.
Well-established numerical methods (finite difference, finiteelements, etc.) for solving these PDEs.
Hydrodynamic PDEs are accurate over a broad range of lengthand time scales.
But at some scales the continuum representation breaks down andmore physics is needed
When is the continuum description of a gas not accurate?
Discreteness of collisions and fluctuations are important
Micro-scale flows, surface interactions, complex fluidsParticles / macromolecules in a flowBiological / chemical processes
Bell, et. al., LBNL AMAR
Hybrid methods
Look at fluid mechanics problems that span kinetic andhydrodynamics scales
Approaches
Molecular description –correct but expensive
Continuum CFD – cheap butdoesn’t model correct physics
Hybrid – Use different modelsfor the physics in differentparts of the domain
Molecular model onlywhere neededCheaper continuummodel in the bulk of thedomain
Bell, et. al., LBNL AMAR
Hybrid approach
Develop a hybrid algorithm for fluid mechanics that couples aparticle description to a continuum description
AMR provides a framework for such a couplingAMR for fluids except change to a particle description at thefinest level of the heirarchy
Use basic AMR design paradigm for development of ahybrid method
How to integrate a levelHow to synchronize levels
Bell, et. al., LBNL AMAR
DSMC
Discrete Simulation Monte Carlo (DSMC) is a leadingnumerical method for molecular simulations of dilute gases
Initialize system with particlesLoop over time steps
Create particles at openboundariesMove all the particlesProcess particle/boundaryinteractionsSort particles into cellsSelect and execute randomcollisionsSample statistical values
Example of flow past asphere
Bell, et. al., LBNL AMAR
Adaptive mesh and algorithm refinement
AMR approach to constructing hybrids –Garcia et al., JCP 1999
Hybrid algorithm – 2 levelAdvance continuum CNS solver
Accumulate flux FC at DSMCboundary
Advance DMSC regionInterpolation – Sampling fromChapman-Enskog distributionFluxes are given by particlescrossing boundary of DSMC region
SynchronizeAverage down – momentsReflux δF = −∆tAFC +
∑p Fp
DSMC
Buffer cells
Continuum
DSMC boundaryconditions
A B C
D E F
1 23
DSMC flux
Bell, et. al., LBNL AMAR
Hydrodynamic Fluctuations
Given particle positions and velocities, measure macroscopicvariables
These quantities naturally fluctuateParticle scheme:
Capture variance of fluctuationsPredict time-correlations at hydrodynamic scalePredict non-equilibrium fluctuations hydrodynamic scale
Bell, et. al., LBNL AMAR
Fluctuations at continuum level
How do molecular-scale fluctuations interact with thecontinuum?
Can / should we capture fluctuations at the continuum level?
Do they matter?
This has been investigated for a number of modelsDiffusion”Train” modelBurgers’ equation
Example: Burgers’ equation – Bell et al. JCP, 2007.
Bell, et. al., LBNL AMAR
AERW / Burgers’
Asymmetric Excluded Random Walk:
uj
A
B
C
ujE
D
Continuum Grid
Particle Patch
Bou
ndar
y S
ites
Bou
ndar
y S
ites
ADVANCE
SYNCHRONIZE
E
B
Probability of jump to the right sets the “Reynolds” number
Mean field given by viscous Burgers’ equationStochastic flux models fluctuation behavior
ut + c((u(1− u))x = εuxx + Sx
Stochastic flux is zero-mean Gaussian uncorrelated in spaceand time with magnitude from fluctuation dissipation theorem
Bell, et. al., LBNL AMAR
AERW / Burgers’
Simple numerical ideas work wellSecond-order Godunov for advective fluxStandard finite difference approximation of viscosityExplicitly add stochastic flux
Bell, et. al., LBNL AMAR
AERW / Burgers’
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.50.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9Mean
Hybrid AERW/SPDEHybrid AERW/DPDEAERWSPDE
−0.5 −0.4 −0.3 −0.2 −0.1 0 0.1 0.2 0.3 0.4 0.50
1
2x 10
−3 Variance
−0.4 −0.2 0 0.2 0.4−5
−4
−3
−2
−1
x 10−5 Correlation
Rarefaction
0 1 2 3 4 5 6 7 8 9 10
x 104
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5x 10
−3
Var
ianc
e of
sho
ck lo
catio
n
time
Hybrid w/ px=0.55
Hybrid w/px=0.7
Hybrid w/ px=0.8
AERW w/px=0.7
Hybrid w/no noise in SPDE region and px=0.7
Figure 9: Variance of shock location 〈δs(t)2〉 versus time t for a determin-istically steady shock (uL = 0.1, uR = 0.9). Methods used are: AERWwith p→ = 0.7 (green); SPDE/AERW hybrid with p→ = 0.55 (blue);SPDE/AERW hybrid with p→ = 0.7 (red); SPDE/AERW hybrid withp→ = 0.8 (black); DPDE/AERW hybrid with p→ = 0.7 with 8 cells ofadditional buffering (magenta).
method we have demonstrated that it is necessary to include the effect offluctuations, represented as a stochastics flux, in the mean field equations toensure that the hybrid preserved key properties of the system. As expected,not representing fluctuations in the continuum regime leads to a decay inthe variance of the solution that penetrates into the particle region. Some-what more surprising is that the failure to include fluctuations was shownto introduce spurious correlations of fluctuations in equilibrium simulationsand for rarefactions. Even more troubling is the observation that using adeterministic PDE solver coupled to the random walk model suppresses thedrift of shock location seen with the pure random walk model and with theAR hybrid using a stochastic PDE solver.
We plan to extend this basic hybrid framework to the solution of the com-pressible Navier Stokes equations in multiple dimension. For that extension
24
Shock Drift
Failure to include the effectof fluctuations at the contin-num level disrupts correla-tions in the particle region
Bell, et. al., LBNL AMAR
Landau-Lifshitz fluctuating Navier Stokes
What about fluid dynamics?Can we model fluctuations at the continuum level?Do they matter in this context?
∂U/∂t +∇ · F = ∇ · D +∇ · S where U =
ρJE
F =
ρvρvv + PI(E + P)v
D =
0τ
κ∇T + τ · v
S =
0S
Q+ v · S
,
〈Sij (r, t)Sk`(r′, t ′)〉 = 2kBηT(δK
ik δKj` + δK
i`δKjk − 2
3δKij δ
Kk`
)δ(r− r′)δ(t − t ′),
〈Qi (r, t)Qj (r′, t ′)〉 = 2kBκT 2δKij δ(r− r′)δ(t − t ′),
Bell, et. al., LBNL AMAR
LLNS Discretization
Experience with Burger’s equations suggests adding astochastic flux to a standard second-order scheme capturesfluctuations reasonably well.
This is not the case with LLNS
〈δρ2〉 〈δJ2〉 〈δE2〉Exact value 2.35× 10−8 13.34 2.84× 1010
MacCormack 2.01× 10−8 13.31 2.61× 1010
PPM 1.97× 10−8 13.27 2.58× 1010
DSMC 2.35× 10−8 13.21 2.78× 1010
Error MacCormack −14.3% −0.3% −8.4%Error PPM −16.0% −0.5% −9.4%Error DSMC 0.0% −1.0% −2.1%
Mass conservation is microscopically exact. There is no diffusion orfluctuation term in the mass conservation equation
Bell, et. al., LBNL AMAR
Numerical methods for stochastic PDE’s
Accurately capturing fluctuations in a hybrid algorithm requiresaccurate methods for PDE’s with a stochastic flux.
∂tU = LU + KW
where W is spatio-temporal white noise
We can characterize the solution of these types of equations interms of the invariant distribution, given by the covariance
S(k , t) =< U(k , t ′)U∗(k , t ′ + t) >=
∫ ∞−∞
eiωtS(k , ω)dω
whereS(k , ω) =< U(k , ω)U∗(k , ω) >
is the dynamic structure factorWe can also define the static structure factor
S(k) =
∫ ∞−∞
S(k , ω)dω
Bell, et. al., LBNL AMAR
Fluctuation dissipation relation
For∂tU = LU + KW
ifL + L∗ = −KK ∗
then the equation satisfies a fluctuation dissipation relation and
S(k) = I
The linearized LLNS equations are of the form
∂tU = −∇ · (AU − C∇U − BW )
When BB∗ = 2C, then the fluctuation dissipation relation issatisfied and the equilibrium distribution is spatially white withS(k) = 1
Bell, et. al., LBNL AMAR
Discretization design issues
Consider discretizations of
∂tU = −∇ · (AU − C∇U − BW )
of the form∂tU = −D(AU − CGU − BW )
Scheme design criteria1 Discretization of advective component DA is skew adjoint;
i.e., (DA)∗ = −DA2 Discrete divergence and gradient are skew adjoint:
D = −G∗
3 Discretization without noise should be relatively standard4 Should have “well-behaved” discrete static structure factor
S(k) ≈ 1 for small k ; i.e. S(k) = 1 + αkp + h.o.tS(k) not too large for all k . (Should S(k) ≤ 1 for all k?)
Bell, et. al., LBNL AMAR
Example: Stochastic heat equation
ut = µuxx +√
2µWx
Explicit Euler discretizaton
un+1j = un
j +µ∆t∆x2
(un
j−1 − 2unj + un
j+1
)+√
2µ∆t1/2
∆x3/2
(W n
j+ 12−W n
j− 12
)Predictor / corrector scheme
unj = un
j +µ∆t∆x2
(un
j−1 − 2unj + un
j+1
)+√
2µ∆t1/2
∆x3/2
(W n
j+ 12−W n
j− 12
)
un+1j =
12
[un
j + unj +
µ∆t∆x2
(un
j−1 − 2unj + un
j+1
)+
√2µ
∆t1/2
∆x3/2
(W n
j+ 12−W n
j− 12
)]
Bell, et. al., LBNL AMAR
Structure factor for stochastic heat equation
0 0.2 0.4 0.6 0.8 1
k / kmax
0
0.5
1
1.5
2
Sk
b=0.125 E ulerb=0.25 E ulerb=0.125 PCb=0.25 PCb=0.5 PCb=0 (I deal)
Euler
S(k) = 1 + βk2/2
Predictor/Corrector
S(k) = 1− β2k4/4
PC2RNG:
S(k) = 1 + β3k6/8
How stochastic fluxes are treated can effect accuracy
Bell, et. al., LBNL AMAR
Elements of discretization of LLNS – 1D
Spatial discretizationStochastic fluxes generated at facesStandard finite difference approximations for diffusion
Fluctuation dissipation
Higher-order reconstruction based on PPM
UJ+1/2=
712
(Uj + Uj+1) −1
12(Uj−1 + Uj+2)
Evaluate hyperbolic flux using Uj+1/2Adequate representation of fluctuations in density flux
Temporal discretizationLow storage TVD 3rd order Runge KuttaCare with evaluation of stochastic fluxes can improveaccuracy
Bell, et. al., LBNL AMAR
Structure factor for LLNS in 1D
0 0.2 0.4 0.6 0.8 1
k / kmax
0.9
0.925
0.95
0.975
1
Sk
Sr
(1R NG)
1 + (Su-1) / 3
ST
Sr
(2R NG)
Small k theory
0 0.2 0.4 0.6 0.8 1
k / kmax
0
0.05
0.1
0.15
Sk
| Sr u
| (1R NG)
| Sr T
|
| SuT
|
| Sr u
| (2R NG)
Bell, et. al., LBNL AMAR
Discretization for LLNS
〈δρ2〉 〈δJ2〉 〈δE2〉Exact value 2.35× 10−8 13.34 2.84× 1010
PPM 1.97× 10−8 13.27 2.58× 1010
RK3 2.29× 10−8 13.54 2.82× 1010
Error PPM −16.0% −0.5% −9.4%Error for RK3 −2.5% 1.5% −0.7%
Basic scheme has been generalized to 3D and two componentmixtures
Additional complication is correlation between elements ofstochastic stress tensor
Several standard discretization approaches do not correctlyrespect these correlations
Do not satisfy discrete fluctuation dissipation relationLeads to spurious correlations
Alternative approach based on randomly selecting faces onwhich to impose correlation
Preserves correlation structure in 2D and 3DBell, et. al., LBNL AMAR
SPDE versus DSMC
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 12
1. 5
1
0. 5
0
0.5
1
1.5x 10
5
x/L
<∂
ρ(x
) ∂
J(x
*)>
DSMCRK3
Correlation of density andmomentum in a thermal
gradient
0 0.5 1
x 107
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6x 10
10
Time<
∂ σ
P(t
)2 >
Mach 1
.2
Mach 1.4
Mach 2.0
Shock drift
Bell, et. al., LBNL AMAR
Hybrid Issues
� �� �� �� �����
����
� �� �� �� � ���
��� 3b
3a
2a
2b1
Maxwellian versus Chapman EnskogMeasuring gradients for C-E is problematicMaxwellian typically gives better results in hybrid
Refinement criterionUse smoothed gradient – width based on noise magnitudePotentially use breakdown parameter to guide refinement
Bell, et. al., LBNL AMAR
EquilibriumDoes deterministic PDE treatment affect fluctuations for fluidsimulations?
Energy Statistics
5 10 15 20 25 30 35 401.514
1.516
1.518x 10
6⟨ E
⟩
5 10 15 20 25 30 35 400
1
2
3x 10
10
⟨ δ E
2 ⟩ Stoch. PDE onlyDeter. hybridStoch. hybridTheory
5 10 15 20 25 30 35 40−3
−2
−1
0x 10
9
⟨ δ E
i δ E
20 ⟩
Bell, et. al., LBNL AMAR
Non-equilibrium
Look at correlations in the presence of a thermal gradient
5 10 15 20 25 30 35 402.5
2
1.5
1
0.5
0
0.5
1
1.5
x 10� 5
⟨ δ ρ
j δ J
20 ⟩
DSMCStoch. hybridRefinement region
Stochastic hybrid5 10 15 20 25 30 35 40
2.5
2
1.5
1
0.5
0
0.5
1
1.5
x 10�5
DSMCDeter. hybridRefinement region
Deterministic hybrid
Bell, et. al., LBNL AMAR
Shock propagation
t0
Ensemble of stochastic PDE runsEnsemble of stochastic hybrid runsSingle stochastic hybrid run
t1
t2
25 50 75 100 125
t3
Propagation of refined shock
Bell, et. al., LBNL AMAR
Piston problem
T1, ρ1 T2, ρ2
Piston
ρ1T1 = ρ2T2
Wall and piston are adiabatic boundariesDynamics driven by fluctuations
Bell, et. al., LBNL AMAR
Piston dynamics
Hybrid simulation of PistonSmall DSMC region near the pistonI-DSMC – see Donev talk
Bell, et. al., LBNL AMAR
Piston position vs. time
Piston versus time
0 2500 5000 7500 10000t
6
6.25
6.5
6.75
7
7.25
7.5
7.75x(
t)ParticleStoch. hybridDet. hybrid
0 250 500 750 10006
6.5
7
7.5
8
Note: Error associated with deterministic hybrid enhanced forheavier pistons
Bell, et. al., LBNL AMAR
Averge piston position vs. time
Ensemble piston motion versus time
0.0 5.0×104
1.0×105
1.5×105
2.0×105
Time
8
9
10
11
12P
isto
n po
sitio
n
Stoch. hybrid (4x40)Det. hybrid (4x40)Stoch. hybrid (8x10)Particle
0 5×104
1×105
0.5
1
2
4P
isto
n of
fset
Note: Some sensitivity to mesh used for continuum simulationBell, et. al., LBNL AMAR
Piston velocity autocorrelation
Start with piston at equilibrium location
0 1 2 3
0
5×10-4
1×10-3
VACF
ParticleStoch. wP=2
Det. wP=2
Det. wP=4
Det. wP=8
0 50 100 150 200 250
Time-5.0×10
-4
-2.5×10-4
0.0
2.5×10-4
5.0×10-4
7.5×10-4
1.0×10-3
VACF
ParticleStoch. hybridDet. hybridDet. hybrid x10
Velocity Autocorrelation FunctionBell, et. al., LBNL AMAR
Summary
Failing to include fluctuations at the continuum level in a hybrid model can pollute themicroscopic model
Hybrid methodology for capturing fluctuations
Microscopic model – DSMC
Continuum model – discretization of LLNS equationsRK3 centered schemeCaptures equilibrium fluctuations
Hybridization based on adaptive mesh refinement constructs
Future issues
Numerics / MathematicsMathematical structure of systemsCriterion for evaluating schemes / hybridsStochastic analogs of limiting – robustnessFluctuations in low Mach number flows
PhysicsReacting systemsComplex fluids
Bell, et. al., LBNL AMAR