Animal farm chapter 3 with ayg quiz, then appositives with practice, then prompt writing
PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga...
-
Upload
matthew-dwight-farmer -
Category
Documents
-
view
236 -
download
0
Transcript of PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga...
![Page 1: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/1.jpg)
PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package
Ananth Grama and Metin Aktulga
![Page 2: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/2.jpg)
Acknowledgements
• Rajiv Kalia, Aiichiro Nakano, Priya Vashishtha (USC)• Adri van Duin (Penn State)• Steve Plimpton, Aidan Thompson (Sandia)
• US Department of Energy• US National Science Foundation
![Page 3: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/3.jpg)
Outline
1. Sequential Realization: SerialReax• Algorithms and Numerical Techniques• Validation• Performance Characterization
2. Parallel Realization: PuReMD• Parallelization: Algorithms and Techniques• Performance and Scalability
3. Applications• Strain Relaxation in Si/Ge/Si Nanobars• Water-Silica Interface• Oxidative Stress in Lipid Bilayers
![Page 4: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/4.jpg)
SerialReax Components
SystemGeometry
Control Parameters
Force Field Parameters
TrajectorySnapshots
System StatusUpdate
Program LogFile
SerialReaxInitialization• Read input data• Initialize data structs
Compute BondsCorrections are applied after all uncorrected bonds are computed
Bonded Interactions1.Bonds2.Lone pairs3.Over/UnderCoord4.Valence Angles5.Hydrogen Bonds6.Dihedral/ Torsion
QEq• Large sparse linear system• PGMRES(50) or PCG• ILUT-based pre-conditioners give good performance
Evolve the System• Ftotal = Fnonbonded + Fbonded
• Update x & v with velocity Verlet• NVE, NVT and NPT ensembles
ReallocateFully dynamic and adaptive memory management:• efficient use of
resources• large systems on a
single processor
Neighbor Generation• 3D-grid based O(n)
neighbor generation• Several optimizations for
improved performance
Init Forces• Initialize the QEq coef matrix• Compute uncorr. bond
orders• Generate H-bond lists
vd Waals & electrostatics• Single pass over the far nbr-list after charges are updated• Interpolation with cubic splines for nice speed-up
![Page 5: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/5.jpg)
Linear Solver for Charge Equilibration
QEq Method: used for equilibrating charges• original QEq paper cited 600+ times• approximation for distributing partial charges
• solution using Lagrange multipliers yields
N = # of atomsH: NxN sparse matrixs & t fictitious charges: used to determine the actual charge qi
Too expensive for direct methods Iterative (Krylov sub-space) methods
![Page 6: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/6.jpg)
Basic Solvers for QEqSample systems
• bulk water: 6540 atoms, liquid• lipid bilayer system: 56,800 atoms, biological system• PETN crystal: 48,256 atoms, solid
Solvers: CG and GMRES• H has heavy diagonal: diagonal pre-conditioning• slowly evolving environment : extrapolation from prev. solutions
Poor Performance:tolerance level = 10-6
which is fairly satisfactory
much worse at 10-10 tolerance level
due to cache effects
more pronounced here
# of iterations = # of matrix-vector multiplications
actual running time in seconds
fraction of total computation time
![Page 7: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/7.jpg)
ILU-based preconditioning
ILU-based pre-conditioners: no fill-in, 10-2 drop tolerance• effective (considering only the solve time)
• no fill-in + threshold: nice scaling with system size• ILU factorization is expensive
bulk water system
bilayer system
cache effects are still evident
system/solver time to compute preconditioner
solve time (s)
total time (s)
bulk water/ GMRES+ILU 0.50 0.04 0.54
bulk water/ GMRES+diagonal ~0 0.11 0.11
![Page 8: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/8.jpg)
ILU-based preconditioning
Observation: can amortize the ILU factorization costslowly changing simulation environment re-usable pre-conditioners
PETN crystal:solid, 1000s
of steps!
Bulk water:liquid, 10-100s
of steps!
![Page 9: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/9.jpg)
Validation
Hexane (C6H14) Structure Comparison
Excellent agreement!
![Page 10: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/10.jpg)
Comparison to MD Methods
Comparisons using hexane: systems of various sizes• Ab-initio MD: CPMD• Classical MD: GROMACS• ReaxFF: SerialReax
![Page 11: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/11.jpg)
Comparison to LAMMPS-ReaxF
Time per time-step comparison
Qeq solver performance
Memory foot-print
• different QEq formulations
• similar results
• LAMMPS: CG / no preconditioner
![Page 12: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/12.jpg)
Outline
1. Sequential Realization: SerialReax• Algorithms and Numerical Techniques• Validation• Performance Characterization
2. Parallel Realization: PuReMD• Parallelization: Algorithms and Techniques• Performance and Scalability
3. Applications• Strain Relaxation in Si/Ge/Si Nanobars• Water-Silica Interface• Oxidative Stress in Lipid Bilayers
![Page 13: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/13.jpg)
Parallelization: Outer-Shell
br
br
br/2
br
full shell half shell
midpoint-shelltower-plate shell
![Page 14: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/14.jpg)
Parallelization: Outer-Shell
br
br
br/2
br
full shell half shell
midpoint-shelltower-plate shell
choose full-shell due to dynamic bonding despite the comm. overhead
![Page 15: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/15.jpg)
Parallelization: Boundary Interactions
rshell= MAX (3xrbond_cut, rhbond_cut, rnonb_cut)
![Page 16: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/16.jpg)
Parallelization: Messaging
![Page 17: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/17.jpg)
Parallelization: Messaging Performance
Performance Comparison: PuReMD with direct vs. staged messaging
![Page 18: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/18.jpg)
Parallelization: Optimizations
Truncate bond related computations• double computation of bonds at the outer-shell• hurts scalability as the sub-domain size shrinks• trim all bonds that are further than 3 hops or more
Scalable parallel solver for the QEq problem• GMRES/ILU factorization: not scalable• use CG + diagonal pre-conditioning• good initial guess:
• redundant computations: to avoid reverse communication• iterate both systems together
![Page 19: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/19.jpg)
PuReMD: Performance and Scalability
Weak scaling test
Strong scaling test
Comparison to LAMMPS-REAX
Platform: Hera cluster at LLNL
• 4 AMD Opterons/node -- 16 cores/node
• 800 batch nodes – 10800 cores, 127 TFLOPS/sec
• 32 GB memory / node
• Infiniband interconnect
• 42nd on TOP500 list as of Nov 2009
![Page 20: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/20.jpg)
PuReMD: Weak Scaling
Bulk Water: 6540 atoms in a 40x40x40 A3 box / core
![Page 21: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/21.jpg)
PuReMD: Weak Scaling
QEq Scaling Efficiency
![Page 22: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/22.jpg)
PuReMD: Strong Scaling
Bulk Water: 52320 atoms in a 80x80x80 A3 box
![Page 23: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/23.jpg)
PuReMD: Strong Scaling
Efficiency and throughput
![Page 24: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/24.jpg)
Outline
1. Sequential Realization: SerialReax• Algorithms and Numerical Techniques• Validation• Performance Characterization
2. Parallel Realization: PuReMD• Parallelization: Algorithms and Techniques• Performance and Scalability
3. Validation Applications• Strain Relaxation in Si/Ge/Si Nanobars• Water-Silica Interface• Oxidative Stress in Lipid Bilayers
![Page 25: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/25.jpg)
PureMD/Reax/C User Community
![Page 26: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/26.jpg)
Si/Ge/Si nanoscale bars
Motivation• Si/Ge/Si nanobars: ideal for MOSFETs• as produced: biaxial strain, desirable: uniaxial• design & production: understanding strain behavior is important
Si
Si
Ge
Width (W) Periodic boundary
conditions
Hei
ght
(H)
[100], Transverse
[001], Vertical
[010] , Longitudinal
Related publication:Strain relaxation in Si/Ge/Si nanoscale bars from molecular dynamics simulationsY. Park, H.M. Aktulga, A.Y. Grama, A. StrachanJournal of Applied Physics 106, 1 (2009)
![Page 27: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/27.jpg)
Si/Ge/Si nanoscale bars
Key Result:When Ge section is roughly square shaped, it has almost uniaxial strain!
0 2 4 6 8 10
-2.0
-1.5
-1.0
-0.5
Height of Ge [nm]
W = 20.09 nm
Str
ain
(%)
W = 20.09 nm
SiGeSi
average transverse Ge strain
0 10 20 30 40 50 60-3-2-10123
Bar width (nm)
-3-2-10123
(b)
(a)
Str
ain
(%)
X, Transverse Y, Longitudinal Z, Vertical
HGe
= 6.39 nm Strain on Ge
Str
ain
(%)
Strain on Si
average strains for Si&Ge in each dimension
0
0.0209 2 0.0209 relW
W
0 10 20 30 40 50 60 70-2.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5 1.06 nm 2.13 nm 3.19 nm 6.39 nm 9.58 nm
Stra
in(%
)
Bar width (nm)
Simple strain model derived from MD results
![Page 28: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/28.jpg)
Water-Silica Interface
Motivation• a-SiO2: widely used in nano-electronic devices• also used in devices for in-vivo screening• understanding interaction with water: critical for reliability of devices
Related publication:A Reactive Simulation of the Silica-Water Interface J. C. Fogarty, H. M. Aktulga, A. van Duin, A. Y. Grama, S. A. Pandit Journal of Chemical Physics 132, 174704 (2010)
![Page 29: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/29.jpg)
Water-Silica Interface
Water model validation
Silica model validation
![Page 30: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/30.jpg)
Water-Silica Interface
Silica
Water
Si OO
O HH
Key ResultSilica surface hydroxylation as evidenced by experiments is observed.
Proposed reaction:H2O + 2Si + O 2SiOH
![Page 31: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/31.jpg)
Water-Silica Interface
Key ResultHydrogen penetration is observed: some H atoms penetrate through the slab.
Ab-initio simulations predict whole molecule penetration.We propose: water is able to diffuse through a thin film of silica via hydrogen hopping, i.e., rather than diffusing as whole units, water molecules dissociate at the surface, and hydrogens diffuse through, combining with other dissociated water molecules at the other surface.
![Page 32: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/32.jpg)
Oxidative Damage in Lipid Bilayers
MotivationModeling reactive processes in biological systemsROS Oxidative stress Cancer & Aging
System Preparation200 POPC lipid + 10,000 water molecules and same system with 1% H2O2 mixed
Mass Spectograph: Lipid molecule weighs 760 u
Key ResultOxidative damage observed:In pure water: 40% damagedIn 1% peroxide: 75% damaged
![Page 33: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/33.jpg)
Conclusions• An efficient and scalable realization of ReaxFF through use of algorithmic
and numerical techniques• Detailed performance characterization; comparison to other methods• Applications on various systems• LAMMPS/Reax/C and PuReMD strand-alone Reax implementations
available over the public domain.
• BUT!• Lots needs to be done – the parallel qEq is the current bottleneck• Use alternate factorizations that parallelize well (SPIKE)• GPU implementation forthcoming.
![Page 34: PuReMD: A Reactive (ReaxFF) Molecular Dynamics Package Ananth Grama and Metin Aktulga ayg@cs.purdue.edu.](https://reader035.fdocuments.us/reader035/viewer/2022081506/56649ec95503460f94bd6d3c/html5/thumbnails/34.jpg)
References
Reactive Molecular Dynamics: Numerical Methods and Algorithmic TechniquesH. M. Aktulga, S. A. Pandit, A. C. T. van Duin, A. Y. GramaSIAM Journal on Scientific Computing, to appear, 2011.
Parallel Reactive Molecular Dynamics: Numerical Methods and Algorithmic TechniquesH. M. Aktulga, J. C. Fogarty, S. A. Pandit, A. Y. GramaParallel Computing, to appear, 2011.
Strain relaxation in Si/Ge/Si nanoscale bars from molecular dynamics simulationsY. Park, H.M. Aktulga, A.Y. Grama, A. StrachanJournal of Applied Physics 106, 1 (2009)
A Reactive Simulation of the Silica-Water Interface J. C. Fogarty, H. M. Aktulga, A. van Duin, A. Y. Grama, S. A. Pandit Journal of Chemical Physics 132, 174704 (2010)