Large-Scale Density Functional Calculations

James E. Raynolds, College of Nanoscale Science and Engineering

Lenore R. Mullin, College of Computing and Information

Overview

• Using computers to carry out “numerical experiments”

in Materials Science, Chemistry and Physics• Quantum Mechanical equations solved for a

system of atoms in a representative unit cell• Measurable properties obtained from

“first-principles”– mechanical, thermodynamic, electronic– optical, magnetic, transport

Example: Transport in molecular wire

Benzene

Phenolate/Benzenediazonium+ V

Peierls DistortionPi stacked pair dimerized pair

metal insulator

mechanical relaxation

Frontier Problems

• Non-equilibrium spin-transport in metals and semiconductors (Spintronics)

• Transport and coupled mechanical / electronic interactions in molecules (metal - insulator transition due to mechanical relaxation)

• Industrial applications: Modeling Chemical Vapor Deposition (CVD) processes atom by atom

• Challenges: correlated motion of electrons• Coupled electron-phonon interactions

(electron - vibration coupling)

Density Functional Theory

• Density Functional Theory (DFT) is a “mean-field” solution to the many-electron problem.

• Each electron interacts with an effective average field produced by all of the other electrons

• Non-linear set of coupled differential equations

Density Functional Equations

€

−∇2 + V (r r )( )Ψ j (

r r ) = E jΨ j (

r r )

Looks linear but...

€

V (r r ) depends on the charge density

€

ρ(r r )

through:

€

ρ(r r ) = Ψ j (

r r )

j

∑2

Example: Local density approximation

€

V (r r ) = d3s

ρ (r s )

r r −

r s

∫ +δ

δρ (r r )

d3∫ sρ (r r )εxc (ρ(

r r ))

DFT solution approach

• Expand the wave-functions in a basis set:

• Matrix eigenvalue-eigenvector problem:

• Orthogonality:

• Iterative solution to “self-consistency” (i.e. output V(r) coincides with input)

€

Ψj (r r ) = Cl

j

l

∑ ϕ l (r r )

€

H jl

l

∑ Clj = EC j

j

€

Clk

( )l

∑*Cl

j = δkj

Popular implementations

• Plane wave basis functions (Fourier Series):

– Drawback: – Benefit: easy to code, sophisticated non-linear

response calculations possible

• Localized “atomic-like” basis functions

€

ϕ j (r r ) =

1

Vexp(ik j •

r r )

€

O(N 3) scaling

€

ϕ j (r r ) = a j (

r r ) - exponential distance decay for

insulators- power law distance decay for - metals

Contrasting Implementations

• Abinit: www.abinit.org– Very sophisticated array of calculated properties– Calculations become prohibitive for more than a few dozen

atoms • VASP (Vienna Ab-Initio Simulation Package)

– Less sophisticated by much faster– few hundred atoms possible

• Siesta: (Spanish Initiative for Electronic Simulations with Thousands of Atoms)– O(N) scaling: fast but less sophisticated– few thousand atoms possible

€

O(N 3)

€

O(N 3)

Public Access

• Many codes are freely available: go to http://psi-k.dl.ac.uk/data/codes.html for a list of more than 20

• Most codes still not user-friendly and take months to years to master

The Brick Wall!!

• All of these methods run out of steam very quickly in terms of run time and memory

• Calculations with scaling take days or weeks to run!!

• Even calculations with scaling run into memory bottlenecks

• Materials Science simulations require thousands of atoms for thousands of time steps

€

O(N 3)

€

O(N)

Key Algorithms

• For plane wave based codes:

the Fast Fourier Transform– We have gained factor’s of 4 improvement in

speed and storage using Conformal Computing– A number of new developments are being

implemented for further increases

• Matrix diagonalization routines for very large matrices

Conformal Computing

• Density Functional Calculations are an ideal setting for Conformal Computing!

• In fact: any array (matrix) based computational setting is ripe for Conformal Computing

• Why? Conformal Computing eliminates temporary arrays and un-necessary loops!

Opportunities

• Current electronic band structures fairly fast (on the order of one hour):

Contrasting: electron-phonon

• Electron-phonon calculations: on the order of 1 day for small systems

• Superconductivity in “conventional” materialsdetermined by the electron -phonon interaction• Aluminum (1 atom) takes roughly 1 day of computing• Imagine several dozen atomswith scaling

€

O(N 3)

Electron-Phonon improvements

• Many quantities currently written to files then later combined

• The size and number of these files is becoming prohibitively expensive

• Opportunities for parallelization of integrals

• Opportunities to eliminate temporaries through the use of direct indexing

Grid Computing

• Even with highly optimized code (which is still a way off) there is always a need for more and more resources

• For example: electron-phonon calculations involve dozens of separate calculations that could be run on independent machines

• Grid computing allows many independent calculations to be run in parallel

Grid Computing: First Steps

• QMolDyn GAT: a template for submitting Density Functional Calculations over the grid

• Vision: QMolDyn will eventually have a variety of codes (modules)

• Presently: Siesta ( ) running on the grid, 8, 16, 32, 64, 128, 256, 512- atom systems

€

O(N)

Summary / Conclusions

• There is a great demand for large-scale array (matrix) based calculations in materials science

• Quantum calculations are increasingly important for Materials Science, Chemistry and Physics

• Grid computing combined with Conformal Computing techniques is very promising