Using HPC To Advance Water Desalination By

Electrodialysis

Clara Druzgalski Department of Mechanical Engineering

Stanford University

Water Desalination

Distillation

Reverse Osmosis

Electrodialysis

Electrodialysis: Industrial Electrodialysis water treatment plants in Barcelona, Spain produce 257 million liters of water per day. Abrera (2007) 200 million liters Sant Boi del Llobregat (2009) 57 million liters

Credit: Sant Boi del Llobregat

Electrodialysis: Applications

Gray White Black

Portable water treatment

Salt production

Biomedical analysis: lab-on-a-chip devices

Electrodialysis

Model Problem

Channel Height 10-6 meters

Smallest Feature 10-9 meters

Applied voltage 1-3 Volts

Example Dimensional Values

Model Problem: Experiments

Well-described by 1D theory

Electroconvective chaos: 1D theory

no longer predictive

“ Should we use a commercial

code like Comsol Multiphysics or build a high performance

code from scratch?

?

Commercial Software

  Commercial codes often use artificial smoothening for numerical robustness. This dissipates small structures generated by turbulent and chaotic fluid motion.

  Commerical codes must be general

enough to handle a wide variety of problems, but this limits the user’s ability to take advantage of crucial time-saving algorithms

Commercial Software

Custom HPC Software

EKaos a high performance direct numerical simulation code that simulates electrokinetic chaos.   No artificial smoothening

 Over 100 times faster than Comsol on a single node in 2D.

EKaos

2D EKaos Simulation Concentration

Charge Density

Experimental Observation

Joeri C. de Valença, R. Martijn Wagterveld, Rob G. H. Lammertink, and Peichun Amy Tsai Phys. Rev. E 92, 031003(R) – Published 8 September 2015

Simulation vs. Experiment

Experiment: De Valenca, et. al.

Simulation: Davidson, et. al. Submitted to Scientific Reports

2D EKaos: Current-Voltage

16

2D EKaos: Current-Voltage

Qualitative matching with experiment 17

3D EKaos Simulation

165 million mesh points That’s over 1 billion degrees of freedom

11 terabytes of data Per simulation

100,000 time steps To reach converged statistics

Each 3D EKaos simulation…

“ Why is a simulation of just one small section of a desalination channel so

computationally expensive?

?

The computational cost is determined by the range of relevant length and time scales that must be

resolved.

Algorithm

Details The mathematical details behind a high performance code

Governing Equations SpeciesConserva.on:

Navier-Stokes:

Gauss’sLaw:

c+ Concentration of cation c- Concentration of anion ϕ Electric potential u Velocity vector P Pressure 22

yx

Governing Equations SpeciesConserva.on:

Navier-Stokes:

Gauss’sLaw:

23

yx

Reservoir:BoundaryCondi.ons

Membrane:Periodicinxandzdirec.ons

Dimensionless Parameters Parameter Description Range Value

ϵ Screening length, EDL size 10-6 – 10-3 10-3 Δϕ Applied voltage 20-120 120 κ Electrohydrodynamic coupling const. O(1) 0.5 c0

+ Cation concentration at membrane >1 2 Sc Schmidt number 103 103

24

Spatial Discretization

25

  EKaos: 2D and 3D Direct numerical simulation (DNS)  3D has over 165 million spatial grid points  Staggered mesh configuration  Non-uniform mesh is used in the membrane-normal direction to handle sharp gradients  Discretization: 2nd order central finite difference scheme

Time Integration SpeciesConserva.on

Navier-Stokes

Gauss’sLaw

26

Time Integration SpeciesConserva.on

2ndOrderImplicitScheme

Semi-Implicit:1storder

27

Time Integration Itera.veAlgorithm

δ-form

Lineariza.on

28

Time Integration Itera.veAlgorithm

δ-form

Lineariza.on

29

Time Integration Equa.oninδ-form

RemoveDirec.onalCoupling

Movenon-s.fftermstoleQhandside

30

Time Integration Equa.oninδ-form

RemoveDirec.onalCoupling

Movenon-s.fftermstoleQhandside

31

Time Integration Equa.oninδ-form

Analy.calsubs.tu.onusingGauss’sLaw

RemoveDirec.onalCoupling

Time Integration FinalEqua.on

•  LeQhandsideoperatorislinearandnowonlyinvolveslocalcouplingbetweenδc+andδc-

•  Weneedtosolveforu*,v*,w*,P*,andϕ*ateachitera.on

33

Pseudo-spectral Solver Conserva.onofmomentum

Pressureequa.on

Gauss’sLaw

34

By taking advantage of the geometry and using physical insight we were able to: 1. Design operators that reduced the

matrix bandwidth 2. Use fast and robust math libraries

such as LAPACK and FFTW 3.  Reduce communication cost across

processors by des ign ing the algorithm with parallelization in mind.

Conclusions  Developed EKaos: a parallel 3D DNS code to

simulate electroconvective chaos.  Developed a numerical algorithm for efficiently

solving the coupled Poisson-Nernst-Planck and Navier-Stokes equations  Improved prediction of mean current density that

has been observed in experiments  Comparison of 2D and 3D simulations show

qualitative similarities, but quantitative differences  Electroconvective chaos can generate structures

similar to turbulence. 36

Thanks! Any questions?

You can find me at: cdruzgal@stanford.edu