Post on 10-Jan-2016
description
Sparse linear solvers applied to Sparse linear solvers applied to parallel simulations of parallel simulations of
underground flow underground flow in porous and fractured mediain porous and fractured media
A. BeaudoinA. Beaudoin11, J.R. De Dreuzy, J.R. De Dreuzy22, J. Erhel, J. Erhel11 and H. and H. MustaphaMustapha11
1 - IRISA / INRIA, Rennes, France1 - IRISA / INRIA, Rennes, France
2 - Department of Geosciences, University of Rennes2 - Department of Geosciences, University of Rennes, , FranceFrance
Matrix Computations and Scientific Computing Seminar
Berkeley, 26 October 2005
2D heterogeneous porous medium2D heterogeneous porous medium
Heterogeneous Heterogeneous permeability fieldpermeability fieldY = ln(K)Y = ln(K)with correlation functionwith correlation function
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
2( ) expY YY
C
rr
91 Y
3D fracture network with impervious matrix3D fracture network with impervious matrix
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
length distribution has a great impact : power law n(l) = l-a
3 types of networks based on the moments of length distribution
mean variation third moment3 < a < 4
mean variation2 < a < 3
mean variation third momenta > 4
EquationsEquations
Q = - K*Q = - K*grad (hgrad (h) )
div (Q) = 0div (Q) = 0 BoundaryBoundary conditions conditions
Flow modelFlow model
Fixed head
Nul flux
3D fracture network3D fracture network
Fix
ed
head
Fix
ed
head
Nul flux
Nul flux
2D porous medium2D porous medium
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
Numerical method for 2D heterogeneous Numerical method for 2D heterogeneous porous mediumporous medium
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
Finite Volume Method with a regular mesh
Large sparse structured matrix with 5 entries per row
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
n=32 zoom
Sparse matrix for 2D heterogeneous porous Sparse matrix for 2D heterogeneous porous mediummedium
Conforming Conforming triangular triangular
meshmesh
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
Mixed Hybrid Finite Element Method with unstructured mesh
Large sparse unstructured matrix with about 5 entries per row
Numerical method for 3D Numerical method for 3D fracture networkfracture network
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
Sparse matrix for 3D fracture Sparse matrix for 3D fracture networknetwork
N = 8181
Intersections and 7 fractures
zoom
Memory requirements for matrices A and LMemory requirements for matrices A and L
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
Complexity analysis with PSPASESComplexity analysis with PSPASES
CPU time of matrix generation, linear solving and flow computationCPU time of matrix generation, linear solving and flow computationobtained with two processorsobtained with two processors
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
Complexity analysis with PSPASESComplexity analysis with PSPASES
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
2D porous medium : memory size and CPU time 2D porous medium : memory size and CPU time with PSPASESwith PSPASES
Theory : NZ(L) = O(N logN) Theory : Time = O(N1.5)
Slope about 1 Slope about 1.5
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
3D fracture network : memory size and CPU time 3D fracture network : memory size and CPU time with PSPASESwith PSPASES
NZ(L) = O(N) ? Time = O(N) ?
Theory to be done
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
2D porous medium : condition number estimated by 2D porous medium : condition number estimated by MUMPSMUMPS
To be ckecked : scaling or not
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
2D porous medium : residuals with PSPASES 2D porous medium : residuals with PSPASES
Parallel architectureParallel architecturedistributed memorydistributed memory
2 nodes of 32 bi – processors 2 nodes of 32 bi – processors (Proc AMD Opteron 2Ghz with 2Go (Proc AMD Opteron 2Ghz with 2Go
of RAM)of RAM)
Parallel architectureParallel architecture
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
Scalability analysis with PSPASES : speed-upScalability analysis with PSPASES : speed-up
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
TpT2S 2
Scalability analysis with PSPASES : isoefficiencyScalability analysis with PSPASES : isoefficiency
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
PTp
TE S
PTp
NR
P N Tp R
2 0.26 106 5.60 1.20 106
8 1.05 106 11.33 1.18 106
32 4.19 106 25.70 1,04 106
4 0.26 106 2.92 1.15 106
16 1.05 106 6.06 1.11 106
64 4.19 106 13.08 1,05 106
P N Tp R
2 0.26 106 13.10
8 1.05 106 22.06
32 4.19 106 38.41
4 0.26 106 7.94
16 1.05 106 16.05
64 4.19 106 No value No value
2D medium 3D fracture network
5.1 ?
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
2D porous medium : number of V cycles with 2D porous medium : number of V cycles with HYPRE/SMGHYPRE/SMG
Comparison between PSPASES and HYPRE/SMG : Comparison between PSPASES and HYPRE/SMG : CPU timeCPU time
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
PSPASESHYPRE
Comparison between PSPASES and HYPRE/SMG : Comparison between PSPASES and HYPRE/SMG : speed-upspeed-up
HYPRE PSPASES
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
PerspectivesPerspectives
Parallel Simulations of Underground Flow in Porous Parallel Simulations of Underground Flow in Porous and Fractured Mediaand Fractured Media
• porous medium : large sigma, up to 9 and large N, up to 108
• porous medium : 3D problems, N up to 1012
• porous medium : scaling, iterative refinement, multigrid adapted to heterogeneous permeability field
• 3D fracture networks : large N, up to 109
• model for complexity and scalability issues• 2-level nested dissection • subdomain method
• parallel architectures : up to 128 processors• Monte-Carlo simulations• grid computing with clusters for each random simulation
• parallel advection-diffusion numerical models