PDE-constrained Optimization Using PETSc/TAO...Using PETSc/TAO Presented to ATPESC 2019 Participants...
Transcript of PDE-constrained Optimization Using PETSc/TAO...Using PETSc/TAO Presented to ATPESC 2019 Participants...
ATPESC Numerical Software Track
PDE-constrained OptimizationUsing PETSc/TAO
Presented to ATPESC 2019 Participants
Alp DenerPostdoctoral Appointee, MCS, ANLQ Center, St. Charles, IL (USA)Date 08/06/2019
ATPESC 2017, July 30 – August 11, 20172
What is optimization?
• Optimization variables 𝑝 ∈ ℝ$– e.g.: boundary conditions, parameters, geometry
• Objective function 𝑓: ℝ$ → ℝ– e.g.: lift, drag, pressure, temperature, stress, strain
minimizep
f(p)<latexit sha1_base64="MetxMs3CmLkjgHxDqD5ro9CMW6c=">AAACK3icbVC7TsMwFHV4vykwslhUIFiqpCDBiGBhBIkCUlNVjnPTWthOsG8QJco38CWMrPARTCBWJn4Ct3TgdSRLR+ecq+t7okwKi77/4o2Mjo1PTE5Nz8zOzS8sVpaWz2yaGw4NnsrUXETMghQaGihQwkVmgKlIwnl0edj3z6/BWJHqU+xl0FKso0UiOEMntStbYa5j5wMWWVmECDdYKKGFErdQljS8yllMN5LNbKtdqfo1fwD6lwRDUiVDHLcrH2Gc8lyBRi6Ztc3Az7BVMIOCSyhnwtxCxvgl60DTUc0U2FYxOKmk606JaZIa9zTSgfp9omDK2p6KXFIx7NrfXl/8z2vmmOy1CqGzHEHzr0VJLimmtN8PjYUBjrLnCONGuL9S3mWGcXQt/dwiO6kLdFXdHePaCX538Zec1WvBdq1+slPdPxj2NEVWyRrZJAHZJfvkiByTBuHkjjyQR/Lk3XvP3qv39hUd8YYzK+QHvPdPnvWo6Q==</latexit>
ATPESC 2017, July 30 – August 11, 20173
What is optimization?
• Simplification: 𝑓(𝑝) is minimized where ∇+𝑓 𝑝 = 0
• Gradient-free: Heuristic search through 𝑝 space
• Gradient-based: Find search directions based on ∇+𝑓
minimizep
f(p)<latexit sha1_base64="MetxMs3CmLkjgHxDqD5ro9CMW6c=">AAACK3icbVC7TsMwFHV4vykwslhUIFiqpCDBiGBhBIkCUlNVjnPTWthOsG8QJco38CWMrPARTCBWJn4Ct3TgdSRLR+ecq+t7okwKi77/4o2Mjo1PTE5Nz8zOzS8sVpaWz2yaGw4NnsrUXETMghQaGihQwkVmgKlIwnl0edj3z6/BWJHqU+xl0FKso0UiOEMntStbYa5j5wMWWVmECDdYKKGFErdQljS8yllMN5LNbKtdqfo1fwD6lwRDUiVDHLcrH2Gc8lyBRi6Ztc3Az7BVMIOCSyhnwtxCxvgl60DTUc0U2FYxOKmk606JaZIa9zTSgfp9omDK2p6KXFIx7NrfXl/8z2vmmOy1CqGzHEHzr0VJLimmtN8PjYUBjrLnCONGuL9S3mWGcXQt/dwiO6kLdFXdHePaCX538Zec1WvBdq1+slPdPxj2NEVWyRrZJAHZJfvkiByTBuHkjjyQR/Lk3XvP3qv39hUd8YYzK+QHvPdPnvWo6Q==</latexit>
ATPESC 2017, July 30 – August 11, 20174
Why do we care?
Inputs
• Boundary Conditions
• Parameters• Geometry
Simulation
• Fluid Dynamics• Continuum
Mechanics• Heat Transfer
Outputs
• Lift• Drag• Stress• Temperature
We know a lot about how to solve the forward problem…
ATPESC 2017, July 30 – August 11, 20175
Why do we care?
Inputs
• Boundary Conditions
• Parameters• Geometry
Simulation
• Fluid Dynamics• Continuum
Mechanics• Heat Transfer
Outputs
• Lift• Drag• Stress• Temperature
We know a lot about how to solve the forward problem…
ControlDecisionDesign
Optimization Objective
…many scientific questions present as inverse problems!
ATPESC 2017, July 30 – August 11, 20176
Why do we care?
ATPESC 2017, July 30 – August 11, 20177
Outline
• Intro to PDE-constrained Optimization– Full-space formulation– Reduced-space formulation
• Introduction to TAO– Sample main program– User/problem function callback
• Sensitivity Analysis– Finite difference method– Adjoint method
• Hands-on Example: Boundary Control w/ 2D Laplace Equation
ATPESC 2017, July 30 – August 11, 20178
PDE-constrained Optimization
• Optimization variables 𝑝 ∈ ℝ$
• State variables 𝑢 ∈ ℝ/
• State equations 𝑅: ℝ$1/ → ℝ/
• “Full-space” formulation – optimization includes both optimization and state variables
minimizep,u
f(p, u)
subject to R(p, u) = 0<latexit sha1_base64="mGNk3+DRQJVjQtGbmfkQyc+lDe4=">AAACWHicbVBdaxNBFL3ZVtvGr9g++jI0KBUk7MZCfRFK++JjFZMWsiHMzt5Nxs7MrjN3xLjsP/OP6KOv9Uc4+UD6dWDgcM653Lknq5R0FMe/WtHG5oOHW9s77UePnzx91nm+O3SltwIHolSlvci4QyUNDkiSwovKIteZwvPs8nThn39D62RpPtO8wrHmUyMLKTgFadIZpt7kwUeqqzfMN3VK+J1qLY3U8gc2DUu/ep6zV8XBwn+dpu1VwvnsCwpiVP6PfFpF2HsWTzrduBcvwe6SZE26sMbZpHOV5qXwGg0JxZ0bJXFF45pbkkJh0069w4qLSz7FUaCGa3Tjenl/w14GJWdFacMzxJbq9Ymaa+fmOgtJzWnmbnsL8T5v5Kl4N66lqTyhEatFhVfhaLYok+XShg7UPBAurAx/ZWLGLRcUKr25RU3LEJjpfjgmtJPc7uIuGfZ7ydte/+Nh9/hk3dM2vIB9OIAEjuAYPsAZDEDAT/gDV/C39TuCaCvaWUWj1npmD24g2v0H5gS07Q==</latexit>
ATPESC 2017, July 30 – August 11, 20179
First-order Optimality Conditions
• Construct the Lagrangian where 𝜆 ∈ ℝ/
• Differentiate w.r.t. every input for first-order optimality
• Also known as the Karush-Kuhn-Tucker (KKT) conditions
L(p, u,�) = f(p, u) + �TR(p, u)<latexit sha1_base64="CweUwrxex4TBqKN6/xN9UdTRjbk=">AAACOXicbZDLSsNAFIYnXmu9RV26GSxCi6UkVagboejGhYsqvUFTy2Q6aYdOLsxMhBL6KD6JS7f6AC7diS59ASdpBNv6w8DPd87hzPntgFEhDeNNW1peWV1bz2xkN7e2d3b1vf2m8EOOSQP7zOdtGwnCqEcakkpG2gEnyLUZadmjq7jeeiBcUN+ry3FAui4aeNShGEmFenrFcpEcYsSim0k+KMKwCC2mxvuoAC+gk6ACPPmF9/W7KerpOaNkJIKLxkxNDqSq9fQvq+/j0CWexAwJ0TGNQHYjxCXFjEyyVihIgPAIDUhHWQ+5RHSj5MAJPFakDx2fq+dJmNC/ExFyhRi7tuqMzxHztRj+V+uE0jnvRtQLQkk8PF3khAxKH8ZpwT7lBEs2VgZhTtVfIR4ijrBUmc5uYQNfNQzdsjpGpWPOZ7FomuWSeVoq357lqpdpThlwCI5AHpigAqrgGtRAA2DwCJ7BC3jVnrR37UP7nLYuaenMAZiR9v0DpauqWA==</latexit>
rpL =@f
@p+ �T @R
@p= 0
ruL =@f
@u+ �T @R
@u= 0
r�L = R(u, p) = 0<latexit sha1_base64="3VjKXQj0UpdcCC2PbvDhovLgEeg=">AAAC+niclVJNTxsxEPVuW6ChhdAeuViNqKhaRbsBiV4qofbCoQeKCCDFIZp1vImF12v5Ayna7q3X9kf0hrj2z/Q38Ac41htWapNwKCNZenrzxm8840QJbmwU/Q7CR4+fLC2vPG2sPnu+tt7ceHFicqcp69Jc5PosAcMEl6xruRXsTGkGWSLYaXLxqcqfXjJteC6P7USxfgYjyVNOwXpq0LwlEhIBA4VJBnZMQRSfS/z6AyapBloQBdpyEDgt/2JV4reYCG8yhPPjOeHRrNDfFGFCGrWNe4CN+18bt2BTV82bHW27d1i9maoHzVbUjqaBF0Fcgxaq43DQvCHDnLqMSUsFGNOLI2X7RdUDFaxsEGeYAnoBI9bzUELGTL+YbqjEW54Z4jTX/kiLp+y/FQVkxkyyxCurls18riLvy/WcTd/3Cy6Vs0zSO6PUCWxzXK0bD7lm1IqJB0A1971iOgY/Sus/xayLGOVeMM46/jF+OvH8LBbBSacd77Q7X3Zb+x/rOa2gTfQKbaMY7aF9dIAOURfR4Dz4FnwPfoRfw5/hVXh9Jw2DuuYlmonw1x9Pae8+</latexit>
ATPESC 2017, July 30 – August 11, 201710
Solving the Problem
• Apply Newton’s method to the KKT conditionsFor k = 0, 1, 2, …
Convergence check (i.e., ∇+ℒ ≤ 𝜀+ and 𝑅 ≤ 𝜀6)
Solve
Step acceptance with globalization (e.g., line search)
2
64r2
ppL r2upL @R
@p
T
r2puL r2
uuL @R@u
T
@R@p
@R@u 0
3
75
k
0
@�p�u��
1
A =
0
@�rpL�ruL
�R(p, u)
1
A
k<latexit sha1_base64="Ccx8Sg02gm954bRDMrcr5+DXVyw=">AAAEA3ichVNNb9MwGPYSPkb5WAdHLhYV05CgSgoSXJAm4MCBw5jWbVJdIsd1UquOY/kDUUU5cuQKP4Ib4soP4TfwJ3C6dGubUl4p0qP3ef089mMnlpxpEwS/tzz/ytVr17dvtG7eun1np71790TnVhHaJznP1VmMNeVM0L5hhtMzqSjOYk5P48nrij/9SJVmuTg2U0mHGU4FSxjBxrWiXQ9AVyimKRNFnGGj2KeyBetCAsccR4WU5YceRI4eE8yLdyXc27sg7VoyUZgUSGJlGObwqLzEbvwYItQ0sZtM1pL/NrGrJpv281+taiCYaSEqRhc5RZPWQnqykd4byg2G0u1jju0CRtxd0whfys4F4Eu4UfdJHYpcysMJzwnbII725WNoHzW9oknU7gTdYFawCcIadEBdh1H7DxrlxGZUGMKx1oMwkGZYVGERTssWsppKTCY4pQMHBc6oHhazp1rCh64zgkmu3CcMnHUXVxQ403qaxW6yOoJe5armOm5gTfJiWDAhraGCnBsllkOTw+rdwxFTlBg+dQATxdxeIRljd+fG/R3LLjzN3cA467nDuHTC1Sya4KTXDZ92e++fdQ5e1Tltg/vgAdgHIXgODsBbcAj6gHip98X76n3zP/vf/R/+z/NRb6tecw8slf/rLzNuQYo=</latexit>
ATPESC 2017, July 30 – August 11, 201711
Solving the Problem
• Apply Newton’s method to the KKT conditionsFor k = 0, 1, 2, …
Convergence check (i.e., ∇+ℒ ≤ 𝜀+ and 𝑅 ≤ 𝜀6)
Solve
Step acceptance with globalization (e.g., line search)
2
64r2
ppL r2upL @R
@p
T
r2puL r2
uuL @R@u
T
@R@p
@R@u 0
3
75
k
0
@�p�u��
1
A =
0
@�rpL�ruL
�R(p, u)
1
A
k<latexit sha1_base64="Ccx8Sg02gm954bRDMrcr5+DXVyw=">AAAEA3ichVNNb9MwGPYSPkb5WAdHLhYV05CgSgoSXJAm4MCBw5jWbVJdIsd1UquOY/kDUUU5cuQKP4Ib4soP4TfwJ3C6dGubUl4p0qP3ef089mMnlpxpEwS/tzz/ytVr17dvtG7eun1np71790TnVhHaJznP1VmMNeVM0L5hhtMzqSjOYk5P48nrij/9SJVmuTg2U0mHGU4FSxjBxrWiXQ9AVyimKRNFnGGj2KeyBetCAsccR4WU5YceRI4eE8yLdyXc27sg7VoyUZgUSGJlGObwqLzEbvwYItQ0sZtM1pL/NrGrJpv281+taiCYaSEqRhc5RZPWQnqykd4byg2G0u1jju0CRtxd0whfys4F4Eu4UfdJHYpcysMJzwnbII725WNoHzW9oknU7gTdYFawCcIadEBdh1H7DxrlxGZUGMKx1oMwkGZYVGERTssWsppKTCY4pQMHBc6oHhazp1rCh64zgkmu3CcMnHUXVxQ403qaxW6yOoJe5armOm5gTfJiWDAhraGCnBsllkOTw+rdwxFTlBg+dQATxdxeIRljd+fG/R3LLjzN3cA467nDuHTC1Sya4KTXDZ92e++fdQ5e1Tltg/vgAdgHIXgODsBbcAj6gHip98X76n3zP/vf/R/+z/NRb6tecw8slf/rLzNuQYo=</latexit>
Conjugate GradientQuasi-NewtonNewton-Krylov…
ATPESC 2017, July 30 – August 11, 201712
Solving the Problem
• Full-space formulation– PDE solution tightly coupled with optimization– Avoid the cost of a complete PDE solution at every optimization iteration– Large saddle-point problem – difficult to solve
2
64r2
ppL r2upL @R
@p
T
r2puL r2
uuL @R@u
T
@R@p
@R@u 0
3
75
k
0
@�p�u��
1
A =
0
@�rpL�ruL
�R(p, u)
1
A
k<latexit sha1_base64="Ccx8Sg02gm954bRDMrcr5+DXVyw=">AAAEA3ichVNNb9MwGPYSPkb5WAdHLhYV05CgSgoSXJAm4MCBw5jWbVJdIsd1UquOY/kDUUU5cuQKP4Ib4soP4TfwJ3C6dGubUl4p0qP3ef089mMnlpxpEwS/tzz/ytVr17dvtG7eun1np71790TnVhHaJznP1VmMNeVM0L5hhtMzqSjOYk5P48nrij/9SJVmuTg2U0mHGU4FSxjBxrWiXQ9AVyimKRNFnGGj2KeyBetCAsccR4WU5YceRI4eE8yLdyXc27sg7VoyUZgUSGJlGObwqLzEbvwYItQ0sZtM1pL/NrGrJpv281+taiCYaSEqRhc5RZPWQnqykd4byg2G0u1jju0CRtxd0whfys4F4Eu4UfdJHYpcysMJzwnbII725WNoHzW9oknU7gTdYFawCcIadEBdh1H7DxrlxGZUGMKx1oMwkGZYVGERTssWsppKTCY4pQMHBc6oHhazp1rCh64zgkmu3CcMnHUXVxQ403qaxW6yOoJe5armOm5gTfJiWDAhraGCnBsllkOTw+rdwxFTlBg+dQATxdxeIRljd+fG/R3LLjzN3cA467nDuHTC1Sya4KTXDZ92e++fdQ5e1Tltg/vgAdgHIXgODsBbcAj6gHip98X76n3zP/vf/R/+z/NRb6tecw8slf/rLzNuQYo=</latexit>
ATPESC 2017, July 30 – August 11, 201713
An Alternative Approach
• Reduced-space formulation– PDE-constraint is eliminated via implicit function theorem– Optimization algorithm and PDE solver are independent– Each objective evaluation requires a full PDE solution
minimizep
f(p, u(p))<latexit sha1_base64="Q2NAkcEiP+Nlwr6pgNMqcpsshuM=">AAACMXicbVDLThsxFPVQKI/SkrbLbiyioiChdCYgtUtEN10GiQBSJoo8njuJhe0x9jUiHc1n9Eu6ZNt+BLuq2/wETsiC15EsHZ1zrq7vyYwUDuP4Nlp6tbzyenVtfePN5tt3W433H05d6S2HHi9lac8z5kAKDT0UKOHcWGAqk3CWXXyf+WdXYJ0o9QlODAwUG2lRCM4wSMPGl9TrPPiAlamrFOEaKyW0UOIn1DVNLz3L6U7RMnvUt8zu7rDRjNvxHPQ5SRakSRboDhvTNC+5V6CRS+ZcP4kNDipmUXAJ9UbqHRjGL9gI+oFqpsANqvlhNf0clJwWpQ1PI52rDycqppybqCwkFcOxe+rNxJe8vsfi26AS2ngEze8XFV5SLOmsJZoLCxzlJBDGrQh/pXzMLOMYunq8RY7KEBirTjgmtJM87eI5Oe20k/125/igeXi06GmNfCLbpEUS8pUckh+kS3qEk1/khvwhf6Pf0W30L/p/H12KFjMfySNE0ztT1qqn</latexit>
ATPESC 2017, July 30 – August 11, 201714
Toolkit for Advanced Optimization (TAO)
• General-purpose continuous optimization toolbox for large-scale problems– Parallel (via PETSc data structures)– Gradient-based– Bound-constrained – Nonlinear constraint support under development
• Support for reduced-space PDE-constrained optimization• Distributed with PETSc (https://www.mcs.anl.gov/petsc/)
• Similar packages:– Rapid Optimization Library (https://trilinos.github.io/rol.html)– HiOP (https://github.com/LLNL/hiop)
Vec
KSP
SNES
TS
Mat
PC
DM
TAO
PETSc
ATPESC 2017, July 30 – August 11, 201715
TAO: The Basics
• Sample main programAppCtx user; Tao tao; Vec P;
PetscInitialize( &argc, &argv,(char *)0,help ); VecCreateMPI(PETSC_COMM_WORLD, user.n, user.N, &P); VecSet(P, 0.0);
TaoCreate(PETSC_COMM_WORLD, &tao); TaoSetType(tao, TAOBQNLS); /* BQNLS: quasi-Newton line search */TaoSetInitialVector(tao, P); TaoSetObjectiveAndGradientRoutine(tao, FormFunctionGradient, (void*) &user); TaoSetFromOptions(tao); TaoSolve(tao);
VecDestroy(&P); TaoDestroy(&tao); PetscFinalize();
ATPESC 2017, July 30 – August 11, 201716
TAO: The Basics
• User provides function for problem implementationAppCtx user; Tao tao; Vec P;
PetscInitialize( &argc, &argv,(char *)0,help ); VecCreateMPI(PETSC_COMM_WORLD, user.n, user.N, &P); VecSet(P, 0.0);
TaoCreate(PETSC_COMM_WORLD, &tao); TaoSetType(tao, TAOBQNLS); /* BQNLS: quasi-Newton line search */TaoSetInitialVector(tao, P); TaoSetObjectiveAndGradientRoutine(tao, FormFunctionGradient, (void*) &user); TaoSetFromOptions(tao); TaoSolve(tao);
VecDestroy(&P); TaoDestroy(&tao); PetscFinalize();
ATPESC 2017, July 30 – August 11, 201717
TAO: User Function
• User function computes objective and gradienttypedef struct {
/* user-created context for storing application data */} AppCtx;
PetscErrorCode FormFunctionGradient(Tao tao, Vec P, PetscReal *fcn, Vec G, void *ptr) { AppCtx *user = (AppCtx*)ptr; const PetscScalar *pp; PetscScalar *gg;
VecGetArrayRead(P, &pp); VecGetArray(G, &gg); /* USER TASK: Compute objective function and store in fcn *//* USER TASK: Compute compute gradient and store in gg */VecRestoreArrayRead(P, &pp); VecRestoreArray(G, &gg);
return 0; }
ATPESC 2017, July 30 – August 11, 201718
TAO: User Function
• State/PDE solution:Solve 𝑅 𝑝, 𝑢 = 0 for 𝑢(𝑝) at new 𝑝
• Objective evaluation: Compute 𝑓(𝑝, 𝑢 𝑝 )
• Sensitivity analysis:Compute G = ∇+𝑓 at 𝑝 and 𝑢(𝑝)
ATPESC 2017, July 30 – August 11, 201719
TAO: User Function
• State/PDE solution:Solve 𝑅 𝑝, 𝑢 = 0 for 𝑢(𝑝) at new 𝑝
• Objective evaluation: Compute 𝑓(𝑝, 𝑢 𝑝 )
• Sensitivity analysis:Compute G = ∇+𝑓 at 𝑝 and 𝑢(𝑝)
• Necessary for gradient-based optimization
• Converges to the local optimum faster than gradient-free methods (i.e., fewer PDE solutions)
• Types:• Finite difference• Discrete adjoint method
ATPESC 2017, July 30 – August 11, 201720
Sensitivity Analysis: Finite Difference
• Easy to implement– Only requires function evaluations
• Inefficient for large numbers of optimization variables• Step-size dilemma – truncation error vs. subtractive cancellation
Simulation𝑝9 + 𝛿 𝑓 + 𝛿
𝑝 𝑓−
+
𝑑𝑓𝑑𝑝9
ATPESC 2017, July 30 – August 11, 201721
Sensitivity Analysis: Discrete Adjoint Method
• Recall the first-order optimality conditions for PDE-constrained optimization
• Reduced-space formulation solves equations (2) and (3) fully at each optimization iteration and substitutes into equation (1)
rpL =@f
@p+ �T @R
@p= 0 (1)
ruL =@f
@u+ �T @R
@u= 0 (2)
r�L = R(u, p) = 0 (3)<latexit sha1_base64="dLQXPwLf7G8xO8cwlaekpdi+Em8=">AAADGXiclZLNbtNAEMfX5quEj6YgceGyIgIlAkV2ikQvSFW5cOBQqqatlA3ReL1OVl2vl/2oFFl+Eo5cy0NwQ1w58Qy8BOvUEnXKAUZa6a+Z//g3nt1ECW5sFP0MwmvXb9y8tXG7c+fuvfub3a0HR6ZwmrIxLUShTxIwTHDJxpZbwU6UZpAngh0np2/q+vEZ04YX8tAuFZvmMJc84xSsT822gkdEQiJgpjDJwS4oiPJdhZ+9xiTTQEuiQFsOAmfVH60q/BwT4SkpfDhcMx60jf5LESYfHaS4Hw8wIZ0G6P4D6P4V6NrA0WVg07+OPei7F1gNWn3bg1m3Fw2jVeCrIm5EDzWxP+v+ImlBXc6kpQKMmcSRstOynosKVnWIM0wBPYU5m3gpIWdmWq4usMJPfSbFWaH9kRavspc7SsiNWeaJd9bDm/VanfxbbeJstjMtuVTOMkkvQJkT2Ba4fg045ZpRK5ZeANXcz4rpAvx6rX8zbYqYF96wyEf+Z/x24vVdXBVHo2G8PRy9f9nb3Wv2tIEeoyeoj2L0Cu2it2gfjRENquBzcB58CT+FX8Nv4fcLaxg0PQ9RK8IfvwEpq/fT</latexit>
ATPESC 2017, July 30 – August 11, 201722
Sensitivity Analysis: Discrete Adjoint Method
1. State/PDE Solution:Solve 𝑅 𝑝, 𝑢 = 0 for 𝑢 at new 𝑝
2. Adjoint Solution:
Solve >?>6
@𝜆 = − >A
>6 for 𝜆 at new 𝑝 and u(𝑝)
3. Gradient Assembly:Compute G = ∇+𝑓 =
>A>+ + 𝜆
@ >?>+
ATPESC 2017, July 30 – August 11, 201723
Sensitivity Analysis: Discrete Adjoint Method
1. State/PDE Solution:Solve 𝑅 𝑝, 𝑢 = 0 for 𝑢 at new 𝑝
2. Adjoint Solution:
Solve >?>6
@𝜆 = − >A
>6 for 𝜆 at new 𝑝 and u(𝑝)
3. Gradient Assembly:Compute G = ∇+𝑓 =
>A>+ + 𝜆
@ >?>+
Solve ∇Cℒ = 0 (eqn. 3)
Solve ∇6ℒ = 0 (eqn. 2)
Evaluate ∇+ℒ (eqn. 1)
ATPESC 2017, July 30 – August 11, 201724
Sensitivity Analysis: Discrete Adjoint Method
• Computational cost is independent of the number of optimization variables– One linear system solution for each scalar function
• Requires transpose operators for the PDE Jacobians (w.r.t. both state and optimization variables)
• PETSc/TS (Time Steppers) package implements checkpointing for backwards-in-time adjoint solutions– Not covered in this tutorial
ATPESC 2017, July 30 – August 11, 201725
Hands-on Example: Boundary Control w/ 2D Laplace Eqn
• Control left, top and bottom Dirichlet bounds to recover target solution at the right boundary
minimizep
1
2
Z 1
0(u(1, y)� utarget)
2dy
governed by@2u
@x2+
@2u
@y2= 0, 8 x, y 2 (0, 1)
@u
@x
����u(1,y)
= 0, 8 y 2 (0, 1)
p =⇥u(x, 0) u(x, 1) u(0, y)
⇤T<latexit sha1_base64="4+Tlkiha2Ox72r+Iwq0gCZCl+6E=">AAADeXicfVJdb9MwFE0TPkbZoINHXi5UVCl0VZIhgYSQJnjhcUjrNqluI8dxUmuOExxnajD5oTzwC/gJvOC01bR1wJUsHd97fO6Hb1RwVirP+9GxnTt3793fedB9uLv36HFv/8lpmVeS0AnJeS7PI1xSzgSdKKY4PS8kxVnE6Vl08amNn11SWbJcnKi6oLMMp4IljGBlXOF+5xJVIjYEqnTRaKToUumMCZaxb7RpAH2tcAwDlEhMtN/ooEFMqNCb++BWrj+qh3AAVagVlilVzXAeQFwj1F0LpbnJLWgMUb0lhQosFcPc8Kvm6gbLedDAa/gvp245H8AbbRRRkkvMOaD3sBxBDaZAcE3UH4IpZIA4TdR4S/JmUlObZOlCfQ/1uqd/6V8Xb7ULw0MRTZnQUYaVZMumcpcjbwgDaIG/Bp6RRFTEV6T5Sdjre2NvZXAb+BvQtzZ2HPZ+oTgnVUaFIhyX5dT3CjXTbQuE06aLqpIWmFzglE4NFDij5UyvFqSBl8YTg+nDHKFg5b3+QuOsLOssMkxT4aLcjrXOv8WmlUrezTQTRaWoIOtEScVB5dBuG8RMUqJ4bQAmkplagSyw+QllVu5mFp7mhrDIAtOMmY6/PYvb4DQY+4fj4Mub/tHHzZx2rGfWC8u1fOutdWR9to6tiUU6P23H3rX37N/Oc8d1Xq2pdmfz5ql1w5zDP5O4FS8=</latexit>
ATPESC 2017, July 30 – August 11, 201726
Hands-on Example: Boundary Control w/ 2D Laplace Eqn
• PDE solution, objective function and gradient evaluations implemented with AMReX(https://amrex-codes.github.io/amrex/)
• Target solution set to 𝑢DEFGHD = 4(𝑦 − 0.5) M−0.5
• Lapace equation is self-adjoint– PDE Jacobian is symmetric
• Command line options:– Problem size: -nx 128– Switch on finite difference gradient: -fd
ATPESC 2017, July 30 – August 11, 201727
Hands-on Example: Boundary Control w/ 2D Laplace Eqn
Initial Final
ATPESC 2017, July 30 – August 11, 201728
Hands-on Example: Boundary Control w/ 2D Laplace Eqn
$ ./main2d.gnu.MPI.ex inputs –tao_monitor –tao_ls_type armijo –tao_fmin 1e-6 –tao_gatol 1e-12
AMReX (19.08) initialized0 TAO, Function value: 0.00564792, Residual: 4.57574e-081 TAO, Function value: 0.00564792, Residual: 4.57574e-082 TAO, Function value: 0.000351831, Residual: 6.72001e-093 TAO, Function value: 0.000104118, Residual: 4.5399e-094 TAO, Function value: 4.39909e-06, Residual: 5.62804e-105 TAO, Function value: 3.20011e-06, Residual: 3.18434e-106 TAO, Function value: 2.90562e-06, Residual: 3.67294e-107 TAO, Function value: 2.82687e-06, Residual: 3.54986e-108 TAO, Function value: 2.74912e-06, Residual: 3.45265e-109 TAO, Function value: 2.74912e-06, Residual: 3.45265e-10
10 TAO, Function value: 2.5271e-06, Residual: 1.99102e-1011 TAO, Function value: 2.41122e-06, Residual: 1.43435e-1012 TAO, Function value: 2.10344e-06, Residual: 8.34685e-1113 TAO, Function value: 1.78759e-06, Residual: 1.11393e-1014 TAO, Function value: 1.30814e-06, Residual: 8.4585e-1115 TAO, Function value: 1.10837e-06, Residual: 9.14759e-1116 TAO, Function value: 1.08865e-06, Residual: 9.52258e-1117 TAO, Function value: 8.87623e-07, Residual: 1.29118e-10
TaoSolve() duration: 1316289 microseconds[The Pinned Arena] space (MB) used spread across MPI: [8 ... 8]AMReX (19.08) finalized
ATPESC 2017, July 30 – August 11, 201729
Hands-on Example: Boundary Control w/ 2D Laplace Eqn
• Tutorial goals:– Compare computational cost of finite difference gradient to the adjoint method
– Verify that the cost of the adjoint method is (mostly) independent of the number of optimization variables
– Solve the problem with different TAO algorithms
– Interpret the TAO monitor output and assess convergence
ATPESC 2017, July 30 – August 11, 201730
Take Away Messages
• PDE-constrained optimization does not need to be intimidating!– PETSc/TAO provides interfaces and algorithms that work with reduced-space formulations– PETSc/TAO can compute gradients and Hessians automatically with finite differencing– PETSc data structures are easy to couple with most PDE solvers (e.g., AMReX)
• The adjoint method is ideal for sensitivity analysis– Some PDE solvers already have necessary building blocks– Possible to take implementation shortcuts in self-adjoint problems
• PETSc/TAO offers parallel optimization algorithms for large-scale problems– Optimization data structures are duplicated from user-generated PETSc vectors– User has full control over parallel distribution and vector type
ATPESC 2017, July 30 – August 11, 201731
Acknowledgements
AMReX: https://amrex-codes.github.io/amrex/
PETSc/TAO: https://www.mcs.anl.gov/petsc/
Special thanks to Don Willcox for support in developing the hands-on example code.
Support for this work was provided through Scientific Discovery through Advanced Computing (SciDAC) program and the Exascale Computing Project funded by U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research.