NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,
description
Transcript of NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,
![Page 1: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/1.jpg)
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
NOX: An Object-Oriented Nonlinear Solver Package
Roger Pawlowski, Tamara Kolda, Russell Hooper, and John Shadid
Sandia National Laboratories Albuquerque, NM
Trilinos User Group MeetingOctober 16th, 2003
![Page 2: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/2.jpg)
Background• NOX is a Trilinos solver package built to:
– enable robust and efficient solutions to systems of nonlinear equations.
– provide a broad array of nonlinear algorithms through a single interface.
– rapid deployment of new solver technology into ASCI codes.– eliminate redundancies in ASCI code development.
• Started ~2.5 years ago. • Funding: ASCI Algorithms and LDRD (2.5 FTE).• Consists of two libraries:
– NOX – nonlinear algorithms– LOCA v2.0 – continuation and bifurcation algorithms (Eric Phipps)
• Website: software.sandia.gov/nox
![Page 3: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/3.jpg)
In-NOX-ulated Applications
Code Application NOX Contact Application ContactXyce Electrical Circuit Simulation Kolda & Pawlowski Scott HutchinsonMPSalsa Chemically Reacting Flows Pawlowski Roger PawlowskiFEAP Solid Mechanics Hooper Mark AdamsCharon General PDE Solver Pawlowski Gary HennigPremo Compressible Flows Hooper & Pawlowski Tom SmithSIERRA Framework Hooper Alan Williams
Code Application NOX Contact Application ContactSalinas Solid Mechanics Hooper Kendall PiersonAdagio Solid Mechanics Hooper John MitchellALEGRA Radiation Transport Pawlowski Tom BrunnerGOMA Incompressible Flows Pawlowski Sam SubiaNEVADA Framework Pawlowski Tom Brunner
Fully Interfaced
Prototype Interface
Primary NOX Team: Tammy Kolda, Roger Pawlowski, Russ Hooper
![Page 4: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/4.jpg)
Nonlinear Equations
Given
find for which
• F is a system of n nonlinear equations with n unknowns.
• x is the unknown or solution vector of size n.
![Page 5: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/5.jpg)
MB f xc Bcd+=
Broyden’s Method
Newton’s MethodMN f xc
Jc d+=
Tensor Method MT f xc
Jcd 12---Tcdd+ +=
Nonlinear Solution Algorithms
Iterative Linear Solvers, Adaptive Forcing Terms
Line SearchInterval Halving
QuadraticCubic
More’-Thuente
HomotopyArtificial Parameter ContinuationNatural Parameter Continuation
Trust RegionDogleg
Inexact Dogleg
Globalizations
User Norms and Merit Functions
![Page 6: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/6.jpg)
Stopping Criteria(Status Test)
Example: Newton’s Method for F (x) = 0
• Choose an initial guess x0
• For k = 0,1,2,...– Compute Fk = F (xk)– Compute Jk where
(Jk )ij = F i(xk)/x j
– Let dk = -Jk-1 Fk
– (Optional) Let k be a calculated step length
– Set xk+1 = xk + kdk
– Test for Convergence or Failure
Calculatingthe Direction
Damping orLine Search
Iterate Control(Solver)
Building Blocks of NOX
![Page 7: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/7.jpg)
NOX Solvers(Iterate Control)
1. LineSearchBased • Compute Direction• Compute Step Length• Scale direction by step length• Update solution• Check Convergence
2. TrustRegionBased (dogleg)• Computes Newton Direction• Computes Cauchy Direction• Adjust trust region radius• Update solution• Check Convergence
3. InexactTrustRegionBased (Prerelease)
4. TensorBased (Prerelease)5. UserDefined
Solver is determined by passing a parameter list to a handle for the class:
NOX::Solver::Manager solver(group, StatusTests, ParameterList);int status = solver.solve();
Derived from NOX::Solver::Genericbool reset(NOX::Abstract::Group& grp, NOX::StatusTest::Generic& tests, NOX::Parameter::List& params) bool reset(NOX::Abstract::Group& grp, NOX::StatusTest::Generic& tests) NOX::StatusTest::StatusType getStatus() NOX::StatusTest::StatusType iterate() NOX::StatusTest::StatusType solve() const NOX::Abstract::Group& getSolutionGroup() const const NOX::Abstract::Group& getPreviousSolutionGroup() const int getNumIterations() const const NOX::Parameter::List& getParameterList() const
![Page 8: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/8.jpg)
NOX Directions• Steepest Descent
• Newton• Requires a linear solve• Adaptive forcing terms
• Broyden• Modified Newton with
rank-1 updates to the most recently computed Jacobian
• Nonlinear-CG (Prerelease)
• Tensor (Prerelease)
User Defined DirectionsDerived from NOX::Direction::GenericPassed into the solver via parameter list using a direction template ctor.
User Defined Merit FunctionDerived from NOX::Parameter::MeritFunctionPassed into the solver via parameter list using the NOX::Parameter::Arbitrary entry
Direction is determined and constructed in a handle called a “Manager”:
NOX::Direction::Manager dir(PrintParams, DirectionParams);
![Page 9: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/9.jpg)
NOX Line Searches
• Full Step• Constant step size.• Defaults to 1.0.
• Backtrack• interval halving
• Polynomial Interpolation• Quadratic and Cubic interpolation• Requires a “Sufficient Decrease”
Condition
• More’-Thuente• Polynomial Interpolation• Sufficient Decrease and Curvature
Conditions
• Nonlinear-CG – (Prerelease) • Curvilinear (Prerelease)
User Defined Line SearchesDerived from NOX::LineSearch::GenericPassed into the solver via parameter list using a template constructor.
User Defined Merit FunctionDerived from NOX::Parameter::MeritFunctionPassed into the solver via parameter list using the NOX::Parameter::Arbitrary entry
User Defined NormsDerived from NOX::Parameter::UserNormPassed into the solver via parameter list using the NOX::Parameter::Arbitrary entry
Line Searches are determined and constructed in a handle called a “Manager”:
NOX::LineSearch::Manager ls(PrintParams, LineSearchParams);
![Page 10: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/10.jpg)
Mix-n-Match Solver Algorithms
• Higly versatile code environment• Solver
– Line Search Based– Trust Region Based– Tensor Based
• Direction– Newton– Broyden– Steepest Descent– Tensor– User-Defined
• Line Search / Damping– Full Step– Backtrack– Polynomial/Quadratic– More’-Thuente– Curvilinear– User-Defined
Newton
Trust-Region Method
Tensor Method
Newly-DeployedBroyden Method
Quadratic Line Search
![Page 11: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/11.jpg)
Stopping Criteria (StatusTests)
Highly Flexible Design: Users build a convergence test hierarchy and registers it with the solver (via solver constructor or reset method).
– Norm F: {Inf, One, Two} {absolute, relative}– Norm Update X: {Inf, One, Two}– Norm Weighted Root Mean Square (WRMS)
– Max Iterations – Failure test if solvers reaches max iters– FiniteValue – Failure test that checks for NaN and Inf on – Stagnation – Failure test that triggers if the convergence rate
fails a tolerance check for n consecutive iterations.
– Combination: {AND, OR}– Users Designed: Derive from NOX::StatusTest::Generic
Return TypesUnconvergedConverged*Failed*Unevaluated
![Page 12: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/12.jpg)
Building a Status Test
• Fail if value of becomes Nan or Inf
NOX::StatusTest::FiniteValue finiteValueTest;
FiniteValue: finiteValueTest
• Fail if we reach maximum iterations
• Converge if both:
MaxIters: maxItersTest
NOX::StatusTest::MaxIters maxItersTest(200);
normFTest
NOX::StatusTest::NormF normFTest();normWRMSTest
NOX::StatusTest::NormWRMS normWRMSTest();
Combo(AND): convergedTest
NOX::StatusTest::Combo convergedTest(NOX::StatusTest::Combo::AND);
Combo(OR)allTests
NOX::StatusTest::Combo allTests(NOX::StatusTest::Combo::OR);allTests.addStatusTest(finiteValueTest);allTests.addStatusTest(maxItersTest);allTests.addStatusTest(convergedTest);
convergedTest.addStatusTest(normFTest);convergedTest.addStatusTest(normWRMSTest);
![Page 13: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/13.jpg)
Status Tests Continued
User Defined are Derived from NOX::StatusTest::GenericNOX::StatusTest::StatusType checkStatus(const NOX::Solver::Generic &problem) NOX::StatusTest::StatusType checkStatusEfficiently(const NOX::Solver::Generic &problem, NOX::StatusTest::CheckType checkType) NOX::StatusTest::StatusType getStatus() const ostream& print(ostream &stream, int indent=0) const
-- Status Test Results --**...........OR Combination -> **...........AND Combination -> **...........F-Norm = 5.907e-01 < 1.000e-08 (Length-Scaled Two-Norm, Absolute Tolerance) **...........WRMS-Norm = 4.794e+01 < 1 (Min Step Size: 1.000e+00 >= 1) (Max Lin Solv Tol: 1.314e-15 < 0.5) **...........Finite Number Check (Two-Norm F) = Finite **...........Number of Iterations = 2 < 200
-- Final Status Test Results --Converged....OR Combination -> Converged....AND Combination -> Converged....F-Norm = 3.567e-13 < 1.000e-08 (Length-Scaled Two-Norm, Absolute Tolerance) Converged....WRMS-Norm = 1.724e-03 < 1 (Min Step Size: 1.000e+00 >= 1) (Max Lin Solv Tol: 4.951e-14 < 0.5) ??...........Finite Number Check (Two-Norm F) = Unknown ??...........Number of Iterations = -1 < 200
![Page 14: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/14.jpg)
NOX Framework
SolverLayer
Abstract Vector & Abstract Group
AbstractLayer
Solvers- Line Search
- Trust Region
Directions- e.g., Newton
Line Searches- e.g., Polynomial
Status Tests- e.g., Norm F
• Don’t need to directly access the vector or matrix entries, only manipulate the objects.
• NOX uses an abstract interface to manipulate linear algebra objects.• Isolate the Solver layer from the linear algebra implementations used by
the application.• This approach means that NOX does NOT rely on any specific linear
algebra format.• Allows the apps to tailor the linear algebra to their own needs!
– Serial or Parallel– Any Storage format: LAPACK, PETSc, Epetra, User Defined
![Page 15: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/15.jpg)
NOX::Abstract::Vector• Initialization
– x = y – x = |y|– xi = 1/yi for i = 1 to n– xi = for i = 1 to n
• Length of Vector
• Scaling– x = x– xi = xiyi for i = 1 to n
• Update– x = a + x, – x = a + b + x
• Norm– k x k1, k x k2 , k x k1– k x kw (weighted norm)
• Dot– x ¢ y
• Clone (create a copy)– y = x
The solver is not allowed nor does it need explicit
access to the vector – just the ability to manipulate
it.
![Page 16: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/16.jpg)
NOX::Abstract::Group
• x = Iterate – Initialize– Update– Access
• F = F (x)– Compute– Access
• J = Jacobian of F at x– Compute– Apply / Apply Transpose– Apply Inverse– Apply Preconditioning
• n = Newton Vector – Compute n = -J-1F to specified
tolerance– Access
• g = Gradient of kF (xk)k2
– g = JTF– Access
• Clone
The linear solver and application interface are combined into
the Group. Vectors x, F, n, g are accessed
as NOX::Abstract::Vecto
r’s. Matrix J is never
directly accessed!
![Page 17: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/17.jpg)
Code Demonstration
![Page 18: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/18.jpg)
NOX Framework
SolverLayer
Abstract Vector & Abstract Group
AbstractLayer
Linear AlgebraInterface
Implementations- EPetra- PETSc- LAPACK- USER DEFINED
EPetra Dependent Features- Matrix-Free Newton-Krylov- Preconditioning- Graph Coloring / Finite Diff.
Solvers- Line Search
- Trust Region
Directions- e.g., Newton
Line Searches- e.g., Polynomial
Status Tests- e.g., Norm F
ApplicationInterface
Layer
User Interface- Compute F- Compute Jacobian- Compute Preconditioner
![Page 19: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/19.jpg)
Linear Algebra Support Features
• NOX’s support libraries define a concrete implementation of the Abstract Vector and Group.
• We only require the user to implement a minimal interface derived from NOX::<Package>::Interface.
bool computeF(const Epetra_Vector &x, Epetra_Vector &f, FillType flag=F)
bool computeJacobian(const Epetra_Vector &x, Epetra_Operator &Jac) bool computePrecMatrix(const Epetra_Vector &x, Epetra_RowMatrix &M) bool computePreconditioner(const Epetra_Vector &x, Epetra_Operator &M)
Group(NOX::Parameter::List& printingParams, NOX::Parameter::List& linearSolverParams, NOX::Epetra::Interface& i, NOX::Epetra::Vector& x, Epetra_Operator& J)
Group(NOX::Parameter::List& printingParams, NOX::Parameter::List& linearSolverParams, NOX::Epetra::Interface& i, NOX::Epetra::Vector& x, Epetra_Operator& J, Epetra_Operator& M)
![Page 20: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/20.jpg)
The Epetra “Goodies”• Matrix-Free Newton-Krylov Operator
• Derived from Epetra_Operator• Can be used to estimate Jacobian action on a
vector • NOX::Epetra::MatrixFree
• Finite Difference Jacobian• Derived from an Epetra_RowMatrix • Can be used as a preconditioner matrix• NOX::Epetra::FiniteDifference
• Graph Colored Finite Difference Jacobian• Derived from NOX::Epetra::FiniteDifference• Fast Jacobian fills – need connectivity/coloring
graph• (NOX::Epetra::FiniteDifferenceColoring)
• Full interface to AztecOO using NOX parameter list• Preconditioners: internal AztecOO, Ifpack, User defined • Scaling object
Jy F x y+ F x –-----------------------------------------=
JjF x ej+ F x –
-------------------------------------------=
![Page 21: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/21.jpg)
Homotopy Algorithms
LOCA: Library of Continuation Algorithms
• Bifurcation and stability analysis package.
• Tightly coupled to NOX to reuse application interface.
• Provided as a single package.• Stepper object provides a robust
and intelligent step control for parameter continuation.
• Eric Phipps
Parameter ContinuationSolve a series of problems thatprogressively become thesystem of interest based on a parameter, :Natural: FH (x) = F(x,)Artificial: FH (x) = F(x) + (1 – )G(x)
0 1
F(x)
G(x) easy
![Page 22: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/22.jpg)
NOX Specific Configure Options
• Compiling NOX library in Trilinos: --enable-nox
• Compiling prerelease code: --enable-prerelease
• Using nox built-in linear algebra support:LAPACK: --enable-nox-lapack
--enable-nox-lapack-examplesEpetra: --enable-nox-epetra
--enable-nox-epetra-examplesPETSc: --enable-nox-petsc
--enable-nox-petsc-examples
• Compiling the test suite:--enable-tests--enable-nox-tests
• Compiling LOCA library in Trilinos/NOX: --enable-loca
![Page 23: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/23.jpg)
Future Work
• Test Suite Development – Darin Diachin• Re-work of the NOX::Epetra support library for efficiency -
Pawlowski• TSF support library – Pawlowski, Hooper• Tensor algorithm development – Brett Bader, Tammy Kolda• Broyden algorithm development – Kolda, Pawlowski• Multi-Physics Solvers – Strong code coupling
![Page 24: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/24.jpg)
Multi-Physics Solvers• MultiPhysics – Bill Spotz and Alfred Lorber
• rapid prototyping environment to explore solver coupling algorithms.• PyNOX - Python interface to call NOX solver.
• SIERRA – Russell Hooper, John Shadid and Roger Pawlowski• Implementation of a Jacobian-Free coupling algorithm in SIERRA.
Loose Coupling Full Coupling
kk1k
k1kT
kkTT
kk1k
kkT
kkTT
CCC
C,TRCJ
TTT
C,TRTJ
kC
kT
k
k
kCC
kCT
kTC
kTT
RR
CT
JJJJ
kkk RUJ
Jacobian-Free Coupling
URpURJp
URpMURpJM
11
with preconditioner:
k
CC
kTT
J00J
MAnalytic
ORFD Coloring (Epetra)
TCTxCD
tC 2
2
2
2
)x()x,0(T6.0)1,t(T)0,t(T
)x()x,0(C
6.00.2)1,t(C)0,t(C
T)1(CTxTD
tT 2
2
2
1
Brusselator Examplein NOX
(Russell Hooper)
![Page 25: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/25.jpg)
Summary• Key Features
– Object-oriented C++ library– Large number of cutting-edge algorithms– Can use any linear algebra implementation– Easy interface – User Flexibility: Add solvers, directions, line searches,
convergence tests
• NOX Support libraries– Fast integration – reduced interface– parallel iterative solvers (AztecOO, PETSc)– preconditioners (Ifpack, ML, Aztec, PETSc)– Matrix-Free Newton-Krylov– Matrix Estimation: Finite Difference/Coloring
![Page 26: NOX: An Object-Oriented Nonlinear Solver Package Roger Pawlowski , Tamara Kolda, Russell Hooper,](https://reader035.fdocuments.us/reader035/viewer/2022062323/56815d46550346895dcb4dd5/html5/thumbnails/26.jpg)
NOX Contributors (Ideas, Code, and Testing)
Sandia– Tammy Kolda *– Roger Pawlowski *– Russ Hooper *– John Shadid– Todd Coffey *– Andy Salinger (LOCA) *– Eric Phipps (LOCA) *– Bill Spotz *– Mike Heroux (Trilinos) – Scott Hutchinson (Xyce)– Rob Hoekstra (Xyce)– Eric Keiter (Xyce)– Alan Williams (SIERRA)– Tom Smith (Premo)– Alfred Lorber (Premo)– Tom Brunner (Alegra)
Academics– Homer Walker, WPI– Joe Simonis, WPI– Tim Kelley, NCSU– Bobby Schnabel, UCB– Richard Byrd, UCB– Ryan McKenzie, Kentucky– Craig Douglas, Kentucky
* NOX Developer
NOX InformationTammy Kolda: [email protected]
Roger Pawlowski: [email protected]: http://software.sandia.gov/nox