Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis....
Transcript of Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis....
![Page 1: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/1.jpg)
Recent Advancements in Differential Equation Solver SoftwareCHRIS RACKAUCKASMASSACHUSETTS INSTITUTE OF TECHNOLOGY
1
Jubilee Symposium 2019: Future Directions of System Modeling and Simulation
![Page 2: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/2.jpg)
DifferentialEquations.jl: Research Platform for Production ~300 methods available, including wrappers to C/Fortran methods
Platform for reproducible research and benchmarking MPI+GPU Compatibility Implicit, IMEX, multirate, symplectic, exponential integrators, etc. Adaptive high order methods for stochastic differential equations Stiff state-dependent delay differential equation discontinuity tracking Mix in Gillespie simulation (Continuous-Time Markov Chains) Automatic sparsity detection and optimization Arbitrary code injection through callbacksNative Julia methods routinely benchmark as one of the fastest libraries in most categories (caveat category: large (>1000) ODE/DAE systems)
2
![Page 3: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/3.jpg)
3 Major Areas
Neural Differential Equations
A Hackable Model Compiler (ModelingToolkit.jl)
Improvements to basic numerical methods
3
![Page 4: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/4.jpg)
There is a new field that is merging AI and domain-specific modeling: Scientific ML/AI
SCIENTIFIC AI: DOMAIN MODELS WITH INTEGRATED MACHINE LEARNING HTTPS://WWW.YOUTUBE.COM/WATCH?V=FGFX8CQHDQATHE ESSENTIAL TOOLS OF SCIENTIFIC MACHINE LEARNINGHTTP://WWW.STOCHASTICLIFESTYLE.COM/THE-ESSENTIAL-TOOLS-OF-SCIENTIFIC-MACHINE-LEARNING-SCIENTIFIC-ML/
4
![Page 5: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/5.jpg)
What is the mathematical structure of machine learning?
5
![Page 6: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/6.jpg)
Neural Networks = Nonlinear Regression Polynomial: 𝑒𝑒𝑥𝑥 = 𝑎𝑎1 + 𝑎𝑎2𝑥𝑥 + 𝑎𝑎3𝑥𝑥2 + ⋯
Nonlinear: 𝑒𝑒𝑥𝑥 = 1 + 𝑎𝑎1 tanh 𝑎𝑎2𝑥𝑥𝑎𝑎3𝑥𝑥−tanh 𝑎𝑎4𝑥𝑥
Neural Network: 𝑒𝑒𝑥𝑥 ≈ 𝑊𝑊3𝜎𝜎 𝑊𝑊2𝜎𝜎 𝑊𝑊1𝑥𝑥 + 𝑏𝑏1 + 𝑏𝑏2 + 𝑏𝑏3. Train the weights (𝑊𝑊, 𝑏𝑏)
6
![Page 7: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/7.jpg)
Universal Approximation
Theorem
NEURAL NETWORKS CAN GET 𝜖𝜖 CLOSE TO ANY 𝑅𝑅𝑛𝑛 → 𝑅𝑅𝑚𝑚 FUNCTION Neural Networks Overcome “the curse of dimensionality”
7
![Page 8: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/8.jpg)
Not Quite a Black Box:Convolutional Neural Networks Encode (Spatial) Structure
8
A Comprehensive Guide to Convolutional Neural Networks — the ELI5 way, Sumit Saha
![Page 9: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/9.jpg)
Now let’s generalize this idea to scientific structures
9
![Page 10: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/10.jpg)
Latent (Neural) Differential Equations
NEURAL ORDINARY DIFFERENTIAL EQUATION: 𝑢𝑢′ = 𝑓𝑓(𝑢𝑢, 𝑝𝑝, 𝑡𝑡)
LET 𝑓𝑓 BE A NEURAL NETWORK
![Page 11: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/11.jpg)
Training a neural differential equation:DiffEqFlux.jl
11
Solve the differential equation
Compute the gradient of the solution with respect to the parameters defining the neural network
Adjoint sensitivity analysisDifferentiable programming
Update the neural network and repeat
![Page 12: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/12.jpg)
Automatically Learning the Model12
![Page 13: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/13.jpg)
The real power comes from incorporating known structure into the ML framework(Mixed Neural Differential Equation)
13
![Page 14: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/14.jpg)
Mix Neural Networks Into DiffEqs!
𝑑𝑑𝑥𝑥𝑑𝑑𝑡𝑡
=𝑑𝑑𝑑𝑑𝑑𝑑𝑡𝑡
= 𝑝𝑝1𝑑𝑑 + 𝑝𝑝2𝑥𝑥
Fit the “mixed neural differential equation” using the same method!
14
![Page 15: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/15.jpg)
ML-Assisted Model Discovery
The chemical reactions imply an evolution of:
?
?
?
? ?
Data
?
15
![Page 16: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/16.jpg)
Biologically-Informed Neural Network
NN(2)
NN(3)
NN(4)
?
DataFind neural networks so the model matches the data
16
![Page 17: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/17.jpg)
Interpretability of Neural Differential Equations
Analyze the Jacobian/Hessian
17
Data-Efficient Physics-Embedded Machine Learning
![Page 18: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/18.jpg)
Nonlinear Optimal Control as a Mixed Neural ODE
𝑥𝑥′ = 𝑓𝑓(𝑥𝑥 𝑡𝑡 ,𝑢𝑢 𝑡𝑡 , 𝑡𝑡)
Minimize 𝐽𝐽 = Φ 𝑥𝑥 𝑡𝑡0 , 𝑡𝑡0, 𝑥𝑥 𝑡𝑡𝑓𝑓 , 𝑡𝑡𝑓𝑓 + ∫𝑡𝑡0𝑡𝑡𝑓𝑓 𝐿𝐿 𝑥𝑥 𝑡𝑡 ,𝑢𝑢 𝑡𝑡 , 𝑡𝑡 𝑑𝑑𝑡𝑡
Example: 𝑥𝑥(𝑡𝑡) is the location of an automated drone, 𝑢𝑢 𝑡𝑡 is the controller, find what the controller should be such that the vehicle goes to the right place for the least energy.
Neural ODE Approach: Make 𝑢𝑢(𝑡𝑡) be a neural network. Find the neural network s.t. 𝑥𝑥 𝑡𝑡 correctly evolves
18
![Page 19: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/19.jpg)
Neural PDEs for Acceleration of Navier-Stokes
Boussinesq Equations (Navier-Stokes) are used in climate models
People attempt to solve this by “parameterizing”, i.e. getting a 1-dimensional approximation through averaging:
where 𝑤𝑤′𝑐𝑐𝑐 is unknown.
Instead of picking a form for 𝑤𝑤′𝑐𝑐𝑐(the current method), replace it with a neural network and learn it from small scale simulations! Discretize. Result: Neural ODE.
19
![Page 20: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/20.jpg)
Solving 1000 dimensional PDEs: Hamilton-Jacobi-Bellman, Nonlinear Black-Scholes
Semilinear Parabolic Form (Diffusion-Advection Equations, Hamilton-Jacobi-Bellman, Black-Scholes)
Make (𝜎𝜎𝑇𝑇∇u) 𝑡𝑡,𝑋𝑋 a neural network. Solve the resulting SDEs and learn 𝜎𝜎𝑇𝑇∇u
via:
Simplified: Transform it into a Backwards SDE. The unknown is a function! Learn the unknown function via neural
network. Once learned, the PDE solution is known.
Solving high-dimensional partial differential equations using deep learning, 2018, PNAS, Han, Jentzen, E
20
![Page 21: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/21.jpg)
Represent 1000 dimensional PDEs as a 1000 dimensional neural SDE
Solving the PDE = training the neural network As a neural SDE, we can solve with higher order (less neural network
evaluations), adaptivity, etc.
21
Submitting to AISTATS 2020
![Page 22: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/22.jpg)
DiffEqFlux.jl
UnifiedFramework
forScientific ML
The first (mixed) Neural Differential Equation solver. Supports: Neural ODEs Neural SDEs (SDDEs) Neural DAEs Neural DDEs Stiff Equations Hybrid Equations Adjoints via reverse-mode AD and adjoint
sensitivity analysis Data-Efficient and Physical Machine Learning
22
![Page 23: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/23.jpg)
ModelingToolkit: An Open Source Compiler + IR for Models and Transformations
23
![Page 24: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/24.jpg)
Defining Principles
A structured and documented IR for describing models
“Model transforms” are compiler passes IR -> IR
Extendable high level “context” system
Let others write domain-specific languages on top DSLs should not have to define
simplification, differentiation, etc!
24
![Page 25: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/25.jpg)
Automatically convert numerical functions to symbolic
25
![Page 26: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/26.jpg)
Current Research
Components and Pantelides for large DAE systems
An extendable framework for automated PDE discretizations
Methods for (nonlinear) model order reduction
Transformations for SDEs
Tearing and other structural optimizations
26
![Page 27: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/27.jpg)
Now assume the form of the differential equation is “good”CAN WE IMPROVE THE SOLVERS?
27
![Page 28: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/28.jpg)
Non-Stiff Methods are still being improved!
28
![Page 29: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/29.jpg)
The Structure of a Runge-KuttaMethod
29
![Page 30: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/30.jpg)
Ways to Judge an RK MethodOptimization of next order coefficients Stability
30
![Page 31: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/31.jpg)
Dormand-Prince 5th Order (1980) 31
![Page 32: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/32.jpg)
Advancements since 2010
Recent methods, Tsit5 and Vern#, reduce the number
of assumptions made in coefficient optimization, leading to more optimal
solutions (>2010)
Methods specialized for wave equations, low-
dispersion results, extended monotonicity equation for PDEs (SSPRK), etc. are hot topics in new high order
Runge-Kutta methods (2017)
32
![Page 33: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/33.jpg)
100x100 Linear ODEs 33
![Page 34: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/34.jpg)
3-Body Problem (CVODE_Adamsfails)
34
![Page 35: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/35.jpg)
And parallelism is not well exploited!
35
![Page 36: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/36.jpg)
Pervasive Allowance of Within-Method parallelism through Julia
Zero GPU/Distributed message passing done by the solver!
36
![Page 37: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/37.jpg)
Multithreading Extrapolation
SimultaneousEuler steppingof differentstep sizes
37
Ketcheson, 2016
![Page 38: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/38.jpg)
Parallel Runge-Kutta methods
5 stagesBut only 3 steps in parallel
38
![Page 39: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/39.jpg)
DiffEqGPU.jl: Automatic GPU-based parameter parallelism of high level code
Demonstrates a 12x-90x speedup over multithreaded ODE solves by using just 1 GPU.
Handles stiff and non-stiff hybrid ODEs/SDEs/DDEs/DAEs with adaptive timestepping.
Support coming soon: Multiple GPUs Mixed with multithreading/distributed
Result: take existing hybrid ODE simulations written in Julia and automatically GPU+distributed+multithreaded parallelize it for the user!
Requires relatively small systems (<50 DEs?)
39
![Page 40: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/40.jpg)
Stiff ODEs: Fall of the BDFWHAT’S COMING TO GET GEAR’S METHOD.
40
![Page 41: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/41.jpg)
Evolution of Gear’s Method
GEAR: Original code. Adaptive order adaptive time via interpolation Lowers the stability!
LSODE series: update of GEAR Adds rootfinding, Krylov, etc
VODE: Variable-coefficient form No interpolation necessary.
CVODE: VODE rewritten in C++ Adds sensitivity analysis
41
![Page 42: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/42.jpg)
42
![Page 43: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/43.jpg)
Problems with BDF
BDF is a multistep method
Needs “Startup Steps”
Inefficient with events
It is only L-stable up to 2nd order
Has high truncation error coefficients
Implicit
Requires good step predictors
43
![Page 44: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/44.jpg)
Orego Benchmarks 44
![Page 45: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/45.jpg)
Rosenbrock Methods
Aren’t new! (ode23s)
Can fix a lot of problems:
Exploit sparse factorization
No step predictions required
Can optimize coefficients to high order
Con: Needs accurate Jacobians
Answer: AD (or symbolic)
45
![Page 46: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/46.jpg)
ODE Problems can fall into different classes
Physical ModelingSecondOrderODEProblem(f,u0,tspan,p)
𝑢𝑢′′ = 𝑓𝑓(𝑢𝑢, 𝑝𝑝, 𝑡𝑡)
PartitionedODEProblem(f1,f2,v0,u0,tspan,p)
𝑣𝑣′ = 𝑓𝑓1(𝑡𝑡,𝑢𝑢)
𝑢𝑢𝑐 = 𝑓𝑓2(𝑣𝑣)
HamiltonianODEProblem(H,p0,q0,tspan,p)
𝐻𝐻(𝑝𝑝, 𝑞𝑞)
PDE Discretizations
SplitODEProblem(f1,f2,u0,tspan,p) (IMEX)
𝑢𝑢′ = 𝑓𝑓1 𝑢𝑢, 𝑝𝑝, 𝑡𝑡 + 𝑓𝑓2(𝑢𝑢, 𝑝𝑝, 𝑡𝑡)
SemilinearODEProblem(A,f2,u0,tspan,p)
𝑢𝑢′ = 𝐴𝐴𝑢𝑢 + 𝑓𝑓(𝑢𝑢, 𝑝𝑝, 𝑡𝑡)
LocalSemilinearODEProblem(A,f2,u0,tspan,p)𝑢𝑢′ = 𝐴𝐴𝑢𝑢 + 𝑓𝑓. (𝑢𝑢, 𝑝𝑝, 𝑡𝑡)
SDIRK Methods can treat half of the problem as explicit, decreasing the nonlinear solver cost
46
![Page 47: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/47.jpg)
Exponential Runge-Kutta
Explicit methods for stiff equations
Small enough: Build matrix exponential
Large enough: Krylov exp(t*A)*v
Crossover Question: Can we automatically divide equations for IMEX/Multirate methods?
47
![Page 48: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/48.jpg)
Current Results (DiffEqBenchmarks.jl)
<20-30 stiff ODEs = High order Rosenbrock <2000 ODEs = Optimized SDIRK Methods >2000 (general) ODEs = SUNDIALS BDF
Stabilized Explicit methods in PDE contexts
IMEX methods when a split is known
…
Ongoing research to crack the code for more types of systems
48
![Page 49: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/49.jpg)
Putting it together for users: polyalgorithms
49
![Page 50: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/50.jpg)
Conclusion
Today you can solve differential equations Tomorrow you will likely be able to solve them much faster
Neural-embedded methods for simplifying models
A compiler for researching transformations methods in context
Ongoing improvements to numerical methods
50
![Page 51: Recent Advancements in Differential Equation Solver Software€¦ · Adjoint sensitivity analysis. Differentiable programming. Update the neural network and repeat. Automatically](https://reader030.fdocuments.us/reader030/viewer/2022013009/5ea622e81091ff1b347bdeb5/html5/thumbnails/51.jpg)
Students, want a paid summer position? Contact me for Google Summer of Code development. No Julia experience is required. https://julialang.org/soc/ideas-page
51
Industry interested in this research? Contact me to help fund development in JuliaDiffEq! We need industry sponsors/interest for CSSI grants, please let us know