Post on 20-Feb-2021
On the numerical solution of high-dimensionaloptimal control problems: approximate dynamic
programming and Smolyak's algorithm
Markus Fischer
Humboldt University Berlin / University of Heidelberg
Torino, MSF 2008
Approximate DP and Smolyak's algorithm
Introduction
Introduction
Classical control problems and approximate DP
Smolyak's algorithm
A posteriori error bounds
Bibliography
Approximate DP and Smolyak's algorithm
Introduction
Aim and scope
Procedure for numerical solution of high-dimensionalcontinuous-time optimal control problems.
Combination of methods:
I Approximate dynamic programming: dynamic programming indiscrete time with continuous state space using a method forfunction approximation.
I Smolyak's algorithm: method for constructing interpolation (orintegration) operators for multivariate functions.
I Computation of a posteriori error bounds using arepresentation of value functions due to [Rogers, 2007].
Approximate DP and Smolyak's algorithm
Introduction
Aim and scope
Procedure for numerical solution of high-dimensionalcontinuous-time optimal control problems.
Combination of methods:
I Approximate dynamic programming: dynamic programming indiscrete time with continuous state space using a method forfunction approximation.
I Smolyak's algorithm: method for constructing interpolation (orintegration) operators for multivariate functions.
I Computation of a posteriori error bounds using arepresentation of value functions due to [Rogers, 2007].
Approximate DP and Smolyak's algorithm
Introduction
Aim and scope
Procedure for numerical solution of high-dimensionalcontinuous-time optimal control problems.
Combination of methods:
I Approximate dynamic programming: dynamic programming indiscrete time with continuous state space using a method forfunction approximation.
I Smolyak's algorithm: method for constructing interpolation (orintegration) operators for multivariate functions.
I Computation of a posteriori error bounds using arepresentation of value functions due to [Rogers, 2007].
Approximate DP and Smolyak's algorithm
Introduction
Aim and scope
Procedure for numerical solution of high-dimensionalcontinuous-time optimal control problems.
Combination of methods:
I Approximate dynamic programming: dynamic programming indiscrete time with continuous state space using a method forfunction approximation.
I Smolyak's algorithm: method for constructing interpolation (orintegration) operators for multivariate functions.
I Computation of a posteriori error bounds using arepresentation of value functions due to [Rogers, 2007].
Approximate DP and Smolyak's algorithm
Introduction
Aim and scope
Procedure for numerical solution of high-dimensionalcontinuous-time optimal control problems.
Combination of methods:
I Approximate dynamic programming: dynamic programming indiscrete time with continuous state space using a method forfunction approximation.
I Smolyak's algorithm: method for constructing interpolation (orintegration) operators for multivariate functions.
I Computation of a posteriori error bounds using arepresentation of value functions due to [Rogers, 2007].
Apart from classical problems, motivation from the numericalsolution of optimal control problems with delay.
Approximate DP and Smolyak's algorithm
Introduction
Numerical solution of continuous-time optimal controlproblems
Standard approach:
I Replace original problem by a sequence of approximatingcontrol problems,
I construct approximating control problems by discretising timeand state space of the original dynamics and costs,
I solve discrete problems by a backward iteration of dynamicprogramming type.
Approximate DP and Smolyak's algorithm
Introduction
Numerical solution of continuous-time optimal controlproblems
Standard approach:
I Replace original problem by a sequence of approximatingcontrol problems,
I construct approximating control problems by discretising timeand state space of the original dynamics and costs,
I solve discrete problems by a backward iteration of dynamicprogramming type.
Approximate DP and Smolyak's algorithm
Introduction
Numerical solution of continuous-time optimal controlproblems
Standard approach:
I Replace original problem by a sequence of approximatingcontrol problems,
I construct approximating control problems by discretising timeand state space of the original dynamics and costs,
I solve discrete problems by a backward iteration of dynamicprogramming type.
Approximate DP and Smolyak's algorithm
Introduction
Numerical solution of continuous-time optimal controlproblems
Standard approach:
I Replace original problem by a sequence of approximatingcontrol problems,
I construct approximating control problems by discretising timeand state space of the original dynamics and costs,
I solve discrete problems by a backward iteration of dynamicprogramming type.
Last step computationally di�cult when the state space ishigh-dimensional, where �high� means dimensions greater thanthree or four � curse of dimensionality.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Introduction
Classical control problems and approximate DP
Smolyak's algorithm
A posteriori error bounds
Bibliography
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Classical stochastic optimal control problemDynamics given by a controlled SDE:
dX (t) = b (t,X (t), u(t)) dt + σ (t,X (t), u(t)) dW (t), t ≥ t0,
with initial condition X (t0) = x ∈ Rd , where W (.) is a d1-dimensionalstandard Wiener process, u(.) ∈ U a control process with values in aspace of control actions Γ, and b : [0,∞)× Rd × Γ → Rd ,σ : [0,∞)× Rd × Γ → Rd×d1 are the drift and di�usion coe�cient.
Cost functional over a �nite time horizon T :
J(t0, x ; u(.)) := E
[∫ Tt0
f (t,X (t), u(t)) dt + g (X (T ))
].
Corresponding value function:
V (t0, x) := infu(.)∈U
J(t0, x ; u(.)), t0 ∈ [0,T ], x ∈ Rd .
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Classical stochastic optimal control problemDynamics given by a controlled SDE:
dX (t) = b (t,X (t), u(t)) dt + σ (t,X (t), u(t)) dW (t), t ≥ t0,
with initial condition X (t0) = x ∈ Rd , where W (.) is a d1-dimensionalstandard Wiener process, u(.) ∈ U a control process with values in aspace of control actions Γ, and b : [0,∞)× Rd × Γ → Rd ,σ : [0,∞)× Rd × Γ → Rd×d1 are the drift and di�usion coe�cient.
Cost functional over a �nite time horizon T :
J(t0, x ; u(.)) := E
[∫ Tt0
f (t,X (t), u(t)) dt + g (X (T ))
].
Corresponding value function:
V (t0, x) := infu(.)∈U
J(t0, x ; u(.)), t0 ∈ [0,T ], x ∈ Rd .
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Classical stochastic optimal control problemDynamics given by a controlled SDE:
dX (t) = b (t,X (t), u(t)) dt + σ (t,X (t), u(t)) dW (t), t ≥ t0,
with initial condition X (t0) = x ∈ Rd , where W (.) is a d1-dimensionalstandard Wiener process, u(.) ∈ U a control process with values in aspace of control actions Γ, and b : [0,∞)× Rd × Γ → Rd ,σ : [0,∞)× Rd × Γ → Rd×d1 are the drift and di�usion coe�cient.
Cost functional over a �nite time horizon T :
J(t0, x ; u(.)) := E
[∫ Tt0
f (t,X (t), u(t)) dt + g (X (T ))
].
Corresponding value function:
V (t0, x) := infu(.)∈U
J(t0, x ; u(.)), t0 ∈ [0,T ], x ∈ Rd .
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Classical control problem and time discretisation
Regularity assumptions on b, σ, f , g : 12-Hölder in time, Lipschitz in the
space variable, continuous and bounded.
Then V is bounded, 12-Hölder
continuous in time, Lipschitz continuous in space, but not necessarilydi�erentiable.
Euler discretisation of dynamics and costs: time step h := T/N,piecewise constant Γ-valued control processes ū ∈ Ū . Costs given by
J̄(n h, x , ū) := E
N−1∑j=n
fj(X̄ (j), ū(j)
)+ g
(X̄ (N)
) ∣∣∣X̄ (n) = x ,
where fj(x , γ) := h · f (j h, x , γ), X̄ controlled Markov chain withtransition probabilities µj(x , γ) d -variate normal distributions with meanx + h · b(j h, x , γ) and covariance h · (σσT)(j h, x , γ).
Error bounds: [Krylov, 1999] and more recent works.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Classical control problem and time discretisation
Regularity assumptions on b, σ, f , g : 12-Hölder in time, Lipschitz in the
space variable, continuous and bounded. Then V is bounded, 12-Hölder
continuous in time, Lipschitz continuous in space, but not necessarilydi�erentiable.
Euler discretisation of dynamics and costs: time step h := T/N,piecewise constant Γ-valued control processes ū ∈ Ū . Costs given by
J̄(n h, x , ū) := E
N−1∑j=n
fj(X̄ (j), ū(j)
)+ g
(X̄ (N)
) ∣∣∣X̄ (n) = x ,
where fj(x , γ) := h · f (j h, x , γ), X̄ controlled Markov chain withtransition probabilities µj(x , γ) d -variate normal distributions with meanx + h · b(j h, x , γ) and covariance h · (σσT)(j h, x , γ).
Error bounds: [Krylov, 1999] and more recent works.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Classical control problem and time discretisation
Regularity assumptions on b, σ, f , g : 12-Hölder in time, Lipschitz in the
space variable, continuous and bounded. Then V is bounded, 12-Hölder
continuous in time, Lipschitz continuous in space, but not necessarilydi�erentiable.
Euler discretisation of dynamics and costs: time step h := T/N,piecewise constant Γ-valued control processes ū ∈ Ū .
Costs given by
J̄(n h, x , ū) := E
N−1∑j=n
fj(X̄ (j), ū(j)
)+ g
(X̄ (N)
) ∣∣∣X̄ (n) = x ,
where fj(x , γ) := h · f (j h, x , γ), X̄ controlled Markov chain withtransition probabilities µj(x , γ) d -variate normal distributions with meanx + h · b(j h, x , γ) and covariance h · (σσT)(j h, x , γ).
Error bounds: [Krylov, 1999] and more recent works.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Classical control problem and time discretisation
Regularity assumptions on b, σ, f , g : 12-Hölder in time, Lipschitz in the
space variable, continuous and bounded. Then V is bounded, 12-Hölder
continuous in time, Lipschitz continuous in space, but not necessarilydi�erentiable.
Euler discretisation of dynamics and costs: time step h := T/N,piecewise constant Γ-valued control processes ū ∈ Ū . Costs given by
J̄(n h, x , ū) := E
N−1∑j=n
fj(X̄ (j), ū(j)
)+ g
(X̄ (N)
) ∣∣∣X̄ (n) = x ,
where fj(x , γ) := h · f (j h, x , γ), X̄ controlled Markov chain withtransition probabilities µj(x , γ) d -variate normal distributions with meanx + h · b(j h, x , γ) and covariance h · (σσT)(j h, x , γ).
Error bounds: [Krylov, 1999] and more recent works.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Classical control problem and time discretisation
Regularity assumptions on b, σ, f , g : 12-Hölder in time, Lipschitz in the
space variable, continuous and bounded. Then V is bounded, 12-Hölder
continuous in time, Lipschitz continuous in space, but not necessarilydi�erentiable.
Euler discretisation of dynamics and costs: time step h := T/N,piecewise constant Γ-valued control processes ū ∈ Ū . Costs given by
J̄(n h, x , ū) := E
N−1∑j=n
fj(X̄ (j), ū(j)
)+ g
(X̄ (N)
) ∣∣∣X̄ (n) = x ,
where fj(x , γ) := h · f (j h, x , γ), X̄ controlled Markov chain withtransition probabilities µj(x , γ) d -variate normal distributions with meanx + h · b(j h, x , γ) and covariance h · (σσT)(j h, x , γ).
Error bounds: [Krylov, 1999] and more recent works.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Approximate dynamic programming
One-step Bellman operators Tn : B(Rd) → B(Rd), n ∈ {0, . . . ,N−1}:
Tn(v)(x) := infγ∈Γ
{fn(x , γ) +
∫Rd
v(y) µn(x , γ)(dy)
}
Recall that Tn is non-expansive under the supremum norm, that is,‖Tn(v)− Tn(w)‖∞ ≤ ‖v − w‖∞ for all v ,w ∈ B(Rd). Moreover, Tnpreserves Lipschitz continuity.
According to Bellman's principle: V̄ := V̄ (0, .) = T0 ◦ . . . ◦ TN−1(g).
Approximate DP: Let A : B(Rd) → B(Rd) be linear. SetṼ := A ◦ T0 ◦ . . . ◦ A ◦ TN−1(g).
Then‖V̄ − Ṽ ‖∞ ≤ N · sup
w∈F‖A(w)− w‖∞,
where F contains {Tj−1 ◦ A ◦ Tj ◦ . . . ◦ A ◦ TN−1(g) | j ∈ {1, . . . ,N}}.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Approximate dynamic programming
One-step Bellman operators Tn : B(Rd) → B(Rd), n ∈ {0, . . . ,N−1}:
Tn(v)(x) := infγ∈Γ
{fn(x , γ) +
∫Rd
v(y) µn(x , γ)(dy)
}Recall that Tn is non-expansive under the supremum norm, that is,‖Tn(v)− Tn(w)‖∞ ≤ ‖v − w‖∞ for all v ,w ∈ B(Rd).
Moreover, Tnpreserves Lipschitz continuity.
According to Bellman's principle: V̄ := V̄ (0, .) = T0 ◦ . . . ◦ TN−1(g).
Approximate DP: Let A : B(Rd) → B(Rd) be linear. SetṼ := A ◦ T0 ◦ . . . ◦ A ◦ TN−1(g).
Then‖V̄ − Ṽ ‖∞ ≤ N · sup
w∈F‖A(w)− w‖∞,
where F contains {Tj−1 ◦ A ◦ Tj ◦ . . . ◦ A ◦ TN−1(g) | j ∈ {1, . . . ,N}}.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Approximate dynamic programming
One-step Bellman operators Tn : B(Rd) → B(Rd), n ∈ {0, . . . ,N−1}:
Tn(v)(x) := infγ∈Γ
{fn(x , γ) +
∫Rd
v(y) µn(x , γ)(dy)
}Recall that Tn is non-expansive under the supremum norm, that is,‖Tn(v)− Tn(w)‖∞ ≤ ‖v − w‖∞ for all v ,w ∈ B(Rd). Moreover, Tnpreserves Lipschitz continuity.
According to Bellman's principle: V̄ := V̄ (0, .) = T0 ◦ . . . ◦ TN−1(g).
Approximate DP: Let A : B(Rd) → B(Rd) be linear. SetṼ := A ◦ T0 ◦ . . . ◦ A ◦ TN−1(g).
Then‖V̄ − Ṽ ‖∞ ≤ N · sup
w∈F‖A(w)− w‖∞,
where F contains {Tj−1 ◦ A ◦ Tj ◦ . . . ◦ A ◦ TN−1(g) | j ∈ {1, . . . ,N}}.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Approximate dynamic programming
One-step Bellman operators Tn : B(Rd) → B(Rd), n ∈ {0, . . . ,N−1}:
Tn(v)(x) := infγ∈Γ
{fn(x , γ) +
∫Rd
v(y) µn(x , γ)(dy)
}Recall that Tn is non-expansive under the supremum norm, that is,‖Tn(v)− Tn(w)‖∞ ≤ ‖v − w‖∞ for all v ,w ∈ B(Rd). Moreover, Tnpreserves Lipschitz continuity.
According to Bellman's principle: V̄ := V̄ (0, .) = T0 ◦ . . . ◦ TN−1(g).
Approximate DP: Let A : B(Rd) → B(Rd) be linear. SetṼ := A ◦ T0 ◦ . . . ◦ A ◦ TN−1(g).
Then‖V̄ − Ṽ ‖∞ ≤ N · sup
w∈F‖A(w)− w‖∞,
where F contains {Tj−1 ◦ A ◦ Tj ◦ . . . ◦ A ◦ TN−1(g) | j ∈ {1, . . . ,N}}.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Approximate dynamic programming
One-step Bellman operators Tn : B(Rd) → B(Rd), n ∈ {0, . . . ,N−1}:
Tn(v)(x) := infγ∈Γ
{fn(x , γ) +
∫Rd
v(y) µn(x , γ)(dy)
}Recall that Tn is non-expansive under the supremum norm, that is,‖Tn(v)− Tn(w)‖∞ ≤ ‖v − w‖∞ for all v ,w ∈ B(Rd). Moreover, Tnpreserves Lipschitz continuity.
According to Bellman's principle: V̄ := V̄ (0, .) = T0 ◦ . . . ◦ TN−1(g).
Approximate DP: Let A : B(Rd) → B(Rd) be linear. SetṼ := A ◦ T0 ◦ . . . ◦ A ◦ TN−1(g).
Then‖V̄ − Ṽ ‖∞ ≤ N · sup
w∈F‖A(w)− w‖∞,
where F contains {Tj−1 ◦ A ◦ Tj ◦ . . . ◦ A ◦ TN−1(g) | j ∈ {1, . . . ,N}}.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Approximate dynamic programming
One-step Bellman operators Tn : B(Rd) → B(Rd), n ∈ {0, . . . ,N−1}:
Tn(v)(x) := infγ∈Γ
{fn(x , γ) +
∫Rd
v(y) µn(x , γ)(dy)
}Recall that Tn is non-expansive under the supremum norm, that is,‖Tn(v)− Tn(w)‖∞ ≤ ‖v − w‖∞ for all v ,w ∈ B(Rd). Moreover, Tnpreserves Lipschitz continuity.
According to Bellman's principle: V̄ := V̄ (0, .) = T0 ◦ . . . ◦ TN−1(g).
Approximate DP: Let A : B(Rd) → B(Rd) be linear. SetṼ := A ◦ T0 ◦ . . . ◦ A ◦ TN−1(g).
Then‖V̄ − Ṽ ‖∞ ≤ N · sup
w∈F‖A(w)− w‖∞,
where F contains {Tj−1 ◦ A ◦ Tj ◦ . . . ◦ A ◦ TN−1(g) | j ∈ {1, . . . ,N}}.
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Approximate dynamic programming II
Approximation error measured in supremum norm; A in this waycompatible with Bellman operators. More complicated for operatorswhich yield good approximations only in Lp-norm [Munos, 2007].
Truncation of state space: from now on, work with [0, 1]d (or[a, b]d ) instead of Rd .
Instead of one approximation operator A, family of operatorsA(q, d), q ≥ d , such that
supw∈F
‖A(q, d)(w)− w‖∞q→∞→ 0
fast enough for suitable function classes F . Use Smolyak'salgorithm for constructing the A(q, d).
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Approximate dynamic programming II
Approximation error measured in supremum norm; A in this waycompatible with Bellman operators. More complicated for operatorswhich yield good approximations only in Lp-norm [Munos, 2007].
Truncation of state space: from now on, work with [0, 1]d (or[a, b]d ) instead of Rd .
Instead of one approximation operator A, family of operatorsA(q, d), q ≥ d , such that
supw∈F
‖A(q, d)(w)− w‖∞q→∞→ 0
fast enough for suitable function classes F . Use Smolyak'salgorithm for constructing the A(q, d).
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
Approximate dynamic programming II
Approximation error measured in supremum norm; A in this waycompatible with Bellman operators. More complicated for operatorswhich yield good approximations only in Lp-norm [Munos, 2007].
Truncation of state space: from now on, work with [0, 1]d (or[a, b]d ) instead of Rd .
Instead of one approximation operator A, family of operatorsA(q, d), q ≥ d , such that
supw∈F
‖A(q, d)(w)− w‖∞q→∞→ 0
fast enough for suitable function classes F . Use Smolyak'salgorithm for constructing the A(q, d).
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
The curse of dimensionality for function approximation
Lipschitz continuous functions:Let Fd := {f : [0, 1]d → R | |f (x)− f (y)| ≤ max |xi − yi |}. Then
errApp(Fd , n) := infΨ=Ψ(n)
supf ∈Fd
‖Ψ(f )− f ‖∞ ≈1
2n−1/d .
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
The curse of dimensionality for function approximation
Lipschitz continuous functions: � Curse!Let Fd := {f : [0, 1]d → R | |f (x)− f (y)| ≤ max |xi − yi |}. Then
errApp(Fd , n) := infΨ=Ψ(n)
supf ∈Fd
‖Ψ(f )− f ‖∞ ≈1
2n−1/d .
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
The curse of dimensionality for function approximation
Lipschitz continuous functions: � Curse!Let Fd := {f : [0, 1]d → R | |f (x)− f (y)| ≤ max |xi − yi |}. Then
errApp(Fd , n) := infΨ=Ψ(n)
supf ∈Fd
‖Ψ(f )− f ‖∞ ≈1
2n−1/d .
Hölder classes of functions:Let k ∈ N0, β ∈ (0, 1]. Set
Ck,βd := {f : [0, 1]
d → R | |Dαf (x)− Dαf (y)| ≤ max |xi − yi |β , |α| = k}.
Then there are constants 0 < cd < Cd < ∞ such that
cd · n−(k+α)/d ≤ errApp(Ck,βd , n) ≤ Cd · n−(k+α)/d .
Approximate DP and Smolyak's algorithm
Classical control problems and approximate DP
The curse of dimensionality for function approximation
Lipschitz continuous functions: � Curse!Let Fd := {f : [0, 1]d → R | |f (x)− f (y)| ≤ max |xi − yi |}. Then
errApp(Fd , n) := infΨ=Ψ(n)
supf ∈Fd
‖Ψ(f )− f ‖∞ ≈1
2n−1/d .
Hölder classes of functions: � curse unless k ∼ dLet k ∈ N0, β ∈ (0, 1]. Set
Ck,βd := {f : [0, 1]
d → R | |Dαf (x)− Dαf (y)| ≤ max |xi − yi |β , |α| = k}.
Then there are constants 0 < cd < Cd < ∞ such that
cd · n−(k+α)/d ≤ errApp(Ck,βd , n) ≤ Cd · n−(k+α)/d .
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Introduction
Classical control problems and approximate DP
Smolyak's algorithm
A posteriori error bounds
Bibliography
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Smolyak's algorithm
Method for constructing operators for the approximation (orintegration) of d -variate functions using certain tensor products ofunivariate operators [Smolyak, 1963]:
Let U i , i ∈ N, be a sequence of continuous linear operators. Set∆i := U i − U i−1, U0 := 0.
Smolyak's algorithm in dimension d of degree q−d is de�ned asthe operator
A(q, d) :=∑
~i∈Nd ,|~i |≤q
(∆i1 ⊗ . . .⊗∆id
), q ≥ d ,
where |~i | := i1 + . . . + id for ~i ∈ Nd .
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Smolyak's algorithm
Method for constructing operators for the approximation (orintegration) of d -variate functions using certain tensor products ofunivariate operators [Smolyak, 1963]:
Let U i , i ∈ N, be a sequence of continuous linear operators. Set∆i := U i − U i−1, U0 := 0.
Smolyak's algorithm in dimension d of degree q−d is de�ned asthe operator
A(q, d) :=∑
~i∈Nd ,|~i |≤q
(∆i1 ⊗ . . .⊗∆id
), q ≥ d ,
where |~i | := i1 + . . . + id for ~i ∈ Nd .
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Smolyak's algorithm II
Dimension one: A(q, 1) =∑q
j=1 ∆j = Uq, q ≥ 1.
Dimension two:
A(q, 2) =q−1∑j1=1
∑j2=1
∆j1 ⊗∆j2 , q ≥ 2,
while
Uq−1 ⊗ Uq−1 =q−1∑j1=1
q−1∑j2=1
∆j1 ⊗∆j2 .
Hierarchical structure:
A(q, d) = A(q−1, d) +∑
~i∈Nd ,|~i |=q
(∆i1 ⊗ . . .⊗∆id
), q ≥ d .
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Smolyak's algorithm II
Dimension one: A(q, 1) =∑q
j=1 ∆j = Uq, q ≥ 1.
Dimension two:
A(q, 2) =q−1∑j1=1
q−j1∑j2=1
∆j1 ⊗∆j2 , q ≥ 2,
while
Uq−1 ⊗ Uq−1 =q−1∑j1=1
q−1∑j2=1
∆j1 ⊗∆j2 .
Hierarchical structure:
A(q, d) = A(q−1, d) +∑
~i∈Nd ,|~i |=q
(∆i1 ⊗ . . .⊗∆id
), q ≥ d .
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Smolyak's algorithm II
Dimension one: A(q, 1) =∑q
j=1 ∆j = Uq, q ≥ 1.
Dimension two:
A(q, 2) =q−1∑j1=1
q−j1∑j2=1
∆j1 ⊗∆j2 , q ≥ 2,
while
Uq−1 ⊗ Uq−1 =q−1∑j1=1
q−1∑j2=1
∆j1 ⊗∆j2 .
Hierarchical structure:
A(q, d) = A(q−1, d) +∑
~i∈Nd ,|~i |=q
(∆i1 ⊗ . . .⊗∆id
), q ≥ d .
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Smolyak's algorithm II
Dimension one: A(q, 1) =∑q
j=1 ∆j = Uq, q ≥ 1.
Dimension two:
A(q, 2) =q−1∑j1=1
q−j1∑j2=1
∆j1 ⊗∆j2 , q ≥ 2,
while
Uq−1 ⊗ Uq−1 =q−1∑j1=1
q−1∑j2=1
∆j1 ⊗∆j2 .
Hierarchical structure:
A(q, d) = A(q−1, d) +∑
~i∈Nd ,|~i |=q
(∆i1 ⊗ . . .⊗∆id
), q ≥ d .
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Grid-based interpolation
Let U i , i ∈ N, be operators on univariate functions of the form
U i (f ) =mi∑j=1
f (x ij ) aij ,
x ij ∈ X i ⊂ [0, 1] grid points, aij basis functions, j ∈ {1, . . . ,mi}.
Then A(q, d) is determined by function values on the sparse grid
H(q, d) :=⋃
~i∈Nd ,|~i |≤q
X i1 × . . .× X id , q ≥ d .
Assumption: nested grids, i. e. X i−1 ⊂ X i for all i ∈ N. Then
H(q, d) =⋃|~i |=q
X i1 × . . .× X id =⋃|~i |≤q
∆X i1 × . . .×∆X id ,
where ∆X i := X i \ X i−1, X 0 := ∅.
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Grid-based interpolation
Let U i , i ∈ N, be operators on univariate functions of the form
U i (f ) =mi∑j=1
f (x ij ) aij ,
x ij ∈ X i ⊂ [0, 1] grid points, aij basis functions, j ∈ {1, . . . ,mi}.Then A(q, d) is determined by function values on the sparse grid
H(q, d) :=⋃
~i∈Nd ,|~i |≤q
X i1 × . . .× X id , q ≥ d .
Assumption: nested grids, i. e. X i−1 ⊂ X i for all i ∈ N. Then
H(q, d) =⋃|~i |=q
X i1 × . . .× X id =⋃|~i |≤q
∆X i1 × . . .×∆X id ,
where ∆X i := X i \ X i−1, X 0 := ∅.
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Grid-based interpolation
Let U i , i ∈ N, be operators on univariate functions of the form
U i (f ) =mi∑j=1
f (x ij ) aij ,
x ij ∈ X i ⊂ [0, 1] grid points, aij basis functions, j ∈ {1, . . . ,mi}.Then A(q, d) is determined by function values on the sparse grid
H(q, d) :=⋃
~i∈Nd ,|~i |≤q
X i1 × . . .× X id , q ≥ d .
Assumption: nested grids, i. e. X i−1 ⊂ X i for all i ∈ N. Then
H(q, d) =⋃|~i |=q
X i1 × . . .× X id =⋃|~i |≤q
∆X i1 × . . .×∆X id ,
where ∆X i := X i \ X i−1, X 0 := ∅.
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Grid-based interpolation
Let U i , i ∈ N, be operators on univariate functions of the form
U i (f ) =mi∑j=1
f (x ij ) aij ,
x ij ∈ X i ⊂ [0, 1] grid points, aij basis functions, j ∈ {1, . . . ,mi}.Then A(q, d) is determined by function values on the sparse grid
H(q, d) :=⋃
~i∈Nd ,|~i |≤q
X i1 × . . .× X id , q ≥ d .
Assumption: nested grids, i. e. X i−1 ⊂ X i for all i ∈ N. Then
H(q, d) =⋃|~i |=q
X i1 × . . .× X id =⋃|~i |≤q
∆X i1 × . . .×∆X id ,
where ∆X i := X i \ X i−1, X 0 := ∅.
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Piecewise multilinear interpolation on sparse grids
Uniform one-dimensional grids: x11 := 1/2,
x ij := (j−1)/(mi−1) for i > 1, j ∈ {1, . . . ,mi},
where m1 := 1, mi := 2i−1 + 1 if i > 1 (Clenshaw-Curtis).
Piecewise linear univariate basis functions: a11 ≡ 1,
aij(t) :=(1− (mi−1) · |t−x ij |
)· 1[x i
j−1/(mi−1),x ij +1/(mi−1)]
(t)
t ∈ [0, 1], i > 1, j ∈ {1, . . . ,mi}.
Denote by A(q, d) the Smolyak algorithm in dimension d of degreeq−d based on piecewise linear univariate interpolation operators.Denote by H(q, d) the corresponding sparse grid.
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Piecewise multilinear interpolation on sparse grids
Uniform one-dimensional grids: x11 := 1/2,
x ij := (j−1)/(mi−1) for i > 1, j ∈ {1, . . . ,mi},
where m1 := 1, mi := 2i−1 + 1 if i > 1 (Clenshaw-Curtis).
Piecewise linear univariate basis functions: a11 ≡ 1,
aij(t) :=(1− (mi−1) · |t−x ij |
)· 1[x i
j−1/(mi−1),x ij +1/(mi−1)]
(t)
t ∈ [0, 1], i > 1, j ∈ {1, . . . ,mi}.
Denote by A(q, d) the Smolyak algorithm in dimension d of degreeq−d based on piecewise linear univariate interpolation operators.Denote by H(q, d) the corresponding sparse grid.
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Piecewise multilinear interpolation on sparse grids
Uniform one-dimensional grids: x11 := 1/2,
x ij := (j−1)/(mi−1) for i > 1, j ∈ {1, . . . ,mi},
where m1 := 1, mi := 2i−1 + 1 if i > 1 (Clenshaw-Curtis).
Piecewise linear univariate basis functions: a11 ≡ 1,
aij(t) :=(1− (mi−1) · |t−x ij |
)· 1[x i
j−1/(mi−1),x ij +1/(mi−1)]
(t)
t ∈ [0, 1], i > 1, j ∈ {1, . . . ,mi}.
Denote by A(q, d) the Smolyak algorithm in dimension d of degreeq−d based on piecewise linear univariate interpolation operators.Denote by H(q, d) the corresponding sparse grid.
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Sparse grids from one-dimensional uniform grids
H(2 + 5, 2) sparse grid of degree 5, levels from 0 to 5:
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Number of points of the sparse grids
Denote by n(q, d) the number of points of the sparse grid H(q, d),q ≥ d . Then (e. g. [Schreiber, 2000])
max
{(q
d
), 2q−2d
}≤ n(q, d) ≤ 2
−(d−1)
(d − 1)!· 2q · qd−1.
In particular, q ≤ 2d + log2(n(q, d)).
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Number of points of the sparse grids
Denote by n(q, d) the number of points of the sparse grid H(q, d),q ≥ d . Then (e. g. [Schreiber, 2000])
max
{(q
d
), 2q−2d
}≤ n(q, d) ≤ 2
−(d−1)
(d − 1)!· 2q · qd−1.
In particular, q ≤ 2d + log2(n(q, d)).
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Hierarchical construction of the interpolant
Let q > d .
A(q, d)(f ) = A(q−1, d)(f ) +∑|~i |=q
(∆i1 ⊗ . . .⊗∆id
)(f )
= A(q−1, d)(f ) +∑|~i |=q
∑~j
β~i~j· ai1j1 ⊗ . . .⊗ a
idjd
,
where ~j ranges over ~j = (j1, . . . , jd ), 1 ≤ jl ≤ #∆X il , and
β~i~j
:= f (x i1j1 , . . . , xidjd
)− A(q−1, d)(f )(x i1j1 , . . . , xidjd
).
Of course,
ai1j1 ⊗ . . .⊗ aidjd
(x) = ai1j1(x1) · . . . · aidjd
(xd ), x = (x1, . . . , xd ) ∈ Rd .
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Approximation error
The space of functions with bounded mixed partial derivatives up toorder k on [0, 1]d is given by
F kd := {f : [0, 1]d → R | ‖f ‖Fkd
< ∞},
where ‖f ‖Fkd
:= maxα∈Nd0,|α|∞≤k‖D
αf ‖∞.
Important properties: ‖.‖Fkd
is a tensor norm, ‖.‖Fk1
induces a tensor
norm on F k1⊗ . . .⊗ F k
1such that cl(F k
1⊗ . . .⊗ F k
1) ' F kd .
Let k ∈ {1, 2}. Then there is a constant c(d) such that
‖A(q, d)(f )− f ‖∞ ≤ c(d) · ‖f ‖Fkd
· n−k · (log2(n))(k+1)(d−1)
for all q > d , where n = n(q, d) is the number of grid points;
[Smolyak, 1963], [Barthelmann et al., 2000], [Schreiber, 2000].
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Approximation error
The space of functions with bounded mixed partial derivatives up toorder k on [0, 1]d is given by
F kd := {f : [0, 1]d → R | ‖f ‖Fkd
< ∞},
where ‖f ‖Fkd
:= maxα∈Nd0,|α|∞≤k‖D
αf ‖∞.
Important properties: ‖.‖Fkd
is a tensor norm, ‖.‖Fk1
induces a tensor
norm on F k1⊗ . . .⊗ F k
1such that cl(F k
1⊗ . . .⊗ F k
1) ' F kd .
Let k ∈ {1, 2}. Then there is a constant c(d) such that
‖A(q, d)(f )− f ‖∞ ≤ c(d) · ‖f ‖Fkd
· n−k · (log2(n))(k+1)(d−1)
for all q > d , where n = n(q, d) is the number of grid points;
[Smolyak, 1963], [Barthelmann et al., 2000], [Schreiber, 2000].
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Approximation error
The space of functions with bounded mixed partial derivatives up toorder k on [0, 1]d is given by
F kd := {f : [0, 1]d → R | ‖f ‖Fkd
< ∞},
where ‖f ‖Fkd
:= maxα∈Nd0,|α|∞≤k‖D
αf ‖∞.
Important properties: ‖.‖Fkd
is a tensor norm, ‖.‖Fk1
induces a tensor
norm on F k1⊗ . . .⊗ F k
1such that cl(F k
1⊗ . . .⊗ F k
1) ' F kd .
Let k ∈ {1, 2}. Then there is a constant c(d) such that
‖A(q, d)(f )− f ‖∞ ≤ c(d) · ‖f ‖Fkd
· n−k · (log2(n))(k+1)(d−1)
for all q > d , where n = n(q, d) is the number of grid points;
[Smolyak, 1963], [Barthelmann et al., 2000], [Schreiber, 2000].
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Introduction
Classical control problems and approximate DP
Smolyak's algorithm
A posteriori error bounds
Bibliography
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Approximate DP and Smolyak's algorithmLet
Fg :=⋃q≥d
{Tj−1 ◦ A(q, d) ◦ Tj ◦ . . . ◦ A(q, d) ◦ TN−1(g) | j = 1, . . . ,N} .
Compute Ṽ = Ṽ (q) by approximate DP using A(q, d). Then
‖V̄ − Ṽ ‖∞ ≤ N · supw∈Fg
‖A(q, d)(w)− w‖∞.
If we had Fg ⊂ F 1d , then, with n = n(q, d), it would follow that
‖V̄ − Ṽ ‖∞ ≤ N · c(d) · supw∈Fg
‖w‖F 1d· n−1 · (log2(n))2(d−1).
However, functions in Fg are guaranteed only to be Lipschitz.Therefore, a priori worst-case error bound of order n−1/d times alogarithmic factor � curse of dimensionality again!
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Approximate DP and Smolyak's algorithmLet
Fg :=⋃q≥d
{Tj−1 ◦ A(q, d) ◦ Tj ◦ . . . ◦ A(q, d) ◦ TN−1(g) | j = 1, . . . ,N} .
Compute Ṽ = Ṽ (q) by approximate DP using A(q, d). Then
‖V̄ − Ṽ ‖∞ ≤ N · supw∈Fg
‖A(q, d)(w)− w‖∞.
If we had Fg ⊂ F 1d , then, with n = n(q, d), it would follow that
‖V̄ − Ṽ ‖∞ ≤ N · c(d) · supw∈Fg
‖w‖F 1d· n−1 · (log2(n))2(d−1).
However, functions in Fg are guaranteed only to be Lipschitz.Therefore, a priori worst-case error bound of order n−1/d times alogarithmic factor � curse of dimensionality again!
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Approximate DP and Smolyak's algorithmLet
Fg :=⋃q≥d
{Tj−1 ◦ A(q, d) ◦ Tj ◦ . . . ◦ A(q, d) ◦ TN−1(g) | j = 1, . . . ,N} .
Compute Ṽ = Ṽ (q) by approximate DP using A(q, d). Then
‖V̄ − Ṽ ‖∞ ≤ N · supw∈Fg
‖A(q, d)(w)− w‖∞.
If we had Fg ⊂ F 1d , then, with n = n(q, d), it would follow that
‖V̄ − Ṽ ‖∞ ≤ N · c(d) · supw∈Fg
‖w‖F 1d· n−1 · (log2(n))2(d−1).
However, functions in Fg are guaranteed only to be Lipschitz.Therefore, a priori worst-case error bound of order n−1/d times alogarithmic factor � curse of dimensionality again!
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Upper and lower bound for �xed initial condition
Apply approximate DP with Smolyak's algorithm A(q, d) tocompute Ṽ (j , .), j ∈ {0, . . . ,N}, in particular Ṽ = Ṽ (0, .). Then Ṽis a candidate for V̄ , the value function at time zero of thesemi-discrete problem.
Let x̂ ∈ Rd be any initial state of interest. Use Ṽ to compute anupper as well as a lower bound on V̄ (x̂), the unknown true value.
Upper and lower values by Monte Carlo simulation.
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Upper and lower bound for �xed initial condition
Apply approximate DP with Smolyak's algorithm A(q, d) tocompute Ṽ (j , .), j ∈ {0, . . . ,N}, in particular Ṽ = Ṽ (0, .). Then Ṽis a candidate for V̄ , the value function at time zero of thesemi-discrete problem.
Let x̂ ∈ Rd be any initial state of interest. Use Ṽ to compute anupper as well as a lower bound on V̄ (x̂), the unknown true value.
Upper and lower values by Monte Carlo simulation.
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Upper and lower bound for �xed initial condition
Apply approximate DP with Smolyak's algorithm A(q, d) tocompute Ṽ (j , .), j ∈ {0, . . . ,N}, in particular Ṽ = Ṽ (0, .). Then Ṽis a candidate for V̄ , the value function at time zero of thesemi-discrete problem.
Let x̂ ∈ Rd be any initial state of interest. Use Ṽ to compute anupper as well as a lower bound on V̄ (x̂), the unknown true value.
Upper and lower values by Monte Carlo simulation.
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Upper bound: feedback control
Assumption: Γ is compact.
Compute, by Monte Carlo forward simulation of the underlyingMarkov chain with X̄ (0) = x̂ ,
J̄(0, x̂ , ũ) = E
N−1∑j=n
fj(X̄ (j), ũ(j , X̄ (j))
)+ g
(X̄ (N)
) ,where ũ is the feedback control obtained from Ṽ as
ũ(j , x) := argminγ∈Γ
{fj(x , γ) +
∫Rd
Ṽ (y) µj(x , γ)(dy)
}.
Evaluation of ũ only at points of the simulated trajectories.
Then, by de�nition, V̄ (x̂) ≤ J̄(0, x̂ , ũ).
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Upper bound: feedback control
Assumption: Γ is compact.
Compute, by Monte Carlo forward simulation of the underlyingMarkov chain with X̄ (0) = x̂ ,
J̄(0, x̂ , ũ) = E
N−1∑j=n
fj(X̄ (j), ũ(j , X̄ (j))
)+ g
(X̄ (N)
) ,where ũ is the feedback control obtained from Ṽ as
ũ(j , x) := argminγ∈Γ
{fj(x , γ) +
∫Rd
Ṽ (y) µj(x , γ)(dy)
}.
Evaluation of ũ only at points of the simulated trajectories.
Then, by de�nition, V̄ (x̂) ≤ J̄(0, x̂ , ũ).
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Upper bound: feedback control
Assumption: Γ is compact.
Compute, by Monte Carlo forward simulation of the underlyingMarkov chain with X̄ (0) = x̂ ,
J̄(0, x̂ , ũ) = E
N−1∑j=n
fj(X̄ (j), ũ(j , X̄ (j))
)+ g
(X̄ (N)
) ,where ũ is the feedback control obtained from Ṽ as
ũ(j , x) := argminγ∈Γ
{fj(x , γ) +
∫Rd
Ṽ (y) µj(x , γ)(dy)
}.
Evaluation of ũ only at points of the simulated trajectories.
Then, by de�nition, V̄ (x̂) ≤ J̄(0, x̂ , ũ).
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Lower bound: representation due to [Rogers, 2007]
Assumption: there is a reference probability measure on Rd such that thetransition probabilities µn(x , γ) possess densities pn(x , ., γ) and there is areference density p∗n(x , .). Assumption satis�ed if di�usion coe�cient σ isuniformly elliptic.
Set φn(x , y , γ) :=pn(x,y ,γ)p∗n (x,y)
. Then, with X̄ (0) = x̂ , X̄ under P∗,
V̄ (x̂) = maxvE∗[
inf(γ0,...,γN−1)
{ N−1∏k=0
φk(X̄ (k), X̄ (k+1), γk
)· g(X̄ (N))
+N−1∑j=0
j−1∏k=0
φk(X̄ (k), X̄ (k+1), γk
) (fj(X̄ (j), γj) + ∆M(j)
)}],
where v ranges over all bounded {0, . . . ,N} × Rd → R with v(N, .) = g ,∆M(j) := v
(j+1, X̄ (j+1)
)φj
(X̄ (j), X̄ (j+1), γj
)−E∗
[v
(j+1, X̄ (j+1)
)φj
(X̄ (j), X̄ (j+1), γj
) ∣∣ X̄ (j)] .
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Lower bound: representation due to [Rogers, 2007]
Assumption: there is a reference probability measure on Rd such that thetransition probabilities µn(x , γ) possess densities pn(x , ., γ) and there is areference density p∗n(x , .). Assumption satis�ed if di�usion coe�cient σ isuniformly elliptic.
Set φn(x , y , γ) :=pn(x,y ,γ)p∗n (x,y)
. Then, with X̄ (0) = x̂ , X̄ under P∗,
V̄ (x̂) = maxvE∗[
inf(γ0,...,γN−1)
{ N−1∏k=0
φk(X̄ (k), X̄ (k+1), γk
)· g(X̄ (N))
+N−1∑j=0
j−1∏k=0
φk(X̄ (k), X̄ (k+1), γk
) (fj(X̄ (j), γj) + ∆M(j)
)}],
where v ranges over all bounded {0, . . . ,N} × Rd → R with v(N, .) = g ,∆M(j) := v
(j+1, X̄ (j+1)
)φj
(X̄ (j), X̄ (j+1), γj
)−E∗
[v
(j+1, X̄ (j+1)
)φj
(X̄ (j), X̄ (j+1), γj
) ∣∣ X̄ (j)] .
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Conclusions
1. Variant of approximate DP using Smolyak's algorithm.
2. Approximation scheme of �rst order almost as e�cient ashigher order schemes, but for a larger class of functions.
3. No useful a priori error bounds; Monte Carlo simulation toobtain a posteriori bounds.
4. To do: numerical robustness of the procedure, numericalexperiments!
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Conclusions
1. Variant of approximate DP using Smolyak's algorithm.
2. Approximation scheme of �rst order almost as e�cient ashigher order schemes, but for a larger class of functions.
3. No useful a priori error bounds; Monte Carlo simulation toobtain a posteriori bounds.
4. To do: numerical robustness of the procedure, numericalexperiments!
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Conclusions
1. Variant of approximate DP using Smolyak's algorithm.
2. Approximation scheme of �rst order almost as e�cient ashigher order schemes, but for a larger class of functions.
3. No useful a priori error bounds; Monte Carlo simulation toobtain a posteriori bounds.
4. To do: numerical robustness of the procedure, numericalexperiments!
Approximate DP and Smolyak's algorithm
A posteriori error bounds
Conclusions
1. Variant of approximate DP using Smolyak's algorithm.
2. Approximation scheme of �rst order almost as e�cient ashigher order schemes, but for a larger class of functions.
3. No useful a priori error bounds; Monte Carlo simulation toobtain a posteriori bounds.
4. To do: numerical robustness of the procedure, numericalexperiments!
Approximate DP and Smolyak's algorithm
Bibliography
Introduction
Classical control problems and approximate DP
Smolyak's algorithm
A posteriori error bounds
Bibliography
Approximate DP and Smolyak's algorithm
Bibliography
Bibliography I
V. Barthelmann, E. Novak and K. Ritter.High dimensional polynomial interpolation on sparse grids.Adv. Comput. Math., 12: 273�288, 2000.
H. J. Bungartz and M. Griebel.Sparse grids.Acta Numerica, 13: 147-269, 2004.
M. Fischer and G. Nappo.Time discretisation and rate of convergence for the optimalcontrol of continuous-time stochastic systems with delay.Appl. Math. Optim., 57(2): 177�206, 2008.
Approximate DP and Smolyak's algorithm
Bibliography
Bibliography II
A. Klimke and B. Wohlmuth.Algorithm 847: spinterp: piecewise multilinear hierarchicalsparse grid interpolation in MATLAB.ACM Trans. Math. Softw., 31(4): 561-579, 2005.
N. V. Krylov.Approximating value functions for controlled degeneratedi�usion processes by using piece-wise constant policies.Electron. J. Probab., 4(Paper 2): 1-19, 1999.
R. Munos.Performance bounds in Lp norm for approximate valueiteration.SIAM Control Optim., 46, 541-561
Approximate DP and Smolyak's algorithm
Bibliography
Bibliography III
E. Novak.Deterministic and Stochastic Error Bounds in NumericalAnalysis.Lecture Notes in Mathematics 1349, Springer, New York, 1988.
L. C. G. Rogers.Pathwise stochastic optimal control.SIAM J. Control Optim., 46(3): 1116-1132, 2007.
A. Schreiber.Die Methode von Smolyak bei der multivariaten Interpolation.Dissertation, Georg-August-Universität zu Göttingen, 2000.
Approximate DP and Smolyak's algorithm
Bibliography
Bibliography IV
S. A. Smolyak.Quadrature and interpolation formulas for tensor products ofcertain classes of functions.Sov. Math., Dokl., 4: 240�243, 1963.
Approximate DP and Smolyak's algorithm
Bibliography
Thank you for your attention.
Approximate DP and Smolyak's algorithm
Approximation error
Let B(X ) := {f : X → R | f bounded} with norm ‖.‖∞.Information of cardinality n given by
In := {ξ : B(X ) → Rn | ξ(f ) = (f (x1), . . . , f (xn)), x i ∈ X}.
The set of algorithms which use information of cardinality n is
An := {Ψ = φ ◦ ξ | φ : Rn → B(X ), ξ ∈ In}.
Let F ⊂ B(X ). Approximation error of an algorithmus Ψ for F :
errApp(F ,Ψ) := supf ∈F
‖Ψ(f )− f ‖∞.
Approximation error based on information of cardinality n for F :
errApp(F , n) := infΨ∈An
errApp(F ,Ψ).
Approximate DP and Smolyak's algorithm
Approximation error
Let B(X ) := {f : X → R | f bounded} with norm ‖.‖∞.Information of cardinality n given by
In := {ξ : B(X ) → Rn | ξ(f ) = (f (x1), . . . , f (xn)), x i ∈ X}.
The set of algorithms which use information of cardinality n is
An := {Ψ = φ ◦ ξ | φ : Rn → B(X ), ξ ∈ In}.
Let F ⊂ B(X ). Approximation error of an algorithmus Ψ for F :
errApp(F ,Ψ) := supf ∈F
‖Ψ(f )− f ‖∞.
Approximation error based on information of cardinality n for F :
errApp(F , n) := infΨ∈An
errApp(F ,Ψ).
Approximate DP and Smolyak's algorithm
Approximation error
Let B(X ) := {f : X → R | f bounded} with norm ‖.‖∞.Information of cardinality n given by
In := {ξ : B(X ) → Rn | ξ(f ) = (f (x1), . . . , f (xn)), x i ∈ X}.
The set of algorithms which use information of cardinality n is
An := {Ψ = φ ◦ ξ | φ : Rn → B(X ), ξ ∈ In}.
Let F ⊂ B(X ). Approximation error of an algorithmus Ψ for F :
errApp(F ,Ψ) := supf ∈F
‖Ψ(f )− f ‖∞.
Approximation error based on information of cardinality n for F :
errApp(F , n) := infΨ∈An
errApp(F ,Ψ).
Approximate DP and Smolyak's algorithm
Approximation error
Let B(X ) := {f : X → R | f bounded} with norm ‖.‖∞.Information of cardinality n given by
In := {ξ : B(X ) → Rn | ξ(f ) = (f (x1), . . . , f (xn)), x i ∈ X}.
The set of algorithms which use information of cardinality n is
An := {Ψ = φ ◦ ξ | φ : Rn → B(X ), ξ ∈ In}.
Let F ⊂ B(X ). Approximation error of an algorithmus Ψ for F :
errApp(F ,Ψ) := supf ∈F
‖Ψ(f )− f ‖∞.
Approximation error based on information of cardinality n for F :
errApp(F , n) := infΨ∈An
errApp(F ,Ψ).
Approximate DP and Smolyak's algorithm
Smolyak's algorithm
Some properties of A(q, d) =∑|~i |≤q(∆
i1 ⊗ . . .⊗∆id ):
A(q, d) =∑
q−d+1≤|~i |≤q
(−1)q−|~i |(d − 1q − |~i |
) (U i1 ⊗ . . .⊗ U id
),
A(q, d) =q−1∑
k=d−1
(A(k , d−1)⊗∆q−k
),
A(q, d) = A(q−1, d) +∑
~i∈Nd ,|~i |=q
(∆i1 ⊗ . . .⊗∆id
).
Approximate DP and Smolyak's algorithm
A general result
[Smolyak, 1963], [Schreiber, 2000]:
Theorem
Let d ∈ N. For each i ∈ N, let U i : F i → V i be a continuous linearoperator with F i ⊂ F , V i ⊂ V Banach spaces. Let I1 : F → V belinear. Set Id := I1 ⊗ . . .⊗ I1.Suppose that, for each l ∈ {2, . . . , d}, the norms on ⊗lF und ⊗lVare uniformly compatible tensor norms. If there are constants B , c ,K such that ‖I1 − U i‖F ,V ≤ c B i , ‖I1‖F ,V ≤ K , then
‖Id −A(q, d)‖⊗dF ,⊗dV ≤ c Bq−d+1(
q
d−1
)max{K , c(1+B)}d−1.
Approximate DP and Smolyak's algorithm
Approximation error on sparse grids of Clenshaw-Curtis-type
Let n(q, d) := #H(q, d) be the number of points of the sparse gridin dimension d of degree q − d .
Assumption: the one-dimensional grids are nested and#X 1 = m1 = 1, #X
i = mi = 2i−1 + 1 se i > 1 (Clenshaw-Curtis).
If B = 2−k for some k ∈ N, then, under the assumptions of thetheorem and with n = n(q, d),
‖Id −A(q, d)‖⊗dF ,⊗dV≤ n−k(2d + log2(n))(1+k)(d−1)
c
(d−1)!k+1max{K , c(1+2−k)}d−1.
Approximate DP and Smolyak's algorithm
Approximation error on sparse grids of Clenshaw-Curtis-type
Let n(q, d) := #H(q, d) be the number of points of the sparse gridin dimension d of degree q − d .
Assumption: the one-dimensional grids are nested and#X 1 = m1 = 1, #X
i = mi = 2i−1 + 1 se i > 1 (Clenshaw-Curtis).
If B = 2−k for some k ∈ N, then, under the assumptions of thetheorem and with n = n(q, d),
‖Id −A(q, d)‖⊗dF ,⊗dV≤ n−k(2d + log2(n))(1+k)(d−1)
c
(d−1)!k+1max{K , c(1+2−k)}d−1.
IntroductionClassical control problems and approximate DPSmolyak's algorithmA posteriori error boundsBibliography