Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof...
Transcript of Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof...
![Page 1: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/1.jpg)
On the construction ofminimax-distance (sub-)optimal
designs
Luc Pronzato
Université Côte d’Azur, CNRS, I3S, France
![Page 2: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/2.jpg)
1) Introduction
1) Introduction & motivation
Objective:Approximation/interpolation of a function f : x ∈X ⊂ Rd −→ R,
(with X compact: typically, X = [0, 1]d)à Choose n points Xn = {x1, . . . , xn} ∈X n (the design)
where to evaluate f (no repetition)
Design criterion = minimax distanceà minimize ΦmM(Xn) = maxx∈X mini=1,...,n ‖x− xi‖ (`2-distance)
= maxx∈X d(x,Xn)= dH(X ,Xn) (Hausdorff distance, `2)= dispersion of Xn in X (Niederreiter, 1992, Chap. 6)
X∗n an optimal n-point design Ô ΦmM-efficiency EffmM(Xn) = Φ∗mM,nΦmM (Xn) ∈ (0, 1]
with Φ∗mM,n = ΦmM(X∗n )
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 2 / 41
![Page 3: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/3.jpg)
1) Introduction
1) Introduction & motivation
Objective:Approximation/interpolation of a function f : x ∈X ⊂ Rd −→ R,
(with X compact: typically, X = [0, 1]d)à Choose n points Xn = {x1, . . . , xn} ∈X n (the design)
where to evaluate f (no repetition)
Design criterion = minimax distanceà minimize ΦmM(Xn) = maxx∈X mini=1,...,n ‖x− xi‖ (`2-distance)
= maxx∈X d(x,Xn)= dH(X ,Xn) (Hausdorff distance, `2)= dispersion of Xn in X (Niederreiter, 1992, Chap. 6)
X∗n an optimal n-point design Ô ΦmM-efficiency EffmM(Xn) = Φ∗mM,nΦmM (Xn) ∈ (0, 1]
with Φ∗mM,n = ΦmM(X∗n )
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 2 / 41
![Page 4: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/4.jpg)
1) Introduction
d = 2, n = 7
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 3 / 41
![Page 5: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/5.jpg)
1) Introduction
Why ΦmM? two good reasons (at least) to minimize ΦmM(Xn):
¬ Suppose f ∈ RKHS H with kernel K (x, y) = C(‖x− y‖), then∀x ∈X , |f (x)− η̂n(x)| ≤ ‖f ‖H ρn(x) where
η̂n(x) = BLUP based on the f (xi ), i = 1, . . . , nρ2n(x) = “kriging variance" at x
see, e.g., Vazquez and Bect (2011); Auffray et al. (2012)
Schaback (1995) à supx∈X ρn(x) ≤ S[ΦmM(Xn)]for some increasing function S[·] (depending on K )
X∗n has no (or few) points on the boundary of X
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 4 / 41
![Page 6: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/6.jpg)
1) Introduction
Why ΦmM? two good reasons (at least) to minimize ΦmM(Xn):
¬ Suppose f ∈ RKHS H with kernel K (x, y) = C(‖x− y‖), then∀x ∈X , |f (x)− η̂n(x)| ≤ ‖f ‖H ρn(x) where
η̂n(x) = BLUP based on the f (xi ), i = 1, . . . , nρ2n(x) = “kriging variance" at x
see, e.g., Vazquez and Bect (2011); Auffray et al. (2012)
Schaback (1995) à supx∈X ρn(x) ≤ S[ΦmM(Xn)]for some increasing function S[·] (depending on K )
X∗n has no (or few) points on the boundary of X
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 4 / 41
![Page 7: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/7.jpg)
1) Introduction
Evaluation of ΦmM(Xn)? Not considered here!
To evaluate ΦmM(Xn) = maxx∈X mini=1,...,n ‖x− xi‖ = maxx∈X d(x,Xn)we need to find x∗ = arg maxx∈X d(x,Xn)
Key idea: replace arg maxx∈X d(x,Xn) by arg maxx∈XQ d(x,Xn)for a suitable finite XQ ∈X Q
Replacing XQ by a regular grid, or first Q points of a Low DiscrepancySequence in X , is not accurate:
à ΦmM(Xn; XQ) ≤ ΦmM(Xn) (optimistic result)requires Q = O(1/εd ) to have ΦmM(Xn) < ΦmM(Xn; XQ) + ε
For d . 5, use tools from algorithmic geometry (Delaunay triangulation orVoronoï tessellation) Þ exact result
For larger d , use MCMC with XQ = adaptive grid (LP, 2017a)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 5 / 41
![Page 8: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/8.jpg)
1) Introduction
Evaluation of ΦmM(Xn)? Not considered here!
To evaluate ΦmM(Xn) = maxx∈X mini=1,...,n ‖x− xi‖ = maxx∈X d(x,Xn)we need to find x∗ = arg maxx∈X d(x,Xn)
Key idea: replace arg maxx∈X d(x,Xn) by arg maxx∈XQ d(x,Xn)for a suitable finite XQ ∈X Q
Replacing XQ by a regular grid, or first Q points of a Low DiscrepancySequence in X , is not accurate:
à ΦmM(Xn; XQ) ≤ ΦmM(Xn) (optimistic result)requires Q = O(1/εd ) to have ΦmM(Xn) < ΦmM(Xn; XQ) + ε
For d . 5, use tools from algorithmic geometry (Delaunay triangulation orVoronoï tessellation) Þ exact result
For larger d , use MCMC with XQ = adaptive grid (LP, 2017a)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 5 / 41
![Page 9: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/9.jpg)
1) Introduction
Bounds on Φ∗mM,n = ΦmM(X∗n ) when X = [0, 1]d
Lower bound: the n balls B(xi ,Φ∗mM,n) cover X
⇒ nVd (Φ∗mM,n)d ≥ vol(X ) (= 1), with Vd = vol[B(0, 1)] = πd/2/Γ(d/2 + 1)
R∗n = (nVd )−1/d ≤ Φ∗mM,n
Upper bound: use any design!
md -point regular grid in X :Φ∗mM,md ≤
√d
2m :
Take m = bn1/dc, so that md ≤ n and Φ∗mM,n ≤ Φ∗mM,md , therefore
Φ∗mM,n ≤ R∗n =√
d2bn1/dc
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 6 / 41
![Page 10: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/10.jpg)
1) Introduction
Bounds on Φ∗mM,n = ΦmM(X∗n ) when X = [0, 1]d
Lower bound: the n balls B(xi ,Φ∗mM,n) cover X
⇒ nVd (Φ∗mM,n)d ≥ vol(X ) (= 1), with Vd = vol[B(0, 1)] = πd/2/Γ(d/2 + 1)
R∗n = (nVd )−1/d ≤ Φ∗mM,n
Upper bound: use any design!
md -point regular grid in X :Φ∗mM,md ≤
√d
2m :
Take m = bn1/dc, so that md ≤ n and Φ∗mM,n ≤ Φ∗mM,md , therefore
Φ∗mM,n ≤ R∗n =√
d2bn1/dc
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 6 / 41
![Page 11: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/11.jpg)
1) Introduction
d = 2
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41
![Page 12: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/12.jpg)
1) Introduction
d = 5
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41
![Page 13: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/13.jpg)
1) Introduction
d = 10
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41
![Page 14: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/14.jpg)
1) Introduction
d = 20
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 7 / 41
![Page 15: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/15.jpg)
1) Introduction
Minimization of ΦmM(Xn) with respect to Xn ∈X n for a given n
® n is not fixed (nmin ≤ n ≤ nmax, we may stop before nmax evaluations of f )
How to obtain good “anytime designs”, such thatall nested designs Xn have a high efficiency EffmM(Xn), nmin ≤ n ≤ nmax
¯ Design measures that minimize a regularized version of ΦmM
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 8 / 41
![Page 16: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/16.jpg)
1) Introduction
Minimization of ΦmM(Xn) with respect to Xn ∈X n for a given n® n is not fixed (nmin ≤ n ≤ nmax, we may stop before nmax evaluations of f )
How to obtain good “anytime designs”, such thatall nested designs Xn have a high efficiency EffmM(Xn), nmin ≤ n ≤ nmax
¯ Design measures that minimize a regularized version of ΦmM
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 8 / 41
![Page 17: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/17.jpg)
1) Introduction
Minimization of ΦmM(Xn) with respect to Xn ∈X n for a given n® n is not fixed (nmin ≤ n ≤ nmax, we may stop before nmax evaluations of f )
How to obtain good “anytime designs”, such thatall nested designs Xn have a high efficiency EffmM(Xn), nmin ≤ n ≤ nmax
¯ Design measures that minimize a regularized version of ΦmM
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 8 / 41
![Page 18: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/18.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed
2) Minimization of ΦmM(Xn), Xn ∈X n, n fixed
General global optimization method (e.g., simulated annealing): notpromising2.1) k-means and centroids2.2) Stochastic gradient
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 9 / 41
![Page 19: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/19.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed
2) Minimization of ΦmM(Xn), Xn ∈X n, n fixed
General global optimization method (e.g., simulated annealing): notpromising2.1) k-means and centroids2.2) Stochastic gradient
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 9 / 41
![Page 20: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/20.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
2.1/ k-means and centroids
Minimize the L2 energy functional
E2(Tn,Xn) =∫
X
( n∑i=1
ICi (x) ‖x− xi‖2)
dx =n∑
i=1
∫Ci
‖x− xi‖2 dx
where Tn = {Ci , i = 1, . . . , n} is a tessellation of XICi = indicator function of Ci
Then (Du et al., 1999):Ci = V(xi ) = Voronoï region for the site xi , for all i
(⇒ E2(Tn,Xn) =∫
X d2(x,Xn) dx)simultaneously xi = centroid of Ci (center of gravity) for all i :
xi = (∫Cix dx)/vol(Ci )
Þ such a Xn should thus perform reasonably well in terms of space-filling(Lekivetz and Jones, 2015)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 10 / 41
![Page 21: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/21.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
2.1/ k-means and centroids
Minimize the L2 energy functional
E2(Tn,Xn) =∫
X
( n∑i=1
ICi (x) ‖x− xi‖2)
dx =n∑
i=1
∫Ci
‖x− xi‖2 dx
where Tn = {Ci , i = 1, . . . , n} is a tessellation of XICi = indicator function of Ci
Then (Du et al., 1999):Ci = V(xi ) = Voronoï region for the site xi , for all i
(⇒ E2(Tn,Xn) =∫
X d2(x,Xn) dx)simultaneously xi = centroid of Ci (center of gravity) for all i :
xi = (∫Cix dx)/vol(Ci )
Þ such a Xn should thus perform reasonably well in terms of space-filling(Lekivetz and Jones, 2015)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 10 / 41
![Page 22: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/22.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
Lloyd’s method (1982): (= fixed-point iterations)
Þ Move each xi to the centroid of its own Voronoï cell, repeat . . .
à Algorithmic geometry (Voronoï tessellation) if d very small,use a finite set XQ otherwise
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 11 / 41
![Page 23: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/23.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
30 points from Sobol’ LDS
k-means clustering (30 clusters) of 1,000 point from Sobol’ LDS
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 12 / 41
![Page 24: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/24.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
k-means clustering (30 clusters) of 1,000 point from Sobol’ LDS
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 12 / 41
![Page 25: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/25.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
However. . . minimax-optimal design is related to the construction of a centroidaltessellation for
Eq(Tn,Xn) =∫
X
( n∑i=1
ICi (x) ‖x− xi‖q
)dx =
n∑i=1
∫Ci
‖x− xi‖q dx
for q →∞à use Chebyshev centers
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 13 / 41
![Page 26: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/26.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
However. . . minimax-optimal design is related to the construction of a centroidaltessellation for
Eq(Tn,Xn) =∫
X
( n∑i=1
ICi (x) ‖x− xi‖q
)dx =
n∑i=1
∫Ci
‖x− xi‖q dx
for q →∞à use Chebyshev centers
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 13 / 41
![Page 27: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/27.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
Variant of Lloyd’s method:
0) Select X (1)n and ε� 1, set k = 1
1) Compute the Voronoï tessellation {Vi , i = 1, . . . , n} of X (or XQ) based onX (k)
n
2) For i = 1, . . . , nä determine the smallest ball B(ci , ri ) enclosing Vi (= convex QP problem)ä replace xi by ci in X (k)
n (Chebyshev center of Vi)3) if ΦmM(X(k)
n )− ΦmM(X(k+1)n ) < ε, then stop; otherwise k ← k + 1, return to
step 1
Þ Move each xi to the Chebyshev center of its own Voronoï cell, repeat . . .
[ΦmM(X(k)n ) decreases monotonically, convergence to a local minimum (or a saddle point)]
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 14 / 41
![Page 28: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/28.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 15 / 41
![Page 29: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/29.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
Determination of the smallest enclosing ball containing Z = {z1, . . . , zN}(vertices of a Voronoï cell, points of XQ closest to xi):
⇔ minimize f (c) = maxi=1,...,N ‖zi − c‖2 with respect to c ∈ Rd
Direct problem = convex QPTake any c0 ∈ Rd , minimize ‖c− c0‖2 + t
with respect to (c, t) ∈ Rd+1,subject to ‖zi − c0‖2 − 2(zi − c0)>(c− c0) ≤ t , i = 1, . . . ,N
(N linear constraints)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 16 / 41
![Page 30: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/30.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
Determination of the smallest enclosing ball containing Z = {z1, . . . , zN}(vertices of a Voronoï cell, points of XQ closest to xi):
⇔ minimize f (c) = maxi=1,...,N ‖zi − c‖2 with respect to c ∈ Rd
Direct problem = convex QPTake any c0 ∈ Rd , minimize ‖c− c0‖2 + t
with respect to (c, t) ∈ Rd+1,subject to ‖zi − c0‖2 − 2(zi − c0)>(c− c0) ≤ t , i = 1, . . . ,N
(N linear constraints)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 16 / 41
![Page 31: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/31.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
Determination of the smallest enclosing ball containing Z = {z1, . . . , zN}
Dual problem = similar to an optimal design problem:maximize trace[V(ξ)], with ξ a prob. measure on Z,
V(ξ) = covariance matrix for ξcenter of the ball = c(ξ) =
∫Z z ξ(dz)
Þ Algorithms of the exchange-type (Yildirim, 2008)(≈ Fedorov algorithm for D-optimal design: optimal step length is available)
Þ One can remove inessential points from Z: (LP, 2017b)à Combine this with the use of a standard QP solver for the direct problem
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 17 / 41
![Page 32: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/32.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.1/ k-means and centroids
Determination of the smallest enclosing ball containing Z = {z1, . . . , zN}
Dual problem = similar to an optimal design problem:maximize trace[V(ξ)], with ξ a prob. measure on Z,
V(ξ) = covariance matrix for ξcenter of the ball = c(ξ) =
∫Z z ξ(dz)
Þ Algorithms of the exchange-type (Yildirim, 2008)(≈ Fedorov algorithm for D-optimal design: optimal step length is available)
Þ One can remove inessential points from Z: (LP, 2017b)à Combine this with the use of a standard QP solver for the direct problem
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 17 / 41
![Page 33: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/33.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient
2.2/ Stochastic gradient
d is large: Lloyd’s algorithm cannot be used (computational geometryis too complicated, regular grids or LDS are not dense enough)
minimize Eq∗(Xn) =
∫X
( n∑i=1
IVi (x) ‖x− xi‖q
)dx
with Vi = Voronoï region for the site xi
Þ Stochastic gradient algorithm:(MacQueen, 1967) for q = 2, (Cardot et al., 2012) for q = 1
0) k = 1, X (1)n , set ni,0 = 0 for all i = 1, . . . , n
1) sample X uniformly distributed in X
2) find i∗ = arg mini=1,...,n ‖X − x(k)i ‖, ni∗,k ← ni∗,k + 1 [← X ∈ cell V∗
i ]
3) x(k+1)i∗ = x(k)
i∗ − γi∗,k q‖X − x(k)i∗ ‖
q−2 (x(k)i∗ − X )︸ ︷︷ ︸
=gradient
, k ← k + 1,
return to step 1, stop when k = K
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 18 / 41
![Page 34: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/34.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient
2.2/ Stochastic gradient
d is large: Lloyd’s algorithm cannot be used (computational geometryis too complicated, regular grids or LDS are not dense enough)
minimize Eq∗(Xn) =
∫X
( n∑i=1
IVi (x) ‖x− xi‖q
)dx
with Vi = Voronoï region for the site xi
Þ Stochastic gradient algorithm:(MacQueen, 1967) for q = 2, (Cardot et al., 2012) for q = 1
0) k = 1, X (1)n , set ni,0 = 0 for all i = 1, . . . , n
1) sample X uniformly distributed in X
2) find i∗ = arg mini=1,...,n ‖X − x(k)i ‖, ni∗,k ← ni∗,k + 1 [← X ∈ cell V∗
i ]
3) x(k+1)i∗ = x(k)
i∗ − γi∗,k q‖X − x(k)i∗ ‖
q−2 (x(k)i∗ − X )︸ ︷︷ ︸
=gradient
, k ← k + 1,
return to step 1, stop when k = K
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 18 / 41
![Page 35: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/35.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient
Typical choice for γi∗,k = c/nαi∗,k , with α ∈ (1/2, 1]and consider X̂n = 1
K∑K
k=1 X (k)n when α < 1
Little information to store (no grid or other finite approximation of X )Þ can also be used with large d
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 19 / 41
![Page 36: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/36.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient
Example: n = 10 dall methods are initialized at the same random design, 100 repetitionsk-means and Lloyd’s method with Chebyshev centers use 2d+8 points
from a LDS (Sobol’)
d = 2, n = 20 (R∗n ≈ 0.1262, R∗n ≈ 0.1768)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 20 / 41
![Page 37: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/37.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient
Example: n = 10 dall methods are initialized at the same random design, 100 repetitionsk-means and Lloyd’s method with Chebyshev centers use 2d+8 points
from a LDS (Sobol’)
d = 3, n = 30 (R∗n ≈ 0.1996, R∗n ≈ 0.2887)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 20 / 41
![Page 38: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/38.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient
Example: n = 10 dall methods are initialized at the same random design, 100 repetitionsk-means and Lloyd’s method with Chebyshev centers use 2d+8 points
from a LDS (Sobol’)
d = 4, n = 40 (R∗n ≈ 0.2668, R∗n = 0.5)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 20 / 41
![Page 39: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/39.jpg)
2) Minimization of ΦmM (Xn), Xn ∈ X n , n fixed 2.2/ Stochastic gradient
Example:
d = 10, n = 100 (R∗n ≈ 0.5746, R∗n ≈ 1.5811)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 20 / 41
![Page 40: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/40.jpg)
3) Nested designs
3) Nested designs
à obtain a high ΦmM-efficiency EffmM(Xn) = Φ∗mM,nΦmM (Xn) for all Xn, nmin ≤ n ≤ nmax
[EffmM(Xn) ∈ (0, 1]]
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 21 / 41
![Page 41: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/41.jpg)
3) Nested designs 3.1/ Coffee-house design
3.1/ Coffee-house design
x1 at the centre of X , then xn+1 furthest point from Xn for all n ≥ 1(called coffee-house design (Müller, 2007, Chap. 4))
Guarantees EffmM(Xn) = Φ∗mM,nΦmM (Xn) ≥
12 and EffMm(Xn) = ΦMm(Xn)
Φ∗Mm,n≥ 1
2 for all n
with ΦMm(Xn) = mini 6=j∈{1,...,n} ‖xi − xj‖ the maximin-distance criterion,and Φ∗Mm,n its optimal (maximum) value
Proof. (Gonzalez, 1985)by construction:ΦMm(Xn+1) , minxi 6=xj∈Xn+1 ‖xi − xj‖ = d(xn+1,Xn) = ΦmM(Xn)X∗n a ΦmM-optimal design: the n balls B(x∗i ,ΦmM(X∗n )), x∗i ∈ X∗n , cover X⇒ one of them contains 2 points xi , xj in Xn+1 for any Xn+1 (n + 1 points)⇒ ΦMm(Xn+1) ≤ ‖xi − xj‖ ≤ 2ΦmM(X∗n )⇒ Φ∗Mm,n+1 ≤ 2ΦmM(X∗n ) ≤ 2ΦmM(Xn) = ΦMm(Xn+1)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 22 / 41
![Page 42: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/42.jpg)
3) Nested designs 3.1/ Coffee-house design
3.1/ Coffee-house design
x1 at the centre of X , then xn+1 furthest point from Xn for all n ≥ 1(called coffee-house design (Müller, 2007, Chap. 4))
Guarantees EffmM(Xn) = Φ∗mM,nΦmM (Xn) ≥
12 and EffMm(Xn) = ΦMm(Xn)
Φ∗Mm,n≥ 1
2 for all n
with ΦMm(Xn) = mini 6=j∈{1,...,n} ‖xi − xj‖ the maximin-distance criterion,and Φ∗Mm,n its optimal (maximum) value
Proof. (Gonzalez, 1985)by construction:ΦMm(Xn+1) , minxi 6=xj∈Xn+1 ‖xi − xj‖ = d(xn+1,Xn) = ΦmM(Xn)X∗n a ΦmM-optimal design: the n balls B(x∗i ,ΦmM(X∗n )), x∗i ∈ X∗n , cover X⇒ one of them contains 2 points xi , xj in Xn+1 for any Xn+1 (n + 1 points)⇒ ΦMm(Xn+1) ≤ ‖xi − xj‖ ≤ 2ΦmM(X∗n )⇒ Φ∗Mm,n+1 ≤ 2ΦmM(X∗n ) ≤ 2ΦmM(Xn) = ΦMm(Xn+1)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 22 / 41
![Page 43: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/43.jpg)
3) Nested designs 3.1/ Coffee-house design
3.1/ Coffee-house design
x1 at the centre of X , then xn+1 furthest point from Xn for all n ≥ 1(called coffee-house design (Müller, 2007, Chap. 4))
Guarantees EffmM(Xn) = Φ∗mM,nΦmM (Xn) ≥
12 and EffMm(Xn) = ΦMm(Xn)
Φ∗Mm,n≥ 1
2 for all n
with ΦMm(Xn) = mini 6=j∈{1,...,n} ‖xi − xj‖ the maximin-distance criterion,and Φ∗Mm,n its optimal (maximum) value
Proof. (Gonzalez, 1985)by construction:ΦMm(Xn+1) , minxi 6=xj∈Xn+1 ‖xi − xj‖ = d(xn+1,Xn) = ΦmM(Xn)X∗n a ΦmM-optimal design: the n balls B(x∗i ,ΦmM(X∗n )), x∗i ∈ X∗n , cover X⇒ one of them contains 2 points xi , xj in Xn+1 for any Xn+1 (n + 1 points)⇒ ΦMm(Xn+1) ≤ ‖xi − xj‖ ≤ 2ΦmM(X∗n )⇒ Φ∗Mm,n+1 ≤ 2ΦmM(X∗n ) ≤ 2ΦmM(Xn) = ΦMm(Xn+1)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 22 / 41
![Page 44: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/44.jpg)
3) Nested designs 3.1/ Coffee-house design
X = [0, 1]2, n = 7
1
2
3
4
5
6
7
EffmM(Xn), n = 1 . . . , 50
0 5 10 15 20 25 30 35 40 45 500.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Regular construction à largefluctuations of EffmM(Xn)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 23 / 41
![Page 45: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/45.jpg)
3) Nested designs 3.1/ Coffee-house design
X = [0, 1]2, n = 7
1
2
3
4
5
6
7
EffmM(Xn), n = 1 . . . , 50
0 5 10 15 20 25 30 35 40 45 500.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Regular construction à largefluctuations of EffmM(Xn)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 23 / 41
![Page 46: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/46.jpg)
3) Nested designs 3.2/ Submodularity and greedy algorithms
3.2/ Submodularity and greedy algorithms
XQ = {x(1), . . . , x(Q)} a finite set with Q points in X(regular grid, first Q points of a LDS — Halton, Sobol’ . . . )
ψ: 2XQ −→ R a set function (to be maximized)non-decreasing: ψ(A ∪ {x}) ≥ ψ(A ) for all A ⊂XQ and x ∈XQ
Definition 1:ψ is submodular iff ψ(A ) +ψ(B) ≥ ψ(A ∪B) +ψ(A ∩B) for all A ,B ⊂XQ
Equivalently, Definition 1’ (diminishing return property):ψ is submodular iff ψ(A ∪ {x})− ψ(A ) ≥ ψ(B ∪ {x})− ψ(B) for allA ⊂ B ⊂XQ and x ∈XQ \B
(a sort of concavity property for set functions)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 24 / 41
![Page 47: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/47.jpg)
3) Nested designs 3.2/ Submodularity and greedy algorithms
3.2/ Submodularity and greedy algorithms
XQ = {x(1), . . . , x(Q)} a finite set with Q points in X(regular grid, first Q points of a LDS — Halton, Sobol’ . . . )
ψ: 2XQ −→ R a set function (to be maximized)non-decreasing: ψ(A ∪ {x}) ≥ ψ(A ) for all A ⊂XQ and x ∈XQ
Definition 1:ψ is submodular iff ψ(A ) +ψ(B) ≥ ψ(A ∪B) +ψ(A ∩B) for all A ,B ⊂XQ
Equivalently, Definition 1’ (diminishing return property):ψ is submodular iff ψ(A ∪ {x})− ψ(A ) ≥ ψ(B ∪ {x})− ψ(B) for allA ⊂ B ⊂XQ and x ∈XQ \B
(a sort of concavity property for set functions)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 24 / 41
![Page 48: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/48.jpg)
3) Nested designs 3.2/ Submodularity and greedy algorithms
Greedy Algorithm:1 set A = ∅2 while |A | < k
find x in XQ such that ψ(A ∪ {x}) is maximalA ← A ∪ {x}
3 end while4 return Ak = A
Denote ψ∗k = maxB⊂XQ , |B|≤k ψ(B)
Theorem (Nemhauser, Wolsey & Fisher, 1978): When ψ is non-decreasing andsubmodular, then for all k ∈ {1, . . . ,Q} the algorithm returns a set Ak such that
ψ(Ak)− ψ(∅)ψ∗k − ψ(∅) ≥ 1− (1− 1/k)k ≥ 1− 1/e > 0.6321
Bad news: we maximize −ΦmM which is non-decreasing but not submodularà no guaranteed efficiency for sequential optimization
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 25 / 41
![Page 49: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/49.jpg)
3) Nested designs 3.2/ Submodularity and greedy algorithms
Greedy Algorithm:1 set A = ∅2 while |A | < k
find x in XQ such that ψ(A ∪ {x}) is maximalA ← A ∪ {x}
3 end while4 return Ak = A
Denote ψ∗k = maxB⊂XQ , |B|≤k ψ(B)
Theorem (Nemhauser, Wolsey & Fisher, 1978): When ψ is non-decreasing andsubmodular, then for all k ∈ {1, . . . ,Q} the algorithm returns a set Ak such that
ψ(Ak)− ψ(∅)ψ∗k − ψ(∅) ≥ 1− (1− 1/k)k ≥ 1− 1/e > 0.6321
Bad news: we maximize −ΦmM which is non-decreasing but not submodularà no guaranteed efficiency for sequential optimization
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 25 / 41
![Page 50: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/50.jpg)
3) Nested designs 3.2/ Submodularity and greedy algorithms
Greedy Algorithm:1 set A = ∅2 while |A | < k
find x in XQ such that ψ(A ∪ {x}) is maximalA ← A ∪ {x}
3 end while4 return Ak = A
Denote ψ∗k = maxB⊂XQ , |B|≤k ψ(B)
Theorem (Nemhauser, Wolsey & Fisher, 1978): When ψ is non-decreasing andsubmodular, then for all k ∈ {1, . . . ,Q} the algorithm returns a set Ak such that
ψ(Ak)− ψ(∅)ψ∗k − ψ(∅) ≥ 1− (1− 1/k)k ≥ 1− 1/e > 0.6321
Bad news: we maximize −ΦmM which is non-decreasing but not submodularà no guaranteed efficiency for sequential optimization
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 25 / 41
![Page 51: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/51.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
3.3/ Submodular alternatives to minimax
A) Covering measure, c.d.f. and dispersion [SIAM UQ, Lausanne, 2016]For any r ≥ 0, any Xn ∈X n, define the covering measure of Xn by
ψr (Xn) = vol{X ∩ [∪ni=1B(xi , r)]} à non-decreasing and submodular
Maximizing ψr (Xn) is equivalent to maximizingFXn (r) = ψr (Xn)/vol(X ) = µL{X ∩[∪n
i=1B(xi ,r)]}µL(X )
which can be considered as a c.d.f., with FXn (r) ∈ [0, 1], increasing in r ,and FXn (0) = 0, FXn (r) = 1 for any r ≥ ΦmM(Xn)
Take any probability measure µ on X (e.g., with finite support XQ)à define FXn (r) = µ{X ∩ [∪n
i=1B(xi , r)]}as a function of r Ô forms a c.d.f.,as a function of Xn Ô non-decreasing and submodular
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 26 / 41
![Page 52: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/52.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
3.3/ Submodular alternatives to minimax
A) Covering measure, c.d.f. and dispersion [SIAM UQ, Lausanne, 2016]For any r ≥ 0, any Xn ∈X n, define the covering measure of Xn by
ψr (Xn) = vol{X ∩ [∪ni=1B(xi , r)]} à non-decreasing and submodular
Maximizing ψr (Xn) is equivalent to maximizingFXn (r) = ψr (Xn)/vol(X ) = µL{X ∩[∪n
i=1B(xi ,r)]}µL(X )
which can be considered as a c.d.f., with FXn (r) ∈ [0, 1], increasing in r ,and FXn (0) = 0, FXn (r) = 1 for any r ≥ ΦmM(Xn)
Take any probability measure µ on X (e.g., with finite support XQ)à define FXn (r) = µ{X ∩ [∪n
i=1B(xi , r)]}as a function of r Ô forms a c.d.f.,as a function of Xn Ô non-decreasing and submodular
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 26 / 41
![Page 53: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/53.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
3.3/ Submodular alternatives to minimax
A) Covering measure, c.d.f. and dispersion [SIAM UQ, Lausanne, 2016]For any r ≥ 0, any Xn ∈X n, define the covering measure of Xn by
ψr (Xn) = vol{X ∩ [∪ni=1B(xi , r)]} à non-decreasing and submodular
Maximizing ψr (Xn) is equivalent to maximizingFXn (r) = ψr (Xn)/vol(X ) = µL{X ∩[∪n
i=1B(xi ,r)]}µL(X )
which can be considered as a c.d.f., with FXn (r) ∈ [0, 1], increasing in r ,and FXn (0) = 0, FXn (r) = 1 for any r ≥ ΦmM(Xn)
Take any probability measure µ on X (e.g., with finite support XQ)à define FXn (r) = µ{X ∩ [∪n
i=1B(xi , r)]}as a function of r Ô forms a c.d.f.,as a function of Xn Ô non-decreasing and submodular
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 26 / 41
![Page 54: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/54.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Which r should we take in FXn (r)?A positive linear combination of non-decreasing submodular functions isnon-decreasing and submodular
à Consider Ψb,B,q(Xn) =∫ B
b rq FXn (r) dr , for B > b ≥ 0, q > 0Ô guaranteed efficiency bounds when maximizing with a greedy algorithm
Justification:
Ψ0,B,q(Xn) = Bq+1
q+1 FXn (B)− 1q+1
∫ B0 rq+1 FXn (dr)
Take any B ≥ ΦmM(Xn) Ô FXn (B) = 1Maximizing Ψ0,B,q(Xn) for B large enough ⇔ minimizing
∫ B0 rq+1 FXn (dr)
⇔ minimizing[∫ B
0 rq+1 FXn (dr)]1/(q+1)
and[∫ B
0 rq+1 FXn (dr)]1/(q+1)
→ ΦmM(Xn) as q →∞
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 27 / 41
![Page 55: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/55.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Which r should we take in FXn (r)?A positive linear combination of non-decreasing submodular functions isnon-decreasing and submodular
à Consider Ψb,B,q(Xn) =∫ B
b rq FXn (r) dr , for B > b ≥ 0, q > 0Ô guaranteed efficiency bounds when maximizing with a greedy algorithm
Justification:
Ψ0,B,q(Xn) = Bq+1
q+1 FXn (B)− 1q+1
∫ B0 rq+1 FXn (dr)
Take any B ≥ ΦmM(Xn) Ô FXn (B) = 1Maximizing Ψ0,B,q(Xn) for B large enough ⇔ minimizing
∫ B0 rq+1 FXn (dr)
⇔ minimizing[∫ B
0 rq+1 FXn (dr)]1/(q+1)
and[∫ B
0 rq+1 FXn (dr)]1/(q+1)
→ ΦmM(Xn) as q →∞
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 27 / 41
![Page 56: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/56.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
ImplementationEasy when
X approximated by XQ = {s1, . . . , sQ} ∈X Q , µ = 1Q∑Q
j=1 δsj
Xn ∈XQn
(inter-distances ‖si − sj‖ are only computed once)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 28 / 41
![Page 57: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/57.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Ex: X = [0, 1]2, XQ = grid with Q = 33× 33 = 1089 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)
à EffmM(Xn) as a function of n
EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - -
Centered L2 discrepancies
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 29 / 41
![Page 58: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/58.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Ex: X = [0, 1]2, XQ = grid with Q = 33× 33 = 1089 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)
à EffmM(Xn) as a function of n
EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - Xnmax with Ψb,B,q(·)
Centered L2 discrepancies
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 29 / 41
![Page 59: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/59.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Ex: X = [0, 1]2, XQ = grid with Q = 33× 33 = 1089 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)
à EffmM(Xn) as a function of n
EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - First nmax points of Sobol’ LDS
Centered L2 discrepancies
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 29 / 41
![Page 60: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/60.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Ex: X = [0, 1]2, XQ = grid with Q = 33× 33 = 1089 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)
à EffmM(Xn) as a function of n
EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - Centered L2 discrepancies
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 29 / 41
![Page 61: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/61.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Ex: X = [0, 1]3, XQ = grid with Q = 113 = 1331 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)
à EffmM(Xn) as a function of n
EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - -
Centered L2 discrepancies
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 30 / 41
![Page 62: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/62.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Ex: X = [0, 1]3, XQ = grid with Q = 113 = 1331 pointsnmin = 15, nmax = 50, q = 2 in Ψb,B,q(·)
à EffmM(Xn) as a function of n
EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - Centered L2 discrepancies
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 30 / 41
![Page 63: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/63.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Large d (d > 3, say): we cannot use a regular grid XQ
Þ adaptive grid with MCMC: illustration for d = 2 (Q ≈ nmaxd)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 31 / 41
![Page 64: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/64.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Large d (d > 3, say): we cannot use a regular grid XQ
Þ adaptive grid with MCMC: illustration for d = 2 (Q ≈ nmaxd)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 31 / 41
![Page 65: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/65.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Ex: X = [0, 1]10, XQ = adaptive grid with Q = 1000 pointsnmin = 30, nmax = 100, q = 2 in Ψb,B,q(·)
à EffmM(Xn) = R∗nΦmM (Xn) as a function of n
EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - -
Centered L2 discrepancies
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 32 / 41
![Page 66: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/66.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Ex: X = [0, 1]10, XQ = adaptive grid with Q = 1000 pointsnmin = 30, nmax = 100, q = 2 in Ψb,B,q(·)
à EffmM(Xn) = R∗nΦmM (Xn) as a function of n
EffmM(Xn): Ψb,B,q(·) —,Halton LDS —, Sobol’ LDS - - Centered L2 discrepancies
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 32 / 41
![Page 67: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/67.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
B) Lq relaxation
Approximate X by XQ with Q elements sk , k = 1, . . . , q, q > 0, minimize
Φq,Q(Xn) ,
1Q
Q∑k=1
(1n
n∑i=1‖sk − xi‖−q
)−11/q
For any Xn, Φq,Q(Xn)→ ΦmM(Xn; XQ), q →∞
where ΦmM(Xn; XQ) = maxx∈XQ d(x,Xn)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 33 / 41
![Page 68: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/68.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
B) Lq relaxation
Approximate X by XQ with Q elements sk , k = 1, . . . , q, q > 0, minimize
Φq,Q(Xn) ,
1Q
Q∑k=1
(1n
n∑i=1‖sk − xi‖−q
)−11/q
For any Xn, Φq,Q(Xn)→ ΦmM(Xn; XQ), q →∞
where ΦmM(Xn; XQ) = maxx∈XQ d(x,Xn)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 33 / 41
![Page 69: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/69.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Efficiency:
If X∗n,q minimizes Φq,Q(·), then
EffmM(X∗n,q; XQ) ≥ (nQ)−1/q
Φq,Q(·) is non-increasingΨ(·) = 1
n Φqq,Q(·) is supermodular
[ongoing joint work with João Rendas (CNRS, I3S, UCA) & Céline Helbert (ÉcoleCentrale Lyon)]
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 34 / 41
![Page 70: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/70.jpg)
3) Nested designs 3.3/ Submodular alternatives to minimax-distance optimal design
Efficiency:
If X∗n,q minimizes Φq,Q(·), then
EffmM(X∗n,q; XQ) ≥ (nQ)−1/q
Φq,Q(·) is non-increasingΨ(·) = 1
n Φqq,Q(·) is supermodular
[ongoing joint work with João Rendas (CNRS, I3S, UCA) & Céline Helbert (ÉcoleCentrale Lyon)]
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 34 / 41
![Page 71: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/71.jpg)
4) Measures minimizing regularized dispersion
4) Measures minimizing regularized dispersion— joint work with Anatoly Zhigljavsky (LP & AZ, 2017)
For a n-point design, Lq relaxation:
Φq,Q(Xn) ,
1Q
Q∑k=1
(1n
n∑i=1‖sk − xi‖−q
)−11/q
, q > 0
For a design measure ξ, integral version:
φq(ξ) ,[∫
X
(∫X
‖s− x‖−q ξ(dx))−1
µ(ds)]1/q
, q > 0
with µ uniform prob. measure on X (µ(X ) = 1)
Th 1: φqq(·), q > 0, is convex, and is strictly convex when 0 < q < d
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 35 / 41
![Page 72: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/72.jpg)
4) Measures minimizing regularized dispersion
4) Measures minimizing regularized dispersion— joint work with Anatoly Zhigljavsky (LP & AZ, 2017)
For a n-point design, Lq relaxation:
Φq,Q(Xn) ,
1Q
Q∑k=1
(1n
n∑i=1‖sk − xi‖−q
)−11/q
, q > 0
For a design measure ξ, integral version:
φq(ξ) ,[∫
X
(∫X
‖s− x‖−q ξ(dx))−1
µ(ds)]1/q
, q > 0
with µ uniform prob. measure on X (µ(X ) = 1)
Th 1: φqq(·), q > 0, is convex, and is strictly convex when 0 < q < d
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 35 / 41
![Page 73: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/73.jpg)
4) Measures minimizing regularized dispersion
q ≥ d
φq(ξ) > 0 for any discrete measure ξφq(ξ) = 0 for any ξ equivalent to the Lebesgue measure on X
. . . not very interesting
0 < q < d
(Strict) convexity of φqq(·) Þ “equivalence theorem”
Th 2: ξq,∗ minimizes φq(·) iff ∀y ∈X , d(ξq,∗, y) ≤ φqq(ξq,∗)
where d(ξ, y) =∫
X
{‖y− x‖−q [∫
X ‖x− z‖−q ξ(dz)]−2}
µ(dx)= directional derivative of φq
q(·) at ξ in the direction of δy
ξq,∗ is unique and d(ξq,∗, y) = φqq(ξq,∗) for ξq,∗-almost all y ∈X
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 36 / 41
![Page 74: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/74.jpg)
4) Measures minimizing regularized dispersion
q ≥ d
φq(ξ) > 0 for any discrete measure ξφq(ξ) = 0 for any ξ equivalent to the Lebesgue measure on X
. . . not very interesting
0 < q < d
(Strict) convexity of φqq(·) Þ “equivalence theorem”
Th 2: ξq,∗ minimizes φq(·) iff ∀y ∈X , d(ξq,∗, y) ≤ φqq(ξq,∗)
where d(ξ, y) =∫
X
{‖y− x‖−q [∫
X ‖x− z‖−q ξ(dz)]−2}
µ(dx)= directional derivative of φq
q(·) at ξ in the direction of δy
ξq,∗ is unique and d(ξq,∗, y) = φqq(ξq,∗) for ξq,∗-almost all y ∈X
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 36 / 41
![Page 75: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/75.jpg)
4) Measures minimizing regularized dispersion
Two distinct situations
0 < q ≤ d − 2
ξq,∗ may be singular
Ex: X = Bd (0, 1); ξq,∗ = δ0 is optimal
max{0, d − 2} < q < d
Th 3: ξq,∗ does not possess atoms in the interior of X
Þ Minimization of Φq,Q(Xn): take q > d − 2 to be space-filling
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 37 / 41
![Page 76: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/76.jpg)
4) Measures minimizing regularized dispersion
Two distinct situations
0 < q ≤ d − 2
ξq,∗ may be singular
Ex: X = Bd (0, 1); ξq,∗ = δ0 is optimal
max{0, d − 2} < q < d
Th 3: ξq,∗ does not possess atoms in the interior of X
Þ Minimization of Φq,Q(Xn): take q > d − 2 to be space-filling
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 37 / 41
![Page 77: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/77.jpg)
4) Measures minimizing regularized dispersion
Construction of ξq,∗?
Discretize X (again): replace µ by µQ = 1Q∑Q
k=1 δsk (grid or LDS)
φqq(ξ;µQ) = trace[M−1(ξ)]
with M(ξ) =∫
X diag{Q ‖x− sk‖−q, k = 1, . . . ,Q} ξ(dx) (Q × Q-dimensional)Þ an A-optimal design problem: multiplicative, or vertex-direction, algorithm
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 38 / 41
![Page 78: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/78.jpg)
4) Measures minimizing regularized dispersion
Ex: X = Bd (0, 1), make use of symmetry(only consider distributions of the radii)
φqq(ξ) function of q for ξ = δ0 (. . .), ξ = µ (- - -) and ξ = ξq,∗ (—)
d = 3
0 0.5 1 1.5 2 2.5 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
d = 5
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 50
0.2
0.4
0.6
0.8
1
1.2
1.4
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 39 / 41
![Page 79: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/79.jpg)
4) Measures minimizing regularized dispersion
µ(r) uniform on Bd (0, r), d = 3
efficiency φqq(ξq,∗)φq
q(µ(r)) of µ(r) function of rq = 0.5, q = 1.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.88
0.9
0.92
0.94
0.96
0.98
1
optimal r function of q
0 0.5 1 1.5 2 2.5 30
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 39 / 41
![Page 80: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/80.jpg)
4) Measures minimizing regularized dispersion
d = 3, optimal density of radii for ξq,∗ (with respect to ϕ(r) = drd−1)
q = 2, q = 2.1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
q = 2.25, q = 2.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 39 / 41
![Page 81: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/81.jpg)
4) Measures minimizing regularized dispersion
d = 3, optimal density of radii for ξq,∗ (with respect to ϕ(r) = drd−1)
q = 2, q = 2.1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
q = 2.25, q = 2.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1.4
Minimization of Φq,Q(Xn):take q > d − 2 to be space-filling,no point near the border of X
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 39 / 41
![Page 82: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/82.jpg)
4) Measures minimizing regularized dispersion
Conclusions
Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient
(without any evaluation of ΦmM(Xn))
Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))
Use an adaptive grid XQ (MCMC) if d is largeConsider projections on lower dimensional subspaces?Which submodular alternative is best?
What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41
![Page 83: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/83.jpg)
4) Measures minimizing regularized dispersion
Conclusions
Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient
(without any evaluation of ΦmM(Xn))Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))
Use an adaptive grid XQ (MCMC) if d is largeConsider projections on lower dimensional subspaces?Which submodular alternative is best?
What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41
![Page 84: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/84.jpg)
4) Measures minimizing regularized dispersion
Conclusions
Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient
(without any evaluation of ΦmM(Xn))Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))
Use an adaptive grid XQ (MCMC) if d is large
Consider projections on lower dimensional subspaces?Which submodular alternative is best?
What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41
![Page 85: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/85.jpg)
4) Measures minimizing regularized dispersion
Conclusions
Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient
(without any evaluation of ΦmM(Xn))Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))
Use an adaptive grid XQ (MCMC) if d is largeConsider projections on lower dimensional subspaces?
Which submodular alternative is best?
What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41
![Page 86: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/86.jpg)
4) Measures minimizing regularized dispersion
Conclusions
Several methods to evaluate ΦmM(Xn) (MCMC if d ≥ 5)d small: optimization by a variant of Lloyd’s method with Chebyshev centers(requires Voronoï tessellation or a fixed finite set approximation XQ)d large: optimization by a stochastic gradient
(without any evaluation of ΦmM(Xn))Greedy methods based on submodular alternatives to dispersion can generatenested designs with reasonably good minimax efficiency (better than LDS,also without any evaluation of ΦmM(Xn))
Use an adaptive grid XQ (MCMC) if d is largeConsider projections on lower dimensional subspaces?Which submodular alternative is best?
What about very large d (d > 20 say)? Random designs may beuseful. . . (Janson, 1986, 1987)
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 40 / 41
![Page 87: Ontheconstructionof minimax-distance(sub-)optimal designs copy.pdf · Ontheconstructionof minimax-distance(sub-)optimal designs Luc Pronzato UniversitéCôted’Azur,CNRS,I3S,France](https://reader031.fdocuments.us/reader031/viewer/2022040408/5eb50cbb06367d39834df423/html5/thumbnails/87.jpg)
References
References I
Auffray, Y., Barbillon, P., Marin, J.-M., 2012. Maximin design on non hypercube domains and kernel interpolation. Statistics and Computing 22 (3),703–712.
Cardot, H., Cénac, P., Monnez, J.-M., 2012. A fast and recursive algorithm for clustering large datasets. Comput. Statist. Data Anal. 56 (6), 1434–1449.
Du, Q., Faber, V., Gunzburger, M., 1999. Centroidal Voronoi tessellations: applications and algorithms. SIAM Review 41 (4), 637–676.
Gonzalez, T., 1985. Clustering to minimize the maximum intercluster distance. Theoretical Computer Science 38, 293–306.
Janson, S., 1986. Random coverings in several dimensions. Acta Mathematica 156 (1), 83–118.
Janson, S., 1987. Maximal spacings in several dimensions. The Annals of Probability 15 (1), 274–280.
Lekivetz, R., Jones, B., 2015. Fast flexible space-filling designs for nonrectangular regions. Quality and Reliability Engineering International 31 (5), 829–837.
Lloyd, S., 1982. Least squares quantization in PCM. IEEE Transactions on Information Theory 28 (2), 129–137.
MacQueen, J., 1967. Some methods for classification and analysis of multivariate observations. In: LeCam, L., Neyman, J. (Eds.), Proceedings of the FifthBerkeley Symposium on Mathematical Statistics and Probability. Vol. 1. pp. 281–297.
Müller, W., 2007. Collecting Spatial Data. Springer, Berlin, [3rd ed.].
Nemhauser, G., Wolsey, L., Fisher, M., 1978. An analysis of approximations for maximizing submodular set functions–I. Mathematical Programming14 (1), 265–294.
Niederreiter, H., 1992. Random Number Generation and Quasi-Monte Carlo Methods. SIAM, Philadelphia.
Pronzato, L., 2017a. Minimax and maximin space-filling designs: some properties and methods for construction. Journal de la Société Française deStatistique 158 (1), 7–36.
Pronzato, L., 2017b. On the elimination of inessential points in the smallest enclosing ball problem. Optimization Methods and SoftwareTo appear.
Pronzato, L., Zhigljavsky, A., 2017. Measures minimizing regularized dispersion. Submitted.
Schaback, R., 1995. Error estimates and condition numbers for radial basis function interpolation. Advances in Computational Mathematics 3 (3), 251–264.
Vazquez, E., Bect, J., 2011. Sequential search based on kriging: convergence analysis of some algorithms. Proc. 58th World Statistics Congress of the ISI,August 21-26, Dublin, Ireland, arXiv preprint arXiv:1111.3866v1.
Yildirim, E., 2008. Two algorithms for the minimum enclosing ball problem. SIAM Journal on Optimization 19 (3), 1368–1391.
Luc Pronzato (CNRS) Minimax-distance (sub-)optimal designs BIRS, Banff, Aug. 11, 2017 41 / 41