Add Isotropic Gaussian Kernels at Own Risk€¦ · No Ghost Modes Theorem In R1, the number of...

Add Isotropic Gaussian Kernels at Own RiskMore and More Resilient Modes in Higher Dimensions

Herbert Edelsbrunner, BRITTANY TERESE FASY, and Gunter Rote

Symposium on Computational Geometry 2012Chapel Hill, North Carolina

18 June 2012

Edelsbrunner, Fasy and Rote (SoCG 2012) Gaussian Mixtures 18 June 2012 1 / 31

Introduction Counting Modes

Counting Modes and Critical Points

Definition

A critical point is a point with a zero gradient.

Definition

A mode is a local maximum.

Definition

How Many Modes (Local Maxima)?

we see

Existence proven in [M. Carreira-Perpinan and C. Williams, Scotland 2003].

In the begining, we see 1 local maximum.

At the end, we see 3 local maxima.

In the middle, we see 4 local maxima.

Contributions

Brief Overview

Define Gaussian kernel and mixture.

Analyze 1-dimensional mixtures.

Locate and count all critical points of an n-dimensional mixture.

Locate and count all modes of an n-dimensional mixtures.

(Describe the resilience of the ghost mode.)

Contributions

Brief Overview

Contributions

Brief Overview

Contributions

Brief Overview

Contributions

Brief Overview

Gaussian Kernel One Dimensional

Gaussian Kernel

Definition

gz(x) =1√

2πσ2e

−(x−z)2

Center: zStandard Deviation: σHeight: 1√

2πσ2

Gaussian Kernel

Definition

gz(x) =1√

2πσ2e

−(x−z)2

Center: z

Standard Deviation: σHeight: 1√

2πσ2

Gaussian Kernel

Definition

gz(x) =1√

2πσ2e

−(x−z)2

Center: zStandard Deviation: σ

Height: 1√2πσ2

Gaussian Kernel

Definition

gz(x) =1√

2πσ2e

−(x−z)2

Center: zStandard Deviation: σHeight: 1√

2πσ2

Standardized Gaussian Kernel

Definition

gz(x) = e−π(x−z)2

Center: zStandard Deviation: σ0 = 1√

2πHeight: 1

Definition

Center: z

Standard Deviation: σ0 = 1√2π

Height: 1

Definition

Height: 1

Definition

2πHeight: 1

Gaussian Kernel n-Dimensional

n-Dimensional Isotropic Gaussian Kernel

Definition

gz(x) = e−π||x−z||2

Center: zWidth: σ0 = 1√

2πHeight: 1

Definition

gz(x) = e−π||x−z||2

Center: z

Width: σ0 = 1√2π

Height: 1

Definition

gz(x) = e−π||x−z||2

Height: 1

Definition

gz(x) = e−π||x−z||2

2πHeight: 1

Separability of the Gaussian Kernel

Separability Lemma

e−π||x−z||2

= e−π||x−y ||2e−π||y−z||

||x − z ||2 = ||x − y ||2 + ||y − z ||2

Separability Lemma

e−π||x−z||2

= e−π||x−y ||2

e−π||y−z||2

||x − z ||2 = ||x − y ||2 + ||y − z ||2

Separability Lemma

e−π||x−z||2

= e−π||x−y ||2e−π||y−z||

||x − z ||2 = ||x − y ||2 + ||y − z ||2

Separability Lemma

e−π||x−z||2

= e−π||x−y ||2e−π||y−z||

||x − z ||2 = ||x − y ||2 + ||y − z ||2

Gaussian Kernel Restrictions

Restrictions of Kernels

Definition

A restriction of gz is theevaluation of the function on alower-dimensional plane P.

gz |P(x) = gy (x).

Definition

gz |P(x) = gz(y)gy (x).

Definition

gz |P(x) = czgy (x).

Gaussian Mixtures

Gaussian Mixture

A Gaussian mixture is the sum of Gaussian kernels.

Gaussian Mixtures

Gaussian Mixture

Gaussian Mixtures

Gaussian Mixture

Gaussian Mixtures

Restrictions of Mixtures

Gaussian Mixtures

No Ghost Modes

Theorem

In R1, the number of modes is at most the number of components.

Balanced sum of two kernels: [Burke, 1956].

Weighted sum of two kernels: [Behboodian, 1970].

General sum: [M. Carreira-Perpinan and C. Williams, LNCS 2003]relies heavily on [Silverman, 1981].

Question

When is the transition between having one mode and two?

Gaussian Mixtures

No Ghost Modes

Theorem

Question

Gaussian Mixtures

No Ghost Modes

Theorem

Question

Gaussian Mixtures

No Ghost Modes

Theorem

Question

Gaussian Mixtures

No Ghost Modes

Theorem

Question

Gaussian Mixtures Sums in R1

Weighted Gaussian Mixture

Gw (x) = ckg−z(x) + c`gz(x).

The Weighted Mixture

1 If z is small enough, thenGw has one critical point.

2 If z is large enough, then Gw

has three critical points.

3 Gw has exactly 2 criticalpoints when ck

c`= r(x) + 1.

Gaussian Mixtures Ghosts Exist

Counting Modes in Rn

For n ≥ 2, there can be moremodes than components of aGaussian mixture in Rn.

Design n-Simplex

Standard n-Simplex, ∆n

An n-simplex is the convex hullof n + 1 vertices.

The standard n-simplex has thestandard basis elements as thevertices:

e1, e2, . . . , en+1.

Design n-Simplex

An n-simplex is the convex hullof n + 1 vertices.The standard n-simplex has thestandard basis elements as thevertices:

e1, e2, . . . , en+1.

Design n-Simplex

e1, e2, . . . , en+1.

Design n-Simplex

e1, e2, . . . , en+1.

Design n-Simplex

Scaled n-Simplex, s∆n

The scaled standard n-simplex inRn+1 is defined by the n + 1standard basis elements, scaledby a factor s

se1, se2, . . . , sen+1.

The barycenter is the averagevertex position:(

n + 1,

n + 1, . . . ,

Design n-Design

Scaled n-Design

Definition

The Scaled n-Design is theGaussian mixture with centers atthe n + 1 vertices of the scaledn-simplex:

Gs(x) =n+1∑i=1

gsei (x)

Design Properties

se1, se2, . . . , sen+1.

n + 1,

n + 1, . . . ,

Design Properties

se1, se2, . . . , sen+1.

n + 1,

n + 1, . . . ,

Design Properties

Complementary Faces

We partition the vertices of thescaled n-simplex into two sets:

Let k = |K | − 1 and ` = |L| − 1.

Design Properties

Complementary Faces

K = {se3},L = {se1, se2}.

Let k = |K | − 1 and ` = |L| − 1.

Design Properties

Complementary Faces

K = {se3},L = {se1, se2}.

Let k = |K | − 1 and ` = |L| − 1.

Design Properties

Complementary Faces

K = {se2, se5},L = {se1, se3, se4}.

Let k = |K | − 1 and ` = |L| − 1.

Design Axes

Location of Critical Points

The axis AK ,L is the line definedby bK and bL.

Location of Critical Values

All critical points of the scaledn-design lie on an axis of s∆n.

Assume a critical point x is noton an axis ...

Design Axes

Design Restrictions

Restriction to an Axis

Gs |A(x) = cke−πh(x) + c`e

−π(Dk,`−h(x)),

where ck = (k + 1)gsei (bL), c` = (`+ 1)gsej (bK ).

Design Restrictions

−π(Dk,`−h(x)),

Design Restrictions

−π(Dk,`−h(x)),

Ghosts

c`= r(x) + 1.

Ghosts Lower Transition Scale Factor

Lower Transition Scale Factor Tk ,`

Definition

Tk,` is the scale factor for which

= r(x) + 1.

1-Dimensional Maxima Lemma

For all s > Tk,`, the axis AK ,L

witnesses two one-dimensionalmaxima.

If s ∈ (Tk,`,Un), then AK ,L witnesses two one-dimensional maxima,one of which is at the barycenter.

Definition

= r(x) + 1.

Definition

= r(x) + 1.

Definition

= r(x) + 1.

Ghosts Mode at Barycenter

Upper Transition Scale Factor Un

Definition

Tk,` is the scale factor for which:

= r(x) + 1.

Definition

√n + 1

Barycenter Lemma

The barycenter of s∆n is a modefor s < Un, and a saddle ofindex 1 for s > Un.

Theorem

If s ∈ (Tk,`,Un), then AK ,L witnesses two one-dimensional maxima, one ofwhich is at the barycenter.

Upper Transition Scale Factor Un

Definition

= r(x) + 1.

Definition

√n + 1

Barycenter Lemma

Theorem

One-Dimensional Maxima

Definition

= r(x) + 1.

Definition

√n + 1

Barycenter Lemma

Theorem

Gs |A(x) = e−πh(x) + c`e−π(D0,n−1−h(x)),

where c` = (`+ 1)gsej (bL).

Ghosts Witnessed Ghost

Witnessing the Modes

Witnessing Modes

If |K | = 1, then AK ,L witnessestwo modes for s ∈ (T0,n−1,Un).

Witnessing Critical Points

If |K | > 1, then M is a criticalpoint, not a mode.

Witnessing Modes

Ghosts Axes

Many Axes

Number of Axes with k = 0:

n + 1.

The scaled design hasn + 2 modes.

Total Number of Axes:

n+1∑k=1

(n + 1

)= 2n − 1.

The scaled design has Θ(2n)critical points.

Ghosts Axes

Many Axes

n + 1.

n+1∑k=1

(n + 1

)= 2n − 1.

Ghosts Axes

Many Axes

n + 1.

n+1∑k=1

(n + 1

)= 2n − 1.

Ghosts Axes

Many Axes

n + 1.

n+1∑k=1

(n + 1

)= 2n − 1.

Ghosts Axes

Many Axes

n + 1.

n+1∑k=1

(n + 1

)= 2n − 1.

Wrapping Up Summary

Summary of Results

The n-design has:

1 at most ONE ghost mode.

2 an exponential number ofcritical points.

3 all critical points on axes.

Wednesday at 2:50 in SN011:

1 How does Un − T0,n−1 (theresilience) grow withdimension?

2 What is the persistence ofthe ghost mode?

Wrapping Up Summary

Summary of Results

The n-design has:

Wrapping Up Summary

Summary of Results

The n-design has:

Wrapping Up Summary

Summary of Results

The n-design has:

Wrapping Up Summary

Summary of Results

The n-design has:

Wrapping Up Summary

Summary of Results

The n-design has:

Wrapping Up Summary

Add Isotropic Gaussian Kernels at Own RiskMore and More Resilient Modes in Higher Dimensions

Herbert Edelsbrunner, BRITTANY TERESE FASY, and Gunter Rote

Symposium on Computational Geometry 2012Chapel Hill, North Carolina

18 June 2012

Wrapping Up Summary

Summary of Results

The n-design has:

Wrapping Up References

References

Behboodian, J.On the modes of a mixture of two normal distributions.Technometrics 12, 1 (1970), 131–139.

Burke, P. J.Solution of problem 4616 [1954, 718], proposed by A. C. Cohen, Jr.Amer. Math. Monthly 63, 2 (Feb. 1956), 129.

Carreira-Perpinan, M., and Williams, C.On the number of modes of a Gaussian mixture.Scale Space Methods in Computer Vision, Lecture Notes in ComputerScience 2695 (2003), 625–640.

Wrapping Up References

References

Carreira-Perpinan, M., and Williams, C.An isotropic Gaussian mixture can have more modes thancomponents.Informatics Research Report EDI-INF-RR-0185, Institute for Adaptiveand Neural Computation, University of Edinbugh, Dec. 2003.

Fasy, B. T.Modes of Gaussian mixtures and an inequality for the distancebetween curves in space, June 2012.PhD Dissertation, Duke University Comput. Sci. Dept.

Silverman, B. W.Using kernel density estimates to investigate multimodality.J. R. Stat. Soc. Ser. B. Stat. Methodol. 43 (1981), 97–99.

Add Isotropic Gaussian Kernels at Own Risk€¦ · No Ghost Modes Theorem In R1, the number of...

Documents

Transcript of Add Isotropic Gaussian Kernels at Own Risk€¦ · No Ghost Modes Theorem In R1, the number of...