1 UNC, Stat & OR Nonnegative Matrix Factorization.
-
Upload
ezra-newton -
Category
Documents
-
view
223 -
download
1
Transcript of 1 UNC, Stat & OR Nonnegative Matrix Factorization.
1
UNC, Stat & OR
Standard Approach:
Lee et al (1999):
Formulate & Solve Optimization
Major Challenge:
Not Nested, ()
Nonnegative Matrix Factorization
2
UNC, Stat & OR
Standard NMF
But Note
Not Nested
No “Multi-scale”
Analysis
Possible (Scores Plot?!?)
Nonnegative Matrix Factorization
3
UNC, Stat & OR
Same Toy
Data Set
Rank 1
Approx.
Properly
Nested
Nonnegative Nested Cone Analysis
4
UNC, Stat & OR
5-d Toy Example Rank 3 NNCA Approx.
Current
Research:
How Many
Nonneg.
Basis El’ts
Needed?
Nonnegative Nested Cone Analysis
5
UNC, Stat & OR
How generally applicable is
Backwards approach to PCA?
Potential Application: Principal Curves
Hastie & Stuetzle, (1989)
(Foundation of Manifold Learning)
An Interesting Question
6
UNC, Stat & OR
How generally applicable is
Backwards approach to PCA?
An Attractive Answer:
James Damon, UNC Mathematics
Key Idea: Express Backwards PCA as
Nested Series of Constraints
An Interesting Question
7
UNC, Stat & OR
New Topic
Curve Registration
8
UNC, Stat & OR
Context
Functional Data AnalysisCurves as Data Objects
Toy Example:
How Can WeUnderstandVariation?
9
UNC, Stat & OR
Context
Functional Data AnalysisCurves as Data Objects
Toy Example:
How Can WeUnderstandVariation?
10
UNC, Stat & OR
Functional Data Analysis
InsightfulDecomposition
Vertical Variation
Horiz’l Var’n
11
UNC, Stat & OR
Challenge
Fairly Large Literature
Many (Diverse) Past Attempts
Limited Success (in General)
Surprisingly Slippery
(even mathematical formulation)
12
UNC, Stat & OR
Challenge (Illustrated)
Thanks to Wei Wu
13
UNC, Stat & OR
Challenge (Illustrated)
Thanks to Wei Wu
14
UNC, Stat & OR
Functional Data Analysis
AppropriateMathematicalFramework? Vertical Variation
Horiz’l Var’n
15
UNC, Stat & OR
Landmark Based Shape Analysis
Approach: Identify objects that are:
• Translations
• Rotations
• Scalings
of each other
Mathematics: Equivalence Relation
Results in: Equivalence Classes
Which become the Data Objects
16
UNC, Stat & OR
Curve Registration
What are the Data Objects?
Consider “Time Warpings”
(smooth)
More Precisely: Diffeomorphisms
17
UNC, Stat & OR
Time Warping Intuition
Elastically Stretch & Compress Axis
18
UNC, Stat & OR
Curve Registration
Say curves and are equivalent,
When so that
19
UNC, Stat & OR
Data Objects I
Equivalence Classes of Curves
(Set of AllWarps ofGiven Curve)
Notation: for a “representor”
20
UNC, Stat & OR
Data Objects I
Equivalence Classes of Curves
(Set of AllWarps ofGiven Curve)
Next Task: Find Metric on Equivalence Classes
21
UNC, Stat & OR
Metrics in Curve Space
Find Metric on Equivalence Classes
Start with Warp Invariance on Curves& Extend
22
UNC, Stat & OR
Metrics in Curve Space
Traditional Approach to Curve
Registration:
• Align curves, say and
• By finding optimal time warp, , so:
• Vertical var’n: PCA after alignment
• Horizontal var’n: PCA on s
23
UNC, Stat & OR
Metrics in Curve Space
Problem:
Don’t have proper metric
Since:
Because:
24
UNC, Stat & OR
Metrics in Curve Space
Thanks toXiaosun Lu
25
UNC, Stat & OR
Metrics in Curve Space
Note:VeryDifferentL2 norms
Thanks toXiaosun Lu
26
UNC, Stat & OR
Metrics in Curve Space
Solution:
Look for Warp Invariant Metric
Where:
27
UNC, Stat & OR
Metrics in Curve Space
I.e. Have “Parallel”
Representatives
Of Equivalence
Classes
28
UNC, Stat & OR
Metrics in Curve Space
Warping Invariant Metric
Developed in context of:
Likelihood Geometry
Fisher – Rao Metric:
29
UNC, Stat & OR
Metrics in Curve Space
Fisher – Rao Metric:
Computation Based on
Square Root Velocity Function (SRVF)
30
UNC, Stat & OR
Metrics in Curve Space
Square Root Velocity Function (SRVF)
31
UNC, Stat & OR
Metrics in Curve Space
Fisher – Rao Metric:
Computation Based on SRVF:
So work with SRVF
32
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
33
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
Thanks to
Xiaosun Lu
34
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
35
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
36
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
37
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
38
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
39
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
40
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
41
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots?
42
UNC, Stat & OR
Metrics in Curve Space
Why
square
roots? Dislikes Pinching
Focusses Well On
Peaks of Unequal Height
43
UNC, Stat & OR
Metrics in Curve Space
Note on SRVF representation:
Can show: Warp Invariance
Follows from Jacobean calculation
44
UNC, Stat & OR
Metrics in Curve Quotient Space
Above was Invariance for Individual
Curves
Now extend to:
Equivalence Classes of Curves
I.e. Orbits as Data Objects
I.e. Quotient Space
45
UNC, Stat & OR
Metrics in Curve Quotient Space
Define Metric on Equivalence Classes:
For & , i.e. &
Independent of Choice of &
By Warping Invariance
46
UNC, Stat & OR
Mean in Curve Quotient Space
Benefit of a Metric:
Allows Definition of a “Mean”
Fréchet Mean
Geodesic Mean
Barycenter
Karcher Mean
47
UNC, Stat & OR
Mean in Curve Quotient Space
Given Equivalence Class Data Objects:
The Karcher Mean is:
48
UNC, Stat & OR
Mean in Curve Quotient Space
The Karcher Mean is:
Intuition: Can Show, for Euclidean
Data
Minimizer = Conventional
49
UNC, Stat & OR
Mean in Curve Quotient Space
Next Define “Most Representative”
Choice of
As Representer of
50
UNC, Stat & OR
Mean in Curve Quotient Space
“Most Representative” in
Given a candidate
Consider warps to each
Choose to make
Karcher mean of warps = Identity
(under Fisher Rao metric)
51
UNC, Stat & OR
Mean in Curve Quotient Space
“Most Representative” in
Thanks to Anuj Srivastava
52
UNC, Stat & OR
Toy Example – (Details Later)
EstimatedWarps
(Note:RepresentedWith KarcherMean At Identity)
53
UNC, Stat & OR
Mean in Curve Quotient Space
“Most Representative” in
Terminology: The “Template Mean”
54
UNC, Stat & OR
More Data Objects
Final Curve Warps:
• Warp Each Data Curve,
• To Template Mean,
• Denote Warp Functions
Gives (Roughly Speaking):
Vertical Components
(Aligned Curves)
Horizontal Components
55
UNC, Stat & OR
More Data Objects
Final Curve Warps:
• Warp Each Data Curve,
• To Template Mean,
• Denote Warp Functions
Gives (Roughly Speaking):
Vertical Components
(Aligned Curves)
Horizontal Components
Data Objects II
56
UNC, Stat & OR
More Data Objects
Final Curve Warps:
• Warp Each Data Curve,
• To Template Mean,
• Denote Warp Functions
Gives (Roughly Speaking):
Vertical Components
(Aligned Curves)
Horizontal Components
Data Objects II
~ Kendall’s Shapes
57
UNC, Stat & OR
More Data Objects
Final Curve Warps:
• Warp Each Data Curve,
• To Template Mean,
• Denote Warp Functions
Gives (Roughly Speaking):
Vertical Components
(Aligned Curves)
Horizontal Components
Data Objects III
58
UNC, Stat & OR
More Data Objects
Final Curve Warps:
• Warp Each Data Curve,
• To Template Mean,
• Denote Warp Functions
Gives (Roughly Speaking):
Vertical Components
(Aligned Curves)
Horizontal Components
Data Objects III
~ Chang’s Transfo’s
59
UNC, Stat & OR
Computation
Several Variations of
Dynamic Programming
Done by Eric Klassen, Wei Wu
60
UNC, Stat & OR
Toy Example
Raw Data
61
UNC, Stat & OR
Toy Example
Raw Data
BothHorizontalAndVerticalVariation
62
UNC, Stat & OR
Toy Example
ConventionalPCAProjections
63
UNC, Stat & OR
Toy Example
ConventionalPCAProjections
PowerSpreadAcrossSpectrum
64
UNC, Stat & OR
Toy Example
ConventionalPCAProjections
PowerSpreadAcrossSpectrum
65
UNC, Stat & OR
Toy Example
ConventionalPCAScores
66
UNC, Stat & OR
Toy Example
ConventionalPCAScores
Views of1-d CurveBendingThrough4 Dim’ns’
67
UNC, Stat & OR
Toy Example
ConventionalPCAScores
PatternsAre“Harmonics”In Scores
68
UNC, Stat & OR
Toy Example
Scores PlotShows DataAre “1”Dimensional
So NeedImprovedPCA Decomp.
69
UNC, Stat & OR
Visualization
Vertical Variation:
• PCA on Aligned Curves,
• Projected Curves
70
UNC, Stat & OR
Toy Example
AlignedCurves
(Clear1-dVerticalVar’n)
71
UNC, Stat & OR
Toy Example
AlignedCurvePCAProjections
All Var’nIn 1st
Component
72
UNC, Stat & OR
Visualization
Horizontal Variation:
• PCA on Warps,
• Projected Curves
73
UNC, Stat & OR
Toy Example
EstimatedWarps
74
UNC, Stat & OR
Toy Example
Warps,PCProjections
75
UNC, Stat & OR
Toy Example
Warps,PCProjections
Mostly1st PC
76
UNC, Stat & OR
Toy Example
Warps,PCProjections
Mostly1st PC,But 2nd
Helps Some
77
UNC, Stat & OR
Toy Example
Warps,PCProjections
Rest isNotImportant
78
UNC, Stat & OR
Toy Example
Horizontal Var’n Visualization Challenge:
(Complicated) Warps Hard to Interpret
Approach:
Apply Warps to Template Mean
(PCA components)
79
UNC, Stat & OR
Toy Example
WarpCompon’ts(+ Mean)Applied toTemplateMean
80
UNC, Stat & OR
Refined Calculations
Current Development:
Better Exploit Manifold Data Space:
Principal Nested Spheres
81
UNC, Stat & OR
PNS on SRVF Sphere
Toy Example
Tangent Space
PCA
(on Horiz. Var’n)
Thanks to Xiaosun Lu
82
UNC, Stat & OR
PNS on SRVF Sphere
Toy Example
PNS Projections
(Fewer Modes)
83
UNC, Stat & OR
PNS on SRVF Sphere
Toy Example
Tangent Space
PCA
Note: 3 Comp’s
Needed for This
84
UNC, Stat & OR
PNS on SRVF Sphere
Toy Example
PNS Projections
Only 2 for This
85
UNC, Stat & OR
TIC testbed
Serious Data Challenge:
TIC (Total Ion Count) Chromatograms
Modern type of “chemical spectra”
Thanks to
Peter
Hoffmann
86
UNC, Stat & OR
TIC testbed
Raw Data: 15 TIC Curves (5 Colors)
87
UNC, Stat & OR
TIC testbed
Special Feature: Answer Key of Known Peaks
Found by MajorTime &LaborInvestment
88
UNC, Stat & OR
TIC testbed
Special Feature: Answer Key of Known Peaks
Goal:FindWarpsTo AlignThese
89
UNC, Stat & OR
TIC testbed
Fisher – Rao Alignment
90
UNC, Stat & OR
TIC testbed
Fisher – Rao Alignment
Spike-InPeaks
91
UNC, Stat & OR
TIC testbed
Next Zoom in on This Region
92
UNC, Stat & OR
TIC testbed
Zoomed Fisher – Rao Alignment
93
UNC, Stat & OR
TIC testbed
Before Alignment
94
UNC, Stat & OR
TIC testbed
Next Zoom in on This Region
95
UNC, Stat & OR
TIC testbed
Zoomed Fisher – Rao Alignment
96
UNC, Stat & OR
TIC testbed
Before Alignment
97
UNC, Stat & OR
TIC testbed
Next Zoom in on This Region
98
UNC, Stat & OR
TIC testbed
Zoomed Fisher – Rao Alignment
99
UNC, Stat & OR
TIC testbed
Before Fisher-Rao Alignment
100
UNC, Stat & OR
TIC testbed
Next Zoom in on This Region
101
UNC, Stat & OR
TIC testbed
Zoomed Fisher – Rao Alignment
102
UNC, Stat & OR
TIC testbed
Zoomed Fisher – Rao Alignment
Note:VeryChallenging
103
UNC, Stat & OR
TIC testbed
Before Alignment
104
UNC, Stat & OR
TIC testbed
Next Zoom in on This Region
105
UNC, Stat & OR
TIC testbed
Zoomed Fisher – Rao Alignment
106
UNC, Stat & OR
TIC testbed
Before Alignment
107
UNC, Stat & OR
TIC testbed
Warping Functions
108
UNC, Stat & OR
References for Much More
Big Picture Survey:Marron, Ramsay, Sangalli & Srivastava (2014)
TIC Proteomics Example:Koch, Hoffman & Marron (2014)
109
UNC, Stat & OR
Take Away Message
Curve Registration is Slippery Thus, Careful Mathematics is Useful Fisher-Rao Approach:
Gets the Math Right Intuitively Sensible Computable Generalizable Worth the Complication