Post on 19-Oct-2020
Learning Topic Models– Going Beyond SVD
Ankur Moitra, IAS
joint with Sanjeev Arora and Rong Ge
October 21, 2012
Ankur Moitra (IAS) LDA October 21, 2012
Topic Models
Large collection of articles, say from the New York Times:
newspaper articles
Question
How can we automatically organize them by topic? (unsupervisedlearning)
Challenge: Develop tools for automatic comprehension of data - e.g.newspaper articles, webpages, images, genetic sequences, userratings...
Topic Models
Large collection of articles, say from the New York Times:
newspaper articles
Question
How can we automatically organize them by topic? (unsupervisedlearning)
Challenge: Develop tools for automatic comprehension of data - e.g.newspaper articles, webpages, images, genetic sequences, userratings...
Topic Models
Large collection of articles, say from the New York Times:
newspaper articles
Question
How can we automatically organize them by topic? (unsupervisedlearning)
Challenge: Develop tools for automatic comprehension of data - e.g.newspaper articles, webpages, images, genetic sequences, userratings...
topics
per
sonal
fin
ance
bas
ebal
l
... movie
rev
iew
s
wo
rds
A
W
topics
document #1: (1.0, personal finance)per
sonal
fin
ance
bas
ebal
l
... movie
rev
iew
s
wo
rds to
pic
s
documents
A
topics
document #1: (1.0, personal finance)per
sonal
fin
ance
bas
ebal
l
... movie
rev
iew
s
wo
rds
=
topic
s
documents
A W
W
topics
document #2: (0.5, baseball); (0.5, movie reviews)
per
sonal
fin
ance
bas
ebal
l
... movie
rev
iew
s
wo
rds
=
topic
s
documents
A
topics
document #2: (0.5, baseball); (0.5, movie reviews)
per
sonal
fin
ance
bas
ebal
l
... movie
rev
iew
s
wo
rds
=
topic
s
documents
A W
topics
per
sonal
fin
ance
bas
ebal
l
... movie
rev
iew
s
wo
rds
=
topic
s
documents
A W = M
topics
per
sonal
fin
ance
bas
ebal
l
... movie
rev
iew
s
wo
rds
=
topic
s
documents
A W M
So Many Models!
Pure Topics: one topic per document
[
Stochastic
Fixed
M
=
A W M
=
A W
So Many Models!
LDA: [Blei et al] Dirichlet distribution
Stochastic
Fixed
M
=
A W M
=
A W
So Many Models!
CTM / Pachinko: structured correlations
Stochastic
Fixed
M
=
A W M
=
A W
Algorithms
Maximum Likelihood: Find the parameters that maximize thelikelihood of generating the observed data.
Hard to compute!
Spectral: Compute the singular value decomposition of M̂ .[Papadimitriou et al], [Azar et al], ...
But the singular vectors are orthonormal!
Question
Can we use tools from nonnegative matrix factorization instead ofspectral methods?
[AGKM]: fixed parameter intractable but there are easy cases
Algorithms
Maximum Likelihood: Find the parameters that maximize thelikelihood of generating the observed data.
Hard to compute!
Spectral: Compute the singular value decomposition of M̂ .[Papadimitriou et al], [Azar et al], ...
But the singular vectors are orthonormal!
Question
Can we use tools from nonnegative matrix factorization instead ofspectral methods?
[AGKM]: fixed parameter intractable but there are easy cases
Algorithms
Maximum Likelihood: Find the parameters that maximize thelikelihood of generating the observed data.
Hard to compute!
Spectral: Compute the singular value decomposition of M̂ .[Papadimitriou et al], [Azar et al], ...
But the singular vectors are orthonormal!
Question
Can we use tools from nonnegative matrix factorization instead ofspectral methods?
[AGKM]: fixed parameter intractable but there are easy cases
Algorithms
Maximum Likelihood: Find the parameters that maximize thelikelihood of generating the observed data.
Hard to compute!
Spectral: Compute the singular value decomposition of M̂ .[Papadimitriou et al], [Azar et al], ...
But the singular vectors are orthonormal!
Question
Can we use tools from nonnegative matrix factorization instead ofspectral methods?
[AGKM]: fixed parameter intractable but there are easy cases
Algorithms
Maximum Likelihood: Find the parameters that maximize thelikelihood of generating the observed data.
Hard to compute!
Spectral: Compute the singular value decomposition of M̂ .[Papadimitriou et al], [Azar et al], ...
But the singular vectors are orthonormal!
Question
Can we use tools from nonnegative matrix factorization instead ofspectral methods?
[AGKM]: fixed parameter intractable but there are easy cases
Our Results
Let E [WW T ] = R be the topic-topic covariance matrix, let κ be its
condition number and let a = maxi ,jE [Wi ]E [Wj ]
be the topic imbalance.
If the topic matrix A satisfies the “anchor word assumption” forp > 0:
Theorem
We can learn the topic matrix A and covariance matrix R to withinaccuracy ε in time and number of docs poly(log n, r , 1/ε, 1/p, κ, a)with n words and r topics
Suffices to have documents of size two!
Our Results
Let E [WW T ] = R be the topic-topic covariance matrix, let κ be its
condition number and let a = maxi ,jE [Wi ]E [Wj ]
be the topic imbalance.
If the topic matrix A satisfies the “anchor word assumption” forp > 0:
Theorem
We can learn the topic matrix A and covariance matrix R to withinaccuracy ε in time and number of docs poly(log n, r , 1/ε, 1/p, κ, a)with n words and r topics
Suffices to have documents of size two!
Our Results
Let E [WW T ] = R be the topic-topic covariance matrix, let κ be its
condition number and let a = maxi ,jE [Wi ]E [Wj ]
be the topic imbalance.
If the topic matrix A satisfies the “anchor word assumption” forp > 0:
Theorem
We can learn the topic matrix A and covariance matrix R to withinaccuracy ε in time and number of docs poly(log n, r , 1/ε, 1/p, κ, a)with n words and r topics
Suffices to have documents of size two!
Our Results
Let E [WW T ] = R be the topic-topic covariance matrix, let κ be its
condition number and let a = maxi ,jE [Wi ]E [Wj ]
be the topic imbalance.
If the topic matrix A satisfies the “anchor word assumption” forp > 0:
Theorem
We can learn the topic matrix A and covariance matrix R to withinaccuracy ε in time and number of docs poly(log n, r , 1/ε, 1/p, κ, a)with n words and r topics
Suffices to have documents of size two!
If an anchor word (for a topic) occurs, the document is at leastpartially about the given topic:
bunt
401k oscar-‐winning
topics
per
son
al f
inan
ce
bas
ebal
l
... mov
ie r
evie
ws
word
s topic
s
documents
A W
Each topic has an anchor word that occurs with probability ≥ p
If an anchor word (for a topic) occurs, the document is at leastpartially about the given topic:
bunt
401k oscar-‐winning
topics
per
son
al f
inan
ce
bas
ebal
l
... mov
ie r
evie
ws
word
s topic
s
documents
A W
Each topic has an anchor word that occurs with probability ≥ p
Anchor Words as Extreme Points [AGKM]
=
A W M
Can we efficiently determine
if a word is an anchor word?
Anchor Words as Extreme Points [AGKM]
=
A W M
Can we efficiently determine
if a word is an anchor word?
Anchor Words as Extreme Points [AGKM]
=
A W M
Can we efficiently determine
if a word is an anchor word?
Anchor Words as Extreme Points [AGKM]
=
A W M
Can we efficiently determine
if a word is an anchor word?
Anchor Words as Extreme Points [AGKM]
=
A W M
Can we efficiently determine
if a word is an anchor word?
Anchor Words as Extreme Points [AGKM]
=
A W M
Can we efficiently determine
if a word is an anchor word?
Anchor Words as Extreme Points [AGKM]
=
A W M
Can we efficiently determine
if a word is an anchor word?
Problem: Sampling “Noise”
A W M
Can we efficiently determine
if a word is an anchor word?
M
Problem: Sampling “Noise”
MA W M
Can we efficiently determine
if a word is an anchor word?
Problem: Sampling “Noise”
A W M
Can we efficiently determine
if a word is an anchor word?
M
Our Algorithm
M̂ is far from M , but let’s use M̂M̂T instead!
M̂M̂T → MMT and WW T → R as number of documents increase
Step
We can recover the anchor words from MMT .
Step
We can recover A and R given MMT and the anchor words.
And we can use matrix perturbation bounds to quantify how erroraccumulates
Our Algorithm
M̂ is far from M , but let’s use M̂M̂T instead!
M̂M̂T → MMT and WW T → R as number of documents increase
Step
We can recover the anchor words from MMT .
Step
We can recover A and R given MMT and the anchor words.
And we can use matrix perturbation bounds to quantify how erroraccumulates
Our Algorithm
M̂ is far from M , but let’s use M̂M̂T instead!
M̂M̂T → MMT and WW T → R as number of documents increase
Step
We can recover the anchor words from MMT .
Step
We can recover A and R given MMT and the anchor words.
And we can use matrix perturbation bounds to quantify how erroraccumulates
Finding the Anchor Words
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
MA W M
Can we efficiently determine
if a word is an anchor word?
M
A
W
M
Can we efficiently determine
if a word is an anchor word?
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
A W M
Can we efficiently determine
if a word is an anchor word?
MA W M
Can we efficiently determine
if a word is an anchor word?
M
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
Finding the Anchor Words
Nonnegative!
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
MA W M
Can we efficiently determine
if a word is an anchor word?
M
A
W
M
Can we efficiently determine
if a word is an anchor word?
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
A W M
Can we efficiently determine
if a word is an anchor word?
MA W M
Can we efficiently determine
if a word is an anchor word?
M
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
Finding the Anchor Words
Nonnegative!
Anchor words from:
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
MA W M
Can we efficiently determine
if a word is an anchor word?
M
A
W
M
Can we efficiently determine
if a word is an anchor word?
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
A W M
Can we efficiently determine
if a word is an anchor word?
MA W M
Can we efficiently determine
if a word is an anchor word?
M
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i!!
! !
! !
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
Our Algorithm
M̂ is far from M , but let’s use M̂M̂T instead!
M̂M̂T → MMT and WW T → R as number of documents increase
Step
We can recover the anchor words from MMT .
Step
We can recover A and R given MMT and the anchor words.
And we can use matrix perturbation bounds to quantify how erroraccumulates
Our Algorithm
M̂ is far from M , but let’s use M̂M̂T instead!
M̂M̂T → MMT and WW T → R as number of documents increase
Step
We can recover the anchor words from MMT .
Step
We can recover A and R given MMT and the anchor words.
And we can use matrix perturbation bounds to quantify how erroraccumulates
Using the Anchor Words
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
Using the Anchor Words
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
Using the Anchor Words
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
Using the Anchor Words
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
Using the Anchor Words
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
Using the Anchor Words
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
DRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
iDRA
A
WT
AT
W(D)
(R)(U)
DR1 = DRAT1
find :
DRD diag( ) = DR1
z
z
DRUT
DRD
DERUT
1 + eps1 ! eps
output: (DRD diag(z))T!1
= M MT
DERE DT
{(E ) 1}T !1
i
Our Algorithm
M̂ is far from M , but let’s use M̂M̂T instead!
M̂M̂T → MMT and WW T → R as number of documents increase
Step
We can recover the anchor words from MMT .
Step
We can recover A and R given MMT and the anchor words.
And we can use matrix perturbation bounds to quantify how erroraccumulates
Our Algorithm
M̂ is far from M , but let’s use M̂M̂T instead!
M̂M̂T → MMT and WW T → R as number of documents increase
Step
We can recover the anchor words from MMT .
Step
We can recover A and R given MMT and the anchor words.
And we can use matrix perturbation bounds to quantify how erroraccumulates
Concluding Remarks
joint work with Arora, Ge, Halpern, Mimno, Sontag, Wu and Zhu
We ran our algorithm on a database of 300,000 New York Timesarticles (from the UCI database) with 30,000 distinct words
Run time: 12 minutes (compared to 10 hours for MALLET andother state-of-the-art topic modeling tools)
Topics are high quality (Ask me if you want to see the results!)
Independently, [Anandkumar et al] gave an algorithm for LDAwithout any assumptions!
Are there other trapdoors – like anchor words – that make machinelearning much easier?
Concluding Remarks
joint work with Arora, Ge, Halpern, Mimno, Sontag, Wu and Zhu
We ran our algorithm on a database of 300,000 New York Timesarticles (from the UCI database) with 30,000 distinct words
Run time: 12 minutes (compared to 10 hours for MALLET andother state-of-the-art topic modeling tools)
Topics are high quality (Ask me if you want to see the results!)
Independently, [Anandkumar et al] gave an algorithm for LDAwithout any assumptions!
Are there other trapdoors – like anchor words – that make machinelearning much easier?
Concluding Remarks
joint work with Arora, Ge, Halpern, Mimno, Sontag, Wu and Zhu
We ran our algorithm on a database of 300,000 New York Timesarticles (from the UCI database) with 30,000 distinct words
Run time: 12 minutes (compared to 10 hours for MALLET andother state-of-the-art topic modeling tools)
Topics are high quality (Ask me if you want to see the results!)
Independently, [Anandkumar et al] gave an algorithm for LDAwithout any assumptions!
Are there other trapdoors – like anchor words – that make machinelearning much easier?
Concluding Remarks
joint work with Arora, Ge, Halpern, Mimno, Sontag, Wu and Zhu
We ran our algorithm on a database of 300,000 New York Timesarticles (from the UCI database) with 30,000 distinct words
Run time: 12 minutes (compared to 10 hours for MALLET andother state-of-the-art topic modeling tools)
Topics are high quality (Ask me if you want to see the results!)
Independently, [Anandkumar et al] gave an algorithm for LDAwithout any assumptions!
Are there other trapdoors – like anchor words – that make machinelearning much easier?
Concluding Remarks
joint work with Arora, Ge, Halpern, Mimno, Sontag, Wu and Zhu
We ran our algorithm on a database of 300,000 New York Timesarticles (from the UCI database) with 30,000 distinct words
Run time: 12 minutes (compared to 10 hours for MALLET andother state-of-the-art topic modeling tools)
Topics are high quality (Ask me if you want to see the results!)
Independently, [Anandkumar et al] gave an algorithm for LDAwithout any assumptions!
Are there other trapdoors – like anchor words – that make machinelearning much easier?
Questions?
Thanks!