Linear Least Squares
and its applications in distance matrix
methods
Presented by Shai Berkovich
June, 2007
Seminar in Phylogeny, CS236805
Based on the paper by Olivier Gascuel
Contents
Background and Motivation
LS in general
LS in phylogeny
UNJ algorithm
LS sense of UNJ
Distance Matrix Methods
A major family of phylogenic methods has been the distance matrix methods. The general idea: calculate a measure of the distance between each pair of species, and then find a tree that predicts the observed set of distances as closely as possible.
Distance Matrix Methods
This lives out all information from higher-order combinations of character states, reducing the data matrix to a simple table of pairwise distances, though computer simulation studies show that the amount of information of phylogeny that is lost is remarkably small.
(As we already saw: Neighbor-Joining and its robustness to noise.)
Additivity
Definition: A distance matrix D is additive if there exists a tree with positive edge weights such that where vk are the edges in the path between species i and j.
Theorem [Waterman et. al., 1977]: Given ad additive n x n distance matrix D there is a unique edge-weighted tree (without nodes of degree 2) in which n nodes in the tree are labeled s1,s2,…,sn so that the path between si and sj is equal to Dij. Furthermore, this unique tree consistent with D is reconstructable in O(n2) time.
ij kD v
Distance-Based reconstruction
Input: distance matrix D
Output: edge-weighted tree – T ( if D is
additive, then DT = D,
otherwise, return a tree best ‘fitting’
the input – D).
A B
CD
E0.05
0.10
0.070.03
0.05
0.08
0.06
A B C D E
A 0 0.23 0.16 0.20 0.17
B 0.23 0 0.23 0.17 0.24
C 0.16 0.23 0 0.15 0.11
D 0.20 0.17 0.15 0 0.21
E 0.17 0.24 0.11 0.21 0
No topology!
0ij
ij ji
ij ik kj
D
D D
D D D
Approximation
In practice, the distance matrix between molecular sequences will not be additive.
So, we want to find a tree T whose distance matrix approximates the given one.
Algorithms give exact results when operating on additive matrix, but it gets unclear when real matrix is handled.
LS Overview
Linear least squares is a mathematical optimization technique to find an approximate solution for a system of linear equations that has no exact solution. This usually happens if the number of equations (m) is bigger than the number of variables (n).
LS Overview
In mathematical terms, we want to find a solution for the "equation"
where A is a known m-by-n matrix (usually with m > n), x is an unknown n-dimensional parameter vector, and b is a known m-dimensional measurement vector.
ˆAx b
x̂
LS Overview
Euclidean norm: on Rn the notion of length of vector is captured by formula
This gives the ordinary distance from the origin to the point x.
More precisely, we want to minimize the Euclidean norm, squared of the residual Ax − b, that is, the quantity
where [Ax]i denotes the i-th component of the vector Ax. Hence the name "least squares".
2 2 2 21 1 2 2|| || ([ ] ) ([ ] ) ([ ] )m mAx b Ax b Ax b Ax b
1 2[ , ,..., ]nx x x x 2 2 21 2|| || ( ... )nx x x x
LS Overview
|| || ( ) ( ) ( ) ( ) ( )T T T T TAx b Ax b Ax b Ax Ax b Ax Ax b b b
ˆ[( ) ( ) 2( ) ] 2 2 0T T T T TdAx Ax Ax b b b A Ax A b
dx
ˆT TA Ax A b
Fact: squared norm of v is vTv
What we do when we want to minimize?
LS Overview
Note that this corresponds to a system of linear equations. The matrix ATA on the left-hand side is a square matrix, which is invertible if A has full column rank (that is, if the rank of A is n). In that case, the solution of the system of linear equations is unique and given by
1ˆ ( )T Tx A A A b
LS in phylogeny
Input: 1. Distance matrix D2. Tree topology
Supposed to receive same tree, since dissimilarity matrix is additive.
A B
CD
E0.05
0.10
0.070.03
0.05
0.08
0.06
A B C D E
A 0 0.23 0.16 0.20 0.17
B 0.23 0 0.23 0.17 0.24
C 0.16 0.23 0 0.15 0.11
D 0.20 0.17 0.15 0 0.21
E 0.17 0.24 0.11 0.21 0
The measure that we use is the measure of disperancy between the observed and expected distances:
Where wij are weights that differ between different LS methods: 1 or or
2
1 1
( ) ( )n n
ij ij iji j
Q T w D d
1/ ijD 21/ ijD
LS in phylogeny
Intuition
LS in phylogeny
,ij ij k kk
d x v A B
CD
Ev7
v2
v4v6
v3
v1
v5
2,
1 1
( ) ( )n n
ij ij ij k ki j k
Q T w D x v
12 1 2 3 4 5 6 7
13 1 2 3 4 5 6 7
45 1 2 3 4 5 6 7
1 1 0 0 0 0 1
1 0 1 0 0 1 0
...
0 0 0 1 1 1 1
d v v v v v v v
d v v v v v v v
d v v v v v v v
introduce an indicator variable , which is 1 if branch lies in the path from species i to species j and 0 otherwise
,ij kxkv
,1 1
2 ( ) 0n n
ij ij ij k ki j kk
dQw D x v
dv
2,
1 1
( ) ( )n n
ij ij ij k ki j k
Q T w D x v
LS in phylogeny
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
4 1 1 1 1 2 2
1 4 1 1 1 2 2
...
2 3 2 3 2 4 6
AB AC AD AE
AB BC BD BE
AB AD BC CD BE DE
D D D D v v v v v v v
D D D D v v v v v v v
D D D D D D v v v v v v v
prop. 1
LS in phylogeny
Number of equations as number of edges => havingone solution if the matrix is fully column-ranked. What matrix?
AB
AC
AD
AE
BC
BD
BE
CD
CE
DE
D
D
D
D
Dd
D
D
D
D
D
1 1 0 0 0 0 1
1 0 1 0 0 1 0
1 0 0 1 0 0 1
1 0 0 0 1 1 0
0 1 1 0 0 1 1
0 1 0 1 0 0 0
0 1 0 0 1 1 1
0 0 1 1 0 1 1
0 0 1 0 1 0 0
0 0 0 1 1 1 1
X
( )T TX D X X v
1( )T Tv X X X D
LS in phylogeny
Example: A
B
Cv2
v1
v3
A B C
A 0 10 12
B 10 0 8
C 12 8 0
1 1 0
1 0 1
0 1 1
X
2 1 1
1 2 1
1 1 2
TX X
1
3 1 1
4 4 41 3 1
( )4 4 41 1 3
4 4 4
TX X
1( ) ...T Tv X X X D
LS in Phylogeny
When we have weighted LS, then previous equations can be written:
where W is a diagonal matrix with distance weights on main diagonal.
( )T TX WD X WX v
1( )T Tv X WX X WD
Simulations usually shows that LS better performance then NJ
LS in Phylogeny
One can imagine an LS method that, for each tree topology, formed the matrix, inverted it and obtained the estimates. This can be done, but its computationally burdensome, even if not all topologies are examined.
Inversion of matrix: O(n3) for a tree with n tips In principle each tree topology should be
considered.
UNJ algorithm
Recall NJ algorithm:
1. Begin with star tree & all sequences as nodes in L2. Find pair of nodes with minimum QA,B
3. Create & insert new join (node K) w/ branch lengths
dA,K = ½ (dA,B + rA – rB)dB,K = ½ (dA,B + rB – rA)
4. For remaining nodes, update distance to K as
dK,C = ½ (dA,C + dB,C – dA,B)
5. Insert K and remove A, B from L6. Repeat steps 2-5 until only two nodes left UNJ
UNJ algorithm
Although the NJ algorithm is widely used and has yielded satisfactory simulation results, certain questions remain:
Proof of correctness of selection criterion (Saito & Nei) was contested but complete proof is still not provided.
NJ reduction formula gives identical importance to nodes x and y, even if one corresponds to a group of several objects and the other is single object.
UNJ algorithm
The manner in which the edge lengths are estimated is inexact in terms of LS when the agglomerated nodes represent not individual object but rather groups of objects.
The paper provides answers to this questions but we will concentrate on the last one.
Weighted/Unweighted
misunderstood
UNJ algorithm
Definitions• E = {1,2…,n} a set of n objects (leaves)• dissimilarity matrix over E• by removing the edge from T we
constitute bipartition where X may be viewed in two ways: as a subset of E or as a rooted subtree of T whose root is situated at the extremity of edge
• T denotes any valued tree• T` denotes its structure
( )ij
{ , }X X
UNJ algorithm
Definitions• cardinality of X – number of leaves in the
subtree X, also denoted as nx
• S = (sij) is an adjusted tree generated by LS
• S`tree structure associated with adjusted tree
• Let and be two bipartitions of S` when then
and
{ , }X X { , }Y YX Y
,XY ij
i X j Y
s s
,
1XY ij
i X j Yx y
s sn n
UNJ algorithm
Definitions• as well as
and
• flow of a rooted subtree X
and
,XY ij
i X j Y
,
1XY ij
i X j Yx yn n
X ixi X
f s
1X X
x
f fn
Our Model
Estimates are unbiased i.e. for every i,j
where the noise variables are i.i.d (result of real observations and measurements)
The paper states that it is coherent to use an unweighted approach which allocates the same level of importance to each of the initial objects. Furthermore, within this model it is justified to use the “ordinary” LS criterion as opposed to “generalized”, which takes into account variances and covariances of the estimates
( )ijij ij ijd
ij
Statistics
UNJ algorithm1. Initialize the running matrix: 2. Initialize the number of remaining nodes: r<-n3. Initialize the numbers of objects per node:4. While the number of nodes r is greater than 3:
{ Compute the sums Find the pair {x,y} to be agglomerated by
maximizing Qxy (3)
Create the node u, and set: Estimate the lengths of edges (x,u) and (y,u) using (2)Reduce the running matrix using (3)Decrease the number of nodes: r<-r-1 }
5. Create a central node and compute the last three edge-lengths using (2)
6. Output the tree found
( ) ( )ij ij
1, {1,... }in i n
, {1,... }iR i r
u x yn n n Reverse
of NJ
fin
UNJ algorithm
1. Selection criterion
2. Estimation formula
3. Reduction formula
( 2) ,xy x y xyQ R R r 1
r
z zii
R
1,
1 1ˆ ( )2 2( )
r
xu xy i xi yiiux y
d nn n
dyu obtained by
symmetry
ˆ ˆ ,ui x xi y yi x xu y yuw w w d w d , yxx y
u u
nnw w
n n
(1)
(2)
(3)
NJ
We won’t prove (1) and (3)
Conservation Property 1
Given dissimilarity matrix, S adjusted tree and S`its structure we have (Vach 1989):For every bipartition of S`we have (and )
Proof: we saw that
Lets have a closer look on a matrices
( )ij
{ , }X X
XX XXs XX XXs
( )T TX D X X v
( )T TX D X Xv
Conservation Property 1
2
nm
Let n be a number of leaves, q=2n-3 be a number of edges and be a number of distances
Xv - mx1 matrix of tree paths between the leaves
D - mx1 matrix of dissimilarity distancesXtXv - qx1 matrix of all “interleave” paths that
pass over the given edgeXtD - qx1 matrix of all distances that pass over
the given edge (slide no. 16). Property is established.
( )T TX D X Xv
Conservation Property 2
For all ternary nodes of S` and for every pair X,Y of subtrees associated with this node, we have ( and )
Proof:according to prop. 1 we have:
(*)
XY XYs XY XYs
X
Y
ZuXX XXs ZZ ZZs YY YYs
XY XZ XY XZs s
X Y Z
Conservation Property 2
Property is established.
XY XZ XY XZ
XY YZ XY YZ
XZ YZ XZ YZ
s s
s s
s s
XY XYs XY XYs
Formula (2) is correct
Using definition of SXY we can
rewrite:
Using prop.2 we can write:
by solving this equations we obtain
X
Y
Zu( )XY Y X Y X xu yu X Ys n f n n s s n f
( )
( )
( )
XY Y X Y X xu yu X Y
XZ Z X Z X xu zu X Z
ZY Y Z Y Z zu yu Z Y
n f n n s s n f
n f n n s s n f
n f n n s s n f
,
1 1 1
2 2 2x u XY XZ YZ Xs f (4)
Formula (2) is correct
Let us consider an agglomerative procedure as described in algorithm: at p-th step it remains to resolve r=n-p+1 nodes when some of them are subtrees. After choosing x and y Z can be viewed as joint of r-2 subtrees, some of them consist of root.
X Y Z
,I X Y
Z I
Formula (2) is correct
Thus, we may rewrite expression (4) as:
where
,
1 1( )
2 2( )
r
xu xy i XI YI XI X Yu
S n fn n
,U X Y I
I X Y
n n n n n
(5)
, ,
, , ,
1 1 1 1 1( )
2 2 2
1 1 1 1( ) ( )
2( ) 2( )
XZ YZ ij iji X j Z i Y j ZX Z Y Z
XI YI XI YI II X Y I X Y I X YX Y X Y u
n n n n
nn n n n n n n
(*)
Formula (2) is correct
Now we will prove by induction the next two statements:
For each iteration of algorithm:(a) for every resolved subtree I,J(b) Formula (2) and equation (5) are equal
Important: at each step (b) evaluation is based on (a) result from previous step and (a) evaluation is based on (b) result at current step.
, ,i j I J I Jf f
Formula (2) is correct
Base: at the first step the “weight” of each node is 1
Thus, for each node i fi=0 =>
(b) also achieved because in first iteration.
Step: Let us consider that (a) and (b) maintained during the step p. Now we’ll show that they are also maintained at step p+1.
ij ij
, , ,i j i j I J I Jf f
Formula (2) is correct
(a) We must check that hypothesis is maintained for new node u:
Thus, formula (1) maintained.
ˆ ˆ
( ) ( )
1 1 1 1( ) ( )
1 1( )
1 1( ( ) )
ui x xi y yi x xu y yu
x XI X I y YI Y I x xu y yu
y yx xXI YI X Y
u x i u y i u x u y
y yx xI I xu yu
u i u i u u
yxUI X Y xu yu
u u u i
w w w d w d
w f f w f f w s w s
n nn nf f
n n n n n n n n n n
n nn nf f s s
n n n n n n
nnf f s s f
n n n n
UI U If f
Formula (2) is correct
(b) We prove correctness of (b) for step p+2:
(b) is correct.
,
,
,
, ,
1 1( )
2 2( )
1 1( ) ( )
2 2( )
1 1 1( ) ( )
2 2( ) 2
1 1
2( ) 2( )
1ˆ (2
r
xu xy i XI YI XI X Yu
r
xy X Y i xi X I yi Y I XI X Yu
r
xy i xi yi I X Y XI X Yu
r r
i X i YI X Y I X Yu u
xu Y X
S n fn n
f f n f f f f fn n
n f f fn n
n f n fn n n n
d f f
1
) ( ).2 X Yf f
UNJ algorithm - implications
The complexity in time of UNJ is O(n3) The property we proved may be exploited
within an algorithm in O(n2), allowing the LS estimation of edge lengths of any fixed structure binary tree
I.e. finding tree topology and then LS edge estimates in O(n3) + O(n2)
UNJ
UNJ algorithm - implications
This new version derives from the original version of Saitou & Nei(1987) (weighted version) and also Vach(1989) (concerning lengths estimation). The simulation shows that UNJ suppresses NJ when data closely follow the chosen model. For certain tree structures, we obtain up to 50% error reduction, in terms of ability to recover the true tree structure.
Top Related