How works

31
How works M. Ram Murty, FRSC Queen’s Research Chair Queen’s University or How linear algebra powers the search engine

description

How works. or How linear algebra powers the search engine. M. Ram Murty, FRSC Queen’s Research Chair Queen’s University. From: gomath.com/geometry/ellipse.php. Metric mishap causes loss of Mars orbiter (Sept. 30, 1999). The limitations of Google!. The web at a glance. - PowerPoint PPT Presentation

Transcript of How works

Page 1: How                    works

How works

M. Ram Murty, FRSCQueen’s Research ChairQueen’s University

or How linear algebra powers the search engine

Page 2: How                    works
Page 3: How                    works
Page 4: How                    works
Page 5: How                    works
Page 6: How                    works

From: gomath.com/geometry/ellipse.php

Page 7: How                    works

Metric mishap causes loss of Mars orbiter (Sept. 30, 1999)

Page 8: How                    works

The limitations of Google!

Page 9: How                    works

The web at a glance

Query-independent

PageRank Algorithm

Page 10: How                    works

The web is a directed graph

The nodes or vertices are the web pages.

The edges are the links coming into the page and going out of the page.

This graph has more than 10 billion vertices and it isgrowing every second!

Page 11: How                    works

The PageRank Algorithm

PageRank Axiom: A webpage is important if it is pointed to by other important pages.

The algorithm was patented in 2001.

Sergey Brin and Larry Page

Page 12: How                    works

Example

C has a higher rank than E, even though there are fewer links to C since the one link to C comes from an “important” page.

Page 13: How                    works
Page 14: How                    works

Mathematical formulation

Let r(J) be the “rank” of page J.

Then r(K) satisfies the equation r(K)= ΣJ→K r(J)/deg+(J), where deg+(J) is the outdegree of J.

12

Page 15: How                    works

Matrix multiplication

Factoid:The word “matrix” comes from the Sanskrit word “matr” which is the root word For “mother”. It was coined byHerman Grassman who was both aSanskrit scholar and a mathematician.

H. Grassman(1809-1877)

Page 16: How                    works

The web and Markov chains

Let puv be the probability of reaching node u from node v.

For example, pAB=1/2 and pAC=1/3 and pAE=0.

Notice the columns add up to 1.Thus, (1 1 1 1 1)P=(1 1 1 1 1). Pt has eigenvalue 1

P is called the transition matrix.

Page 17: How                    works

Markov process If a web user is on page C, where will she

be after one click? After 2 clicks? … After n clicks?

A.A. Markov (1856-1922)

After n steps, Pnp0.

Page 18: How                    works

Eigenvalues and eigenvectors A vector v is called an eigenvector of a matrix

P if Pv = λv for some number λ. The number λ is called an eigenvalue. One can determine practically everything

about P from the knowledge of its eigenvalues and eigenvectors.

The study of such objects is called linear algebra and this subject is about a 100 years old.

Page 19: How                    works

Eigenvalues and eigenvectors of P

Therefore, P and Pt have the same eigenvalues.

In particular, P also has an eigenvalue equal to 1.

Page 20: How                    works

Theorem of Frobenius All the eigenvalues of the

transition matrix P have absolute value ≤ 1.

Moreover, there exists an eigenvector corresponding to the eigenvalue 1, having all non-negative entries.

Georg Frobenius (1849-1917)

Page 21: How                    works

Perron’s theorem Theorem (Perron): Let

A be a square matrix with strictly positive entries. Let λ* = max{ |λ|: λ is an eigenvalue of A}.

Then λ* is an eigenvalue of A of multiplicity 1 and there is an eigenvector with all its entries strictly positive. Moreover, |λ|< λ* for any other eigenvalue.

O. Perron (1880-1975)

Page 22: How                    works

Frobenius’s refinement

Call a matrix A irreducible if An has strictly positive entries for some n.

Theorem (Frobenius): If A is an irreducible square matrix with non-negative entries, then λ* is again an eigenvalue of A with multiplicity 1. Moreover, there is a corresponding eigenvector with all entries strictly positive.

Page 23: How                    works

Why are these theorems important? We assume the following concerning the matrix P: (a) P has exactly one eigenvalue with absolute

value 1 (which is necessarily =1); (b) The corresponding eigenspace has dimension

1; (c) P is diagonalizable; that is, its eigenvectors

form a basis. Under these hypothesis, there is a unique

eigenvector v such that Pv = v, with non-negative entries and total sum equal to 1.

Frobenius’s theorem together with (a) implies all the other eigenvalues have absolute value strictly less than 1.

Page 24: How                    works

Computing Pnp0. Let v1, v2, …, v5 be a basis of eigenvectors of P,

with v1 corresponding to the eigenvalue 1. Write p0 = a1v1 + a2v2 + … + a5v5. It is not hard to show that a1=1. Indeed, p0= a1v1 + a2v2 + … + a5v5

Let J=(1,1,1,1,1). Then 1 = J p0= a1 Jv1 + a2 Jv2 + … + a5 Jv5 Now Jv1=1, by construction and normalization. For i≥2, J(Pvi) = (JP)vi = Jvi. But Pvi = λivi. Hence λi Jvi = Jvi. Since λi ≠1, we get Jvi =0. Therefore a1=1.

18

Page 25: How                    works

Computing Pnp0 continued

Pnp0= Pnv1 + a2Pnv2 + … + a5Pnv5

= v1+ λ2n a2v2+ … + λ5

n a5v5. Since the eigenvalues λ2, …, λ5 have

absolute value strictly less than 1, we see that Pnp0→v1 as n tends to infinity.

Moral: It doesn’t matter what p0 is, the stationary vector for the Markov process is v1.

Page 26: How                    works

Returning to our example …

The vector (12, 16, 9, 1, 3) is an eigenvector of P with eigenvalue 1.

We can normalize it by dividing by 41 so that the sum of the components is 1.

But this suffices to give the ranking of the nodes:B, A, C, E, D.

20

Page 27: How                    works

How to compute the eigenvector We can apply the power method:

Compute Pnp0 for very large n to get an approximation for v1.

This is called the power method and there are efficient algorithms for this large matrix computation.

It seems usually 50 iterations (i.e. n=50) are sufficient to get a good approximation of v1.

Page 28: How                    works

Improved PageRank If a user visits F, then she is

caught in a loop and it is not surprising that the stationary vector for the Markov process is (0,0,0,0,0, ½, ½ )t.

To get around this difficulty, the authors of the PageRank algorithm suggest adding to P a stochastic matrix Q that represents the “taste” of the surfer so that the final transition matrix is P’ =xP + (1-x)Q for some 0≤x≤1.

Note that P’ is again stochastic. One can take Q=J/N where N is

the number of vertices and J is the matrix consisting of all 1’s.

Brin and Page suggested x=.85 is optimal.

22

Page 29: How                    works

References

Mathematics and Technology, by C. Rousseau and Y. Saint-Aubin, Springer, 2008. available online through Queen’s library.

Google’s PageRank and Beyond, The Science of Search Engines, A. Langville and C. Meyer, Princeton University Press, 2006.

The 25 billion dollar eigenvector, K. Bryan and T. Liese, SIAM Review, 49 (2006), 569-581.

Page 30: How                    works

Mathematical genealogy

P.L.Chebychev(1821-1894) A.A.Markov

(1856-1922)J.D.Tamarkin(1888-1945) D.H. Lehmer

(1905-1991)

H.M. Stark(1939-

Page 31: How                    works

Thank you for your attention.

Have a day!