L23: PageRank

Je↵ M. Phillips

April 13, 2020

Final Report

At most 4 pages/student. Don’t cram in too much!

I Succinct title (and names)

I Problem definition and motivation.

I Explain your Data.

I key idea

I What did you do (which techniques, an implementation, acomparison, an extension)

I What did you learn? Artifacts (charts, plots, examples, math)and Intuition (in words, did it work?)

I ← Same ten Posters

o

webpage-simia.it( Search ). InvertedIndex

year ;i¥¥ . .

bout→ page 't

, P ?,"

car"

→¥pQgirl)

-

Define most relevantwebpages-

D- d-'

ne'EE¥÷÷÷÷:÷:i:smarm :÷÷÷i÷"¥xE÷÷g÷÷:X

Crawlers : program ; that walksaround web : ① read page

update fentwiuecfor

② follow random

inverting hyperlinks

use hyperlink infoL a hired "

www.pie.com" )pie

- 3

Spammersbuild fleet pages i link to your

Page w/ hyperlink tag .

studies : Alternative to search Engine

Yahoo ! and Look Smart

Built an organized , curatedcollection of websites

pageR-ankfslpjiturml-fktytg.fi?ggignhspiddeieafe

•

1ttqitlinkedto

balanceboth pages .

•

page is important ifa

random MCMC' ' random surfer "

were

to fond it.

-

Web is a big graph G- L V,

E)

V = I set do all pages }

F- = l Ei ; = link pi → B- 3

Petone ML → g* ⇐ converged to vectordistribution

q*Lj ) says how important page ; is .

Compute q* of lwepgraph-

• Keep truck of crawlers : how frequentreturn .

• Buy bag computer : Compute eigcp )← probtra.ca)

• Precompute P#

=P - P - p .

. . ..

. p

← too big°

q*= go ←last night

a :÷÷÷o÷÷÷t÷÷¥power

method

Anatomy of Web

Strongly ConnectedComponent

IN

OUTTubes

tendrilsOUT

tendrilsIN

disconnected

ANATOMY of WEB

is this G ergodic

÷ . . Is•

i'④ yo

Anatomy of Web

0

O

canwemaheGergodi.ci#• Teleportation ,/ taxation

→ about once every I steps

→ jumpto randomnode

.

P prob trans (G)13--0-15 p

R= a - is ) Pepe Q¥fIf¥¥Z

↳ dense

Rq . - Ul - B) Ptp'

G)ginLt B) Pain + qt nxt vector- -

→

Spam Farms

÷ .

Google counter

^seafhs¥at

Trust ( noes ? )

only teleport to trustedpages .

r ← qx , pagerank

t ← get trusted teleport

rts ) - ft ; ) if l arse → spam

↳ truthfulness of webpage

Word CountConsider as input all of English Wikipedia stored in DFS. Goal is tocount how many times each word is used.

Inverted IndexConsider as input all of English Wikipedia stored in DFS. Goal is tobuild an index, so each word has a list of pages it is in.

PhrasesConsider as input all of English Wikipedia stored in DFS. Goal is tobuild an index, on 3-grams (sequence of 3 words) that appears onexactly one page, with link to page.

Label Propagation (Graph)Consider a large graph G = (V ,E ) (e.g., a social network), with asubset of notes V 0 ⇢ V with labels (e.g., {pos, neg}). Each nodestores its label (if any) and edges.Assign a vertex a label if (a) unlabled, (b) has � 5 labeledneighbors, (c) based on majority vote.

Label Propagation (Embedding)Consider a data set X ⇢ Rd , with a subset of points X 0 ⇢ X withlabels (e.g., {pos, neg}). Implicitly defines graph with V = X andE using k = 20 nearest neighbors.Assign a vertex a label if (a) unlabled, (b) has � 5 labeledneighbors, (c) based on majority vote.

Example PageRank

M =

2

664

0 1/2 0 01/3 0 1 1/21/3 0 0 1/21/3 1/2 0 0

3

775

Stripes:

M1 =

2

664

01/31/31/3

3

775 M2 =

2

664

1/2001/2

3

775 M3 =

2

664

0100

3

775 M4 =

2

664

01/21/20

3

775

These are stored as�1 : (1/3, 2), (1/3, 3), (1/3, 4)

�,�

2 : (1/2, 1)(1/2, 4)�,�3 : (1, 3)

�, and

�4 : (1/3, 1), (1/2, 2)

�.

Example PageRank

M =

2

664

0 1/2 0 01/3 0 1 1/21/3 0 0 1/21/3 1/2 0 0

3

775

Stripes:

M1 =

2

664

01/31/31/3

3

775 M2 =

2

664

1/2001/2

3

775 M3 =

2

664

0100

3

775 M4 =

2

664

01/21/20

3

775

These are stored as�1 : (1/3, 2), (1/3, 3), (1/3, 4)

�,�

2 : (1/2, 1)(1/2, 4)�,�3 : (1, 3)

�, and

�4 : (1/3, 1), (1/2, 2)

�.

Example PageRank

M =

2

664

0 1/2 0 01/3 0 1 1/21/3 0 0 1/21/3 1/2 0 0

3

775

Blocks:

M1,1 =

0 1/2

1/3 0

�M1,2 =

0 01 1/2

�M2,1 =

1/3 01/3 1/2

�M2,2 =

0 1/20 0

�

These are stored as�1 : (1/2, 2)

�,�2 : (1/3, 1)

�, as�

2 : (1, 3), (1/2, 4)�, as

�3 : (1/3, 1)

�,�4 : (1/3, 1), (1/2, 2)

�, and

as�3 : (1/2, 4)

�.