Recommendation Systemsand

Web Search

Eytan Adar HP Labs

Baruch Awerbuch Johns Hopkins University

Boaz Patt-Shamir Tel Aviv University

David Peleg Weizmann Institute

Mark Tuttle HP Labs

Slide 2

Recommendation systems

• eBay– Buyers and sellers rate each transaction– Users check the ratings before each transaction

• Amazon– System monitors user purchases– System recommends products based on this history

• SpamNet (a collaborative filter)– SpamNet filters mail, users identify mistakes– Users gain reputations– Reputations affect weight of recommendations

Slide 3

The problem• There are many known attacks

– Many villains boost each other’s reputation– One villain acts honest for a while

• How can honest users collaborate effectively

… in the presence of dishonest, malicious users

• What is a good model for recommendation systems?• What are algorithms can we prove something about?• We give fast, robust, practical algorithms.

Slide 4

A recommendation model

• n players, both honest and dishonest (n honest)– dishonest players can be arbitrarily malicious, collude

• m objects, both good and bad (m good)– players probe objects to learn whether good or bad– there is a cost to probing a bad object

• a public billboard– players post the results of their probes (the honest do…)– billboard is free: no cost to read or write

• We think this is a direct abstraction of eBay

Slide 5

A simple game

• At each step of the game, one player takes a turn:– consult the billboard– probe an object– post the result on the billboard

• Honest players follow the protocol• Dishonest players can collude in arbitrary

ways

Goal: help honest users find good objects at minimal cost.

Slide 6

Bad ways to play the game

• Always try the object with the highest number of positive recommendations– dishonest players recommend bad objects– honest players try all bad objects first

• Always try the object with the least number of negative recommendations– dishonest players slander good objects– honest players try all bad objects first– (a popular strategy on eBay, but quite vulnerable)

• Simple combinations also fail

Slide 7

What about other approaches?• Collaborative filtering

[Goldberg, Nichols, Oki, Terry 1998], [Azar, Fiat, Karlin, McSherry, Saia 2001], [Drineas, Kerenidis, Raghavan 2002], [Kleinberg, Sandler 2003]

– all players are honest, solutions are centralized• Web-searching algorithms [Brin, Page 1998], [Kleinberg 1999]

– compute a “transitive popular vote” of the participants– easily spammed by a clique of colluding crooks

• Trust [Kamvar, Schlosser, Garcia-Molina 2003]

– use trusted authorities to assign trust to players– do we really care about trust? we care about cost!

Simple randomized algorithms can minimize cost.

Slide 8

Our results

• A simple, efficient algorithm for many contexts– Objects come and go over time– Players access different subsets of objects– Players have different tastes for objects

• The most interesting model• Our algorithm can be substantially better than others

• Players probe few objects– Only a constant when most players are honest

• Corporate web search is the perfect application– Implemented on HP’s internal search engine

Slide 9

Good ways to play the game

• Exploration rule: – “choose a random object, and probe it”– okay if most objects are good

• Exploitation rule:– “choose a random player, and probe an object it liked”– okay if most players are honest

• But are there many good objects/honest players?!?

Slide 10

The balanced rule

• The balanced rule: flip a coin– if heads, follow exploration rule– if tails, follow exploitation rule

• How well does it work?

• We compute expected cost of probe sequence cost() = number of bad objects probed by honest players

Slide 11

• Split each time a good player finds a good object = … a b c d e d e d e e …

D0 D1 D2

• expected cost of Di =

– each probe hits good object with probability

• total expected cost is

• expected cost of D0 =

– each probe hits good object with probability

Slide 12

Understanding that number

Finding a good object

Spreading the good news

Probes by a player to learn the good news

This is just a log factor away from optimal (lower bounds later)

Slide 13

Different models

• Dynamic object model– objects come and go over time

• Partial access model– players access overlapping subsets of the objects

• Taste model– players have different notions of good and bad

• In each model, analysis has similar feel…

Slide 14

Dynamic object model

• Objects come and go with each step:– first the objects change (changes are announced to all)– then some player takes a step

• Algorithm for player p:if p has found a good object o

then p probes o

else p follows the Balanced Rule

Slide 15

Competitive analysis

• Optimal probe sequence opt has simple form: opt = a a a x x x c c c c d d d e e e …

good objectno good objects

• Optimality cost depends on the environment:– player and object schedules, honest players, good

objects– adversary à powerful, adaptive, and Byzantine!

switches(opt) = number of distinct objects in opt

• Our probe sequence partitioned by the switches: = o o o o o o o o o o o o o o o o …

Slide 16

Competitive analysis

• Compare cost of and opt switch-by-switch

cost()

· cost(opt) + switches(opt) (cost to satisfy)

· cost(opt) + switches(opt) (1/ + n log n)

· cost(opt) + switches(opt) (m + n log n)

• Lower bound (Yao’s Lemma):cost() ¸ cost(opt) + switches(opt) (m)

Slide 17

Partial access model

• Each player can access only some objectswxaybz

p

q

r good objectbad object

• common interest essential for help from neighbors– q gets help from p, but not r– r gets no useful help at all

P O

• hard to measure help from neighbors accurately– we bound work of players P with common

interest O

Slide 18

Partial access algorithm• Similar algorithm:

– choose randomly from neighbors, not all players

work until one in P finds O

work until all in P learn of O

+=

¿ m + n log n

exploration + exploitation

• Similar analysis:

• Good analysis in this model is very difficult:– we know common interest is essential for others to

help– sometimes others don’t help (players might as well

sample)even if each player has common interest with many players

Slide 19

Simulation

0

20

40

60

80

100

120

Max

Ave

Min

exploration probability

step

s

10,000 players, 50% honest10,000 objects, 1% good 1,000 objects/player

• Balanced rule tolerates coin bias quite well• Can we optimize the rule by changing the bias?

• many good objects emphasize exploration• many honest players emphasize exploitation

Slide 20

Differing tastes model

• Player tastes differ: good(player) µ objects• Player tastes overlap: good(p1) Å good(p2) ;

• Special interest group S=(P,O):

“anything in O could satisfy anyone in P”O µ good(p) 8 p 2 P

• Players don’t know which SIG they are in.• Players follow the Balanced Rule.

Slide 21

Complexity

• Theorem: Given a SIG (P,O), the expected total work to satisfy everyone in P is

• Pretty good: within a log factor when |P|=O(n)

Slide 22

A very popular model

• Most algorithms are off-line: prepare in advance– Assume underlying object set structure: The web

• PageRank [Google], HITS [Kleinberg]

– Assume underlying stochastic model: Analyze past choices• [KRRT’98], [KS’03], [AFKMS’01]

– Heuristically identify similar users/objects• GroupLens Project

• Drineas, Kerenidis, Raghavan: first on-line algorithm– Algorithm recommends, observers user response

Slide 23

DKR’02

• Assumes:– User preferences given by types (up to noise)– O(1) dominant types cover most users

• Must know the number of dominant types

– Type orthogonality: • No two types like the same object

– Type gap:• The dominant types are by far the most popular

• The algorithm:– Uses SVD to compute the types, learn user types– Runs in time O(mn)

Slide 24

The Balanced Rule

• No types: SIGs a much looser characterization• No type orthogonality: SIGs can overlap, even

“subtyping” is allowed• No type gap: SIG popularity irrelevant• A simple distributed algorithm• Tolerates malicious players• Shares work evenly if players run at similar rates• Fast: runs in time O(m + n log n)

Slide 25

A look at individual work

• Consider a synchronous, round-based model– Each player probes an object each round until satisfied– One model of players running at similar speeds– A good model for studying individual work

• Theorem: Lower bounds

• Theorem: Balanced rule halts in

Slide 26

p n + 1 objects, n honest probes

good object gets p n votes

n objects, n honest probesgood object gets a vote

How is constant even possible? honest players, 1 good objectnn

all objects

all objects with 1 vote

all objects with votes

2n

probe, probe

probe, probe

probe, probe, probe

Contains the good object

Expect the good objectat most bad objectsn

Expect the good objectat most 2 bad objects

so probe them all

Slide 27

Candidate sets work well

• Theorem: “Distilling candidate sets” is faster:– Constant 1/ rounds if n1-honest players – Even with many dishonest players, expected

rounds is only

• Remember: Balanced rule was

Slide 28

Conclusions

• Contributions– a new model for the study of reputation– simple algorithms for collaboration in spite of

• asynchronous behavior• changing objects, differing access, differing tastes• arbitrarily malicious behavior from players

• Our work generalizes to multiple values– players want a “good enough” object

• reduces to our binary values, even for multiple thresholds

– players want “the best” or “in the top 10%”• early stopping is no longer possible

Slide 29

Future work• Better lower bounds, better analysis…• We never used

– the identities of the malicious players– the negative votes on the billboard– the number of good objects or honest players

• can we get rid of the log factors if we do?

• What if the supply of each object is finite?• What if objects have prices? And the prices

affect whether a player wants the object?

Slide 30

Many interesting problems remain…

But now, an application to Intranet search…

Slide 31

xSearch: Improving Intranet search

• Internet search engines are wonderful• Intranet search is disappointing

• Some simple ideas to improve intranet search– Heuristic for deducing user recommendations– Algorithms for reordering search results

• Implemented on the @HP search engine– Getting 10% of the queries – Collecting data on performance– Results are preliminary, but look promising…

Slide 32

Who is “lampman”?

@hp says:

Slide 33

Who is “lampman”?

We say:

Slide 34

What is going on?

• Searching corporate webs should be so easy

• Google is phenomenal on the external web• Why not just run Google on the internal web?

• Because it doesn’t work, and not just at HP…

Slide 35

IBM vs Google et al (2003)

• Identified the top 200 and median 150 queries• Found the “right” answer to each query by hand • Ran Google over the IBM internal web• Now how does Google do on these queries?

– Popular queries: only 57% succeed– Median queries: only 46% succeed– Weak notion of success: “right answer in top 20 results”

• Compare that to your normal Google experience!– “Right answer in top 5 results 85% of the time”

Slide 36

Link structure is crucial

• Internet search algorithms depend on link structure:– Good pages point to great pages, etc.– Important pages are at the “center of the web”

• Link structure has a strongly connected component:

strongly connected component

incoming

pages

outgoing

pages

otherreachable

pages

30% of the Internet,10% of IBMThe meaty part of the Intranet is just a third that of the

Internet!

Slide 37

Corporate webs are different

• More structure– Small groups build large parts of web (silos)– Periodically reviewed and branded

• No outgoing links from many pages– Documents intended to be informative not “interesting”– Pages generated automatically from databases

• No reward for creating/updating/advertising pages– Do you advertise your project page?– Do you advertise your track club page?

Slide 38

Corporate queries are different

• Queries are focused and have few “right” answers

• What is “vacation” looking for?– Inside HP:

• what are the holidays and how to report vacation time• one or two pages on the HR web.

– Outside HP: • interesting vacation spots or vacation deals• any number of answers pages would be satisfactory

Slide 39

What does @hp do now?

• Based on the Verity Ultraseek search engine– Classical information retrieval on steroids– Many tunable parameters and knobs

• Manually constructs “best bets” for popular queries• Works as well as any other intranet search engine

Slide 40

Our approach: user collaboration

@hp user

interface

our reorderin

g heuristic

s

@hp search engine

query

hits

How to collect user feedback?

How to use user feedback?

query’

hits’

Slide 41

Collecting feedback: last clicks• The “last click” heuristic:

– Assume the last link the user clicks is the right link– Assume users stop hunting when they find the right page

• Easy, unobtrusive implementation:– Tag each query by a user with a session id– Log each link clicked on with the session id – Periodically scan the log to compute the “last clicks”

• Effective heuristic: – Agreement: 44% of last clicks go to same page – Quality: 72% of last clicked pages are good pages

Slide 42

Last clicks example: “mis”

455

http://persweb.corp.hp.com/comp/employee/program/tr/sop/mis_guide.htm

78 http://hrexchange.corp.hp.com/HR_News/newslink040204-sopenroll.htm

13 http://hrexchange.corp.hp.com/HR_News/newslink100303-SOPenroll.htm

10 http://dimpweb.dublin.hp.com/factsyseng/mis/

5 http://hrexchange.corp.hp.com/HR_News/newslink121203-stock_holidayclosure.htm

2 http://canews-stage.canada.hp.com/archives/april04/SOP/index.asp

2 http://canews.canada.hp.com/archives/april04/SOP/index.asp

1 http://hpcc886.corp.hp.com/comp/employee/program/tr/sop/stockcert.htm

1 http://hpcc886.corp.hp.com/comp/employee/program/tr/sop/purhpshares.htm

“Mellon Investor Services” or “Manufacturing Information Systems”

Surprisingly, the @hp top result was never the last click

Slide 43

Using feedback: ranking algorithms• Statistical ordering

– Interpret last clicks as recommendation– Rank by popularity (most recommended first, etc)– Robust: highly stable in presence of spam

• Move-to-front ordering– Each time a page is recommended (last-clicked),

move it to the top of the list.– Fast: once the good page is found, it moves to the top (tsunami)

• Probabilistic ordering– Choose pages one after another– Probability of selection = frequency of endorsement– Best of both worlds, and many theoretical results– Plus: all new pages have some chance of being seen!

Slide 44

An experimental implementation

• Working with @hp search (Anne Murray Allen):– Ricky Ralston– Chris Jackson– Richard Boucher

• Now part of the @hp search engine:– Getting 10% of queries sent to @hp search– A few months of experience shows promising results…

Slide 45

Example: “paycheck” at @hp

a movie!!

notice from 1999

three copies of slides on redeemin

g eawards

Slide 46

Example: “paycheck” with move2front

how to access

(was #6)how to set

up(was #7)May 2004 upgrade notice

(was #41)

401(k) catchup

(almost no votes)

Slide 47

Example: “active gold” at @hp

hydra migration

sports award

can’t read

can’t read

dead link

Slide 48

Example: “active gold” with statisticalhome page(was #8)(10 of 12

votes)

Slide 49

Example: “payroll” at @hp

Slide 50

Example: “payroll” with statistical

US main payroll

(was #6)payroll forms

(was #17)

the best bet(was #16)

payroll phone numbers

(was #33)

Slide 51

Example: “payroll” with move2front

lots of churn among top hits

Slide 52

Moving in the right direction

• Many compelling instances of progress

• How effective are we in general?– Let’s track position of last-clicked link– We win when we move it up the page– We lose when we move it down

Slide 53

Number of times we win

0.00%

5.00%

10.00%

15.00%

20.00%

25.00%

30.00%

35.00%

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61

stat win

stat lose

mtf win

mtf lose

Slide 54

Number of positions we move

0

1

2

3

4

5

6

7

8

9

10

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61

stat win

stat lose

mtf win

mtf lose

Slide 55

Some improvements hidden

• Some effects of our improvements are masked– Manually constructed best bet lists– Extensive fine-tuning of Ultraseek engine

• Our hope or claim:– We should be able to replace best bets– We are cheaper/easier than tuning Ultraseek

• Even now we improve 5 places 20% of the time

• Hope for a real user study…

Slide 56

Conclusion

• Intranet search is notoriously hard• User collaboration and feedback can help

• Intranet search small part of larger vision:– How to use user feedback and collaboration for

• Searching unstructured data: eg, books scanned at a9.com• Building general purpose recommendation systems