Evaluating, Combining and Generalizing Recommendations with Prerequisites
description
Transcript of Evaluating, Combining and Generalizing Recommendations with Prerequisites
Evaluating, Combining and Generalizing Recommendations with Prerequisites
Aditya Parameswaran Stanford University
(with Profs. Hector Garcia-Molina and Jeffrey D. Ullman)
1
2
Statistics (at Stanford): >10,000 registered users 37,000 listed courses 160,000 evaluationsOverall
172 universities100,000 users
Course Recommendations• Instead of a traditional ranked list …• Recommend a good package satisfying
– Prerequisites (e.g., algebra calculus)– Requirements (e.g., > 3 math courses)– Planning constraints (e.g., no two in same slot)
• Recent work on recommending packages– Yahoo! Travel Plans [De Choudhury et. al. WWW
10]Yahoo! Composite items [Roy et. al. SIGMOD 10]
– Minimizing Cost [Xie et. al. RecSys 10]3
Prior Work
Intuitive Example• Nodes represent all items not taken yet• Edges imply prerequisites
4
E(2)B(6)
I(2)K(9)
C(3)
J(8) H(8)
A(5)
G(7)
D(7)
Prerequisites: NOscore: 32
Prerequisites: YES score: 29
Example: General Prerequisites
5
Algebra
Optimization Algorithms
Geometry
Arithmetic
Statistics
Adv. Math
Set Theory
Probability
InformationTheory
Formal Problem• Directed acyclic graph G(V, E)
– with some nodes Labeled AND or OR• Every node x has a score(x)• Recommend k = |A| courses such that
– score(a) is maximized {a ϵ A}– Prerequisites of all nodes are met
66
OR Graphs
AND-OR Graphs
AND Graphs
Chain Graphs
Outline of Work
7
• Complexity• Chain Graphs: PTIME DP• AND / OR / AND-OR: NP-Hard
• Adaptable Approx Algorithms1) Breadth First2) Greedy 3) Top Down• Worst case per structure• Complexity: DP > Greedy > Top Down > BF
• Merge Algorithm• Experiments• Extensions to Fuzzy Prerequisites
OR Graphs
AND-OR Graphs
AND Graphs
Chain Graphs
For Chain Graphs
Sample
Chain Graph AlgorithmChain 0 Chain 1 …. Chain i -1 Chain i Chain n
8
0
j
To pick j items from i chains: Pick• x items from i-1 chains• First j – x items from the ith chain
Score of best feasible set of j items from first
i chains
B [j, i] = max over all x{B [x, i–1] + 1 … (j—x) of ith chain}
Complexity: O(nk2)
k
Breadth First Algorithm Illustration• K = 4• Add items until k = 4• Swap items
9
E(2)B(6)
I(2)K(9)
C(3)
J(8) H(8)
A(5)
G(7)
D(7)
Top Down & Greedy Algorithms
• Algorithms between extremes– Efficient: Breadth First– Inefficient but Exact: Dynamic Programming
• Top Down is the reverse of Breadth First– Add best items first, then try to add prerequisites
• Greedy reasons about entire chains at once– Tries to add prefixes of chains with high avg score
10
Outline of Work
11
• Complexity• Chain Graphs: PTIME DP• AND / OR / AND-OR: NP-Hard
• Adaptable Approx Algorithms1) Breadth First2) Greedy 3) Top Down• Worst case per structure• Complexity: DP > Greedy > Top Down > BF
• Merge Algorithm• Experiments• Extensions to Fuzzy Prerequisites
OR Graphs
AND-OR Graphs
AND Graphs
Chain Graphs
For Chain Graphs
Sample
Breadth First: Worst Case• Worst case in terms of:
– d: maximum length of chain– m: max difference in score in a given chain
12
a+m-Є
a - Є
a+m-Є
a+m-Є
a - Є
a+m-Є
a+m-Є
a - Є
a+m-Є
a a a
lots of chains of depth d lots of singleton elements
difference = k/d x (da + (d-1)m) - ka
= k/d x (d-1)m
Experimental Setup
• Measure How we perform• As fraction of DP (for chains) & no-prereqs (for AND graphs)• Vary:
– n: number of components– d: max depth of chain / component– p: probability of a long chain / large component– k: size of package
• Score: exponentially distributed13
Chain Graphs AND GraphsThree Algorithms: greedy, td, bf Three Algorithms: greedy, td, bfTop-2 of td, bf Top-2 of td, bfMerge-2 of td, bf Top-3 of greedy, td, bf
Chain Graphs on Varying k
14Size of the desired package
R
atio
of D
ynam
ic P
rogr
amm
ing
Sol
utio
n
AND Graphs on Varying
15Probability of Long Component
Rat
io o
f No-
prer
equi
site
Sol
utio
n
Conclusions• Dynamic Programming Algorithm
– Only for Chain Graphs– Guaranteed best recommendations
• Greedy Value Algorithm– Adaptable to any structure– Almost as good recommendations as DP – With less complexity
• Top Down and Breadth First– Even better complexity– Not as good recommendations as Greedy– Can be improved using Merge algorithm
16
Chain Graphs on Varying p
17Probability of a long chain (k small)
R
atio
of D
ynam
ic P
rogr
amm
ing
Sol
utio
n