Cu Presents Magazine Takács Quartet Winter 2015, Nov. 8-9, 2015
Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I....
-
Upload
blanche-warner -
Category
Documents
-
view
227 -
download
0
Transcript of Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I....
![Page 1: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/1.jpg)
Investigation of Various Factorization Methods for Large Recommender
Systems
G. Takács, I. Pilászy, B. Németh and D. Tikk
www.gravityrd.com
10th International Workshop on High Performance Data Mining (in conjunction with ICDM)
Pisa, December 15th 2008
![Page 2: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/2.jpg)
Content
Problem definition Approaches Matrix factorization
Basics, BRISMF, Semipositive, Retraining Further enhancements
Transductive MF, Neighbor based correction Experimental results
![Page 3: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/3.jpg)
Collaborative filtering
![Page 4: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/4.jpg)
Problem definition I.
1 4 3
44
2 44
![Page 5: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/5.jpg)
Problem definition II.
The phenomenon can be modeled by the random triplet (U, I, R).
A realization of the phenomenon (u, i, r) means that the u-th user rated the i-th item with value r.
user id (range: {1, …, M})item id (range: {1, …, N})rating value (range: {r1, …, rL})
![Page 6: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/6.jpg)
Problem definition III.
The goal: predict R from on (U, I). Error criterion: mean squared error (RMSE). The task is nothing else than the classical
regression estimation. Classical methods fail because of the unusual
characteristics of the predictor variables.
![Page 7: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/7.jpg)
Content
Problem definition Approaches Matrix factorization
Basics, BRISMF, Semipositive, Retraining Further enhancements
Transductive MF, Neighbor based correction Experimental results
![Page 8: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/8.jpg)
Approaches
Matrix factorization: approximates the rating matrix by the product of two lower-rank matrices.
Neighbor based approach: defines similarity between the rows or the columns of the rating matrix.
Support based approach: characterizes the users based on the binarized rating matrix.
Restricted Boltzmann machine: models each user by a stochastic, recurrent neural network.
Global effects: cascades 1-variable predictors.
![Page 9: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/9.jpg)
Content
Problem definition Approaches Matrix factorization
Basics, BRISMF, Semipositive, Retraining Further enhancements
Transductive MF, Neighbor based correction Experimental results
![Page 10: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/10.jpg)
Matrix Factorization (MF)
Idea: approximate the rating matrix as the product of two lower-rank matrices
R ≈ P ∙ Q
Problem: huge number of parameters (e.g. 10
million), R is partially unknown. Solution: incremental gradient descent.
P: user feature matrix (M x K)
Q: item feature matrix: (K x N)R: rating matrix (M x N)
![Page 11: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/11.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.2
1.2
0.4
-0.5
0.9
-0.4
1.4 0.8 -1.3 -0.1 0.5
-0.2 0.5 -0.4 1.6 0.3
![Page 12: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/12.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.2
1.2
0.4
-0.5
0.9
-0.4
1.4 0.8 -1.3 -0.1 0.5
-0.2 0.5 -0.4 1.6 0.3
![Page 13: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/13.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.4
-0.4
0.9
-0.4
1.3 0.8 -1.3 -0.1 0.5
-0.1 0.5 -0.4 1.6 0.3
![Page 14: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/14.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.4
-0.4
0.9
-0.4
1.3 0.8 -1.3 -0.1 0.5
-0.1 0.5 -0.4 1.6 0.3
![Page 15: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/15.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.2
1.2
0.4
-0.3
0.9
-0.4
1.3 0.9 -1.3 -0.1 0.5
-0.1 0.4 -0.4 1.6 0.3
![Page 16: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/16.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.2
1.2
0.4
-0.3
0.9
-0.4
1.3 0.9 -1.3 -0.1 0.5
-0.1 0.4 -0.4 1.6 0.3
![Page 17: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/17.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.4
-0.2
0.9
-0.4
1.3 0.9 -1.3 -0.0 0.5
-0.1 0.4 -0.4 1.5 0.3
![Page 18: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/18.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.4
-0.2
0.9
-0.4
1.3 0.9 -1.3 -0.0 0.5
-0.1 0.4 -0.4 1.5 0.3
![Page 19: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/19.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.1
0.4
-0.2
0.8
-0.4
1.3 0.9 -1.2 -0.0 0.5
-0.1 0.4 -0.3 1.5 0.3
![Page 20: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/20.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.1
0.4
-0.2
0.8
-0.4
1.3 0.9 -1.2 -0.0 0.5
-0.1 0.4 -0.3 1.5 0.3
![Page 21: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/21.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.4
-0.2
0.9
-0.4
1.3 0.9 -1.2 0.1 0.5
-0.1 0.4 -0.3 1.6 0.3
![Page 22: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/22.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.4
-0.2
0.9
-0.4
1.3 0.9 -1.2 0.1 0.5
-0.1 0.4 -0.3 1.6 0.3
![Page 23: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/23.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.5
-0.2
0.9
-0.3
1.5 0.9 -1.2 0.1 0.5
0.0 0.4 -0.3 1.6 0.3
![Page 24: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/24.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.5
-0.2
0.9
-0.3
1.5 0.9 -1.2 0.1 0.5
0.0 0.4 -0.3 1.6 0.3
![Page 25: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/25.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.4
-0.2
0.9
-0.2
1.5 0.9 -1.1 0.1 0.5
0.0 0.4 -0.2 1.6 0.3
![Page 26: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/26.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.4
-0.2
0.9
-0.2
1.5 0.9 -1.1 0.1 0.5
0.0 0.4 -0.2 1.6 0.3
![Page 27: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/27.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.1
1.2
0.5
-0.2
0.9
-0.1
1.5 0.9 -1.1 0.1 0.6
0.0 0.4 -0.2 1.6 0.2
![Page 28: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/28.jpg)
After a while...
![Page 29: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/29.jpg)
MF sample - learning
Q
P1 4 3
44
2 44
R1.4
0.9
2.5
1.1
1.9
-0.3
1.5 2.1 1.0 0.7 1.6
-1.0 0.8 1.6 1.8 0.0
![Page 30: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/30.jpg)
MF sample - prediction
Q
P1 4 3
44
2 44
R1.4
0.9
2.5
1.1
1.9
-0.3
1.5 2.1 1.0 0.7 1.6
-1.0 0.8 1.6 1.8 0.0
-0.5 3.5
4.9 1.1
3.3 2.4
1.5
![Page 31: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/31.jpg)
BRISMF
Enhancements on the previous model: User and item Biases (offsets). Regularization.
We can call this Biased Regularized Incremental Simultaneous MF (BRISMF).
This is a very effective MF variant indeed. Leaving out any of these characteristics
(B, R, I, S) leads to inferior accuracy.
![Page 32: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/32.jpg)
Semipositive MF
It is useful to put a nonnegativity constraint on the user feature matrix P.
There are many possible ways to implement this (e.g. PLSA, alternating least squares).
Our solution: if a user feature becomes negative after the update, then it is set to zero.
![Page 33: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/33.jpg)
Reset User Features
Disadvantage of BRISMF: user features updated at the beginning of an epoch may be inappropriate at the end of the epoch.
Solution: 1) Reset user features at the end of the training. 2A) Retrain user features. 2B) Retrain both user and item features.
R P
Q
P'
![Page 34: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/34.jpg)
Content
Problem definition Approaches Matrix factorization
Basics, BRISMF, Semipositive, Retraining
Further enhancements Transductive MF, Neighbor based correction
Experimental results
![Page 35: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/35.jpg)
Transductive MF
How is it possible to use the Netflix Qualifying set in the correction phase?
We use the following simple solution:
![Page 36: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/36.jpg)
Fast and Accurate NB Correction I.
Neighbor based (NB) methods can improve the accuracy of factor models, but conventional NB methods are not scalable.
Is it possible to integrate the NB approach into the factor model without losing scalability?
![Page 37: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/37.jpg)
Fast and Accurate NB Correction II.
Where sjk is (normalized scalar product based similarity):
OR (normalized Euclidean distance based similarity)
![Page 38: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/38.jpg)
NB Correction sample
Q
P1 4
R1.4 1.6
1.5 2.1 1.0 2.2 1.6
-1.0 0.8 1.6 0.7 0.0
4.20.5 4.2
4.1
Similarity: 0.2, Error: -0.5
Similarity: 0.8, Error: +0.2
Correction: -0.1
![Page 39: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/39.jpg)
Content
Problem definition Approaches Matrix factorization
Basics, BRISMF, Semipositive, Retraining Further enhancements
Transductive MF, Neighbor based correction
Experimental results
![Page 40: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/40.jpg)
Results I.
Method Name of our methodOur Probe 10 Our Quize Bell et al's QuizeSimple MF BRISMF#10000.8938 0.8939 0.8998Retrained MF BRISMF#1000UM0.8921 0.8918 N/AMF with neighbor correctionBRISMF#1000UM,S10.8905 0.8904 0.8953
MethodName of our
methodOur Probe10 Our Quiz Bell et al's Quiz
Simple MFBRISMF
#10000.8938 0.8939 0.8998
Retrained MFBRISMF
#1000UM0.8921 0.8918 N/A
MF with neighbor correction
BRISMF
#1000UM/S20.8905 0.8904 0.8953
![Page 41: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/41.jpg)
Results II.
Method Name of our methodOur Probe 10 Our Quize Bell et al's QuizeSimple MF BRISMF#10000.8938 0.8939 0.8998Retrained MF BRISMF#1000UM0.8921 0.8918 N/AMF with neighbor correctionBRISMF#1000UM,S10.8905 0.8904 0.8953
Epoch Training Time (sec) RMSE
1 120 0.9188
2 200 0.9071
3 280 0.9057
4 360 0.9028
5 440 0.9008
6 520 0.9002
![Page 42: Investigation of Various Factorization Methods for Large Recommender Systems G. Takács, I. Pilászy, B. Németh and D. Tikk 10th International.](https://reader035.fdocuments.us/reader035/viewer/2022062321/56649f265503460f94c3dd55/html5/thumbnails/42.jpg)
Thanks!
?