IN SEARCH OF RELEVANCE - Amazon Web...
Transcript of IN SEARCH OF RELEVANCE - Amazon Web...
![Page 1: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/1.jpg)
Munich, 11.10.2016
Michael Muckel
IN SEARCH OF RELEVANCE
MODERN RECOMMENDER SYSTEMS FOR MEDIA INDUSTRY
![Page 2: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/2.jpg)
2
Glomex Overview
Publishers
Content providers
Video Value Service
Media Delivery Service
Media Exchange Service
Glomex
External broadcasters
Web-only content owners
Non-P7S1 publishers
![Page 3: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/3.jpg)
3
List Model for Recommendations
1 Item
2 Item
3 Item
4 Item
N Item
Top-N List
Decreasing Relevance
…
![Page 4: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/4.jpg)
4
Temporal Model for Recommendations
1 Item
Continuous Playlist
2 Item 3 Item 4 Item ∞ Item
Current Item
…
![Page 5: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/5.jpg)
5
Relevance – Search Perspective
Exploration
PrecisionSe
arch
![Page 6: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/6.jpg)
6
Relevance – Recommender Perspective
Exploration
PrecisionSe
arch
Recommender
![Page 7: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/7.jpg)
7
Two Sides of the Relevance Medal
Search
Recommender
![Page 8: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/8.jpg)
8
Glomex Recommender - Customer Perspective
Enable our Glomex Exchange Customers to efficiently select content with high potential for conversions
Metrics: Playlist Add, Pick Top-N
![Page 9: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/9.jpg)
9
Glomex Recommender – User Perspective
Keep attention of users as long as possible
Metrics: Video-View per Playlist, Video-Views per Session, Average View-Time
![Page 10: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/10.jpg)
10
Anatomy of a Recommender System
SCORE RANK FILTER
![Page 11: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/11.jpg)
11
Anatomy of a Recommender System
SCORE RANK FILTER
![Page 12: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/12.jpg)
12
Popular Recommender Models - in a nutshell
![Page 13: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/13.jpg)
13
SCORE - Content-Based Recommenders
Tags:
Publish Date:
Title:
Cast:
Description:
Brand:
Hierachy:
fun, joko, claas, comedy, pro7 ,circus halligalli, show, sport, interview, formula1, race of champions
11.12.2015
Ein unnötig kompliziertes Interview
Joko Winterscheidt, Klaas Heufer-Umlauf, David Coulthard, Sebastian Vettel, Nico Hülkenberg
Wir haben Joko endlich seinen Traum erfüllt, einmal in einemFormel 1-Wagen mitzufahren. Die einzige kleine Bedingung: Ermuss währenddessen ein Interview führen.
Prosieben
Episode: 13 Series: 6
Watch: http://bit.ly/2bO4Z3u
![Page 14: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/14.jpg)
14
SCORE - Content-Based Recommenders – Search Problem
Tags:
Publish Date:
Title:
Cast:
Description:
Brand:
Hierachy:
fun, joko, claas, comedy, pro7 ,circus halligalli, show, sport, interview, formula1, race of champions
11.12.2015
Ein unnötig kompliziertes Interview
Joko Winterscheidt, Klaas Heufer-Umlauf, David Coulthard, Sebastian Vettel, Nico Hülkenberg
Wir haben Joko endlich seinen Traum erfüllt, einmal in einemFormel 1-Wagen mitzufahren. Die einzige kleine Bedingung: Ermuss währenddessen ein Interview führen.
Prosieben
Episode: 13 Series: 6
Find similar content based on descriptive metadata
![Page 15: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/15.jpg)
15
SCORE - Context-Based Recommendation – Use Context
Palina Rojinski
Joko Winterscheidt
Ina Müller
Olli Schulz
Web Mining with:
Named Entity Recognition Word Embeddings
![Page 16: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/16.jpg)
16
SCORE - Collaborative Filtering
User U 3 - 5 1 - -
User U 1 0 1 1 0 0
Explicit Feedbackaka Ratings
Implicit Feedback
Circus HalligalliDawn of the Gag 2
Length of vector depends on number of items available
![Page 17: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/17.jpg)
17
SCORE - Collaborative Filtering
Item 1 Item 2 Item 3 Item 4 … Item N
User 1 x x x
User 2 x x
…
User X x x ? x ? ?
User-Rating Matrix
![Page 18: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/18.jpg)
18
SCORE – User-User Collaborative Filtering
Item 1 Item 2 Item 3 Item 4 … Item N
User 1 x x x
User 2 x x
…
User X x x ? x ? ?
Conceptual Model:
Find users with similar preferences1
Find and Score items from matching users that User X has not watched2
![Page 19: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/19.jpg)
19
SCORE – Item-Item Collaborative Filtering
Item 1 Item 2 Item 3 Item 4 … Item N
User 1 x x x
User 2 x x
…
User X x x ? x ? ?
Conceptual Model:
Find items with similar usage patterns for item 31
Find and Score items from matching users that User X has not watched2
![Page 20: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/20.jpg)
20
SCORE - Collaborative Filtering with Matrix Factorization
Item 1 Item 2 Item 3
User 1 x x
User 2 x
…
User X x x ?
≅ ✕
Conceptual Model:
Decompose rating matrix into dense, low-rank matrices (e.g. Alternating Least Squares)1
Use resulting “Taste-Space” for finding similar videos2
Matrix U Matrix V
![Page 21: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/21.jpg)
21
SCORE – Statistical Models
Detecting (Hidden) Periodicities
Statistical Trend Detection
Possible Methods
![Page 22: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/22.jpg)
22
Anatomy of a Recommender System
SCORE RANK FILTER
![Page 23: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/23.jpg)
23
RANK - Basic Ranking - Sorting
Modele.g. Collaborative
Filtering
1 Item
2 Item
3 Item
N Item
Sort by Relevance Score
…
![Page 24: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/24.jpg)
24
Which model should we then choose?
![Page 25: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/25.jpg)
25
RANK - Hybrid Recommenders – Static Ensembles
f(xi,wi)
Score: xi
Weight: wi
1 Item
2 Item
3 Item
N Item
Model 1Collaborative
Filtering
Model 2Collaborative
Filtering
Model 3Content-Based
Filtering
Model 4Trending
Genre
Model NNew
Content
Sort by Adjusted Relevance Score
![Page 26: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/26.jpg)
26
RANK - Hybrid Recommenders – Learning to Rank
Model 1Collaborative
Filtering
Model 2Collaborative
Filtering
Model 3Content-Based
Filtering
Model 4Trending
Genre
Model NMost-Watched
Content
f(xi,wi)
Score: xi
1 Item
2 Item
3 Item
N Item
Ranking Model
Weight: wi
User Feedback: ClicksUser Context: Location, DeviceSort by Adjusted Relevance Score
…
![Page 27: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/27.jpg)
27
Anatomy of a Recommender System
SCORE RANK FILTER
![Page 28: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/28.jpg)
28
FILTER – Basic Filtering
1 Content
2 Content
3 Content
N Content
1 Content
4 Content
6 Content
M Content
Ranked ListBy Relevance Score
Filtered List
2 Content
3 Content
5 Content
X
X
X
Filter Logic
Example Filter: Geo-Location
…
![Page 29: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/29.jpg)
29
FILTER – Multi-Dimensional Filters
1 Content
2 Content
3 Content
N Content
1 Content
4 Content
Ranked ListBy Relevance Score
Filtered List
2 Content
3 Content
5 Content
X
X
XFilter Logic
Real-world Filters
Geo-Location Device (type) Age-Restriction
Brands Genres ….
5 Content
5 Content
5 Content
X
X
X
!
![Page 30: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/30.jpg)
30
Alternate Approaches
SCORE RANK FILTER
Attribute XExamples: Genre, Rights (Geo-Restriction )
Integrate into Models Integrate into Filter?
![Page 31: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/31.jpg)
31
Glomex on AWS
![Page 32: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/32.jpg)
32
Glomex Data Service
Video Value Service Media Delivery Service Media Exchange Service
Data Service
Real-time-Monitoring Analytics Machine Learning
![Page 33: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/33.jpg)
33
Glomex Data Service
![Page 34: IN SEARCH OF RELEVANCE - Amazon Web Servicesaws-de-media.s3.amazonaws.com/images/Media_Entertainment_Roundtable/... · munich, 11.10.2016 michael muckel in search of relevance modern](https://reader035.fdocuments.us/reader035/viewer/2022070800/5f0243a87e708231d40365c7/html5/thumbnails/34.jpg)
34
Thank you for your attention!