“Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies...
Transcript of “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies...
![Page 1: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/1.jpg)
“Recommended For You”: A First Look at Content Recommendation
Networks
Muhammad Ahmad Bashir, Sajjad Arshad, Christo Wilson
![Page 2: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/2.jpg)
Content Recommendation Networks (CRNs)
2
![Page 3: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/3.jpg)
Content Recommendation Networks (CRNs)
2
![Page 4: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/4.jpg)
Content Recommendation Networks (CRNs)
2
![Page 5: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/5.jpg)
Content Recommendation Networks (CRNs)
2
![Page 6: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/6.jpg)
Content Recommendation Networks (CRNs)
2
![Page 7: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/7.jpg)
Content Recommendation Networks (CRNs)
2
![Page 8: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/8.jpg)
Some Terminology
3
![Page 9: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/9.jpg)
Some Terminology
3
Wid
gets
![Page 10: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/10.jpg)
Some Terminology
3
Wid
gets
Headline
![Page 11: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/11.jpg)
Some Terminology
3
Wid
gets
DisclosureHeadline
![Page 12: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/12.jpg)
Some Terminology
3
Wid
gets
DisclosureHeadline
• Link to the publisher (first-party link) — Recommendation
![Page 13: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/13.jpg)
Some Terminology
3
Wid
gets
DisclosureHeadline
• Link to the publisher (first-party link) — Recommendation
• Links to third-party websites — Ads
![Page 14: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/14.jpg)
Some Terminology
3
Wid
gets
DisclosureHeadline
• Link to the publisher (first-party link) — Recommendation
• Links to third-party websites — Ads
Links + $$$
![Page 15: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/15.jpg)
Some Terminology
3
Wid
gets
DisclosureHeadline
• Link to the publisher (first-party link) — Recommendation
• Links to third-party websites — Ads
Links + $$$ Widgets
![Page 16: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/16.jpg)
Some Terminology
3
Wid
gets
DisclosureHeadline
• Link to the publisher (first-party link) — Recommendation
• Links to third-party websites — Ads
Links + $$$
ClickWidgets
![Page 17: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/17.jpg)
Some Terminology
3
Wid
gets
DisclosureHeadline
• Link to the publisher (first-party link) — Recommendation
• Links to third-party websites — Ads
Links + $$$
Click$$
Widgets$$
![Page 18: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/18.jpg)
Why study CRNs?
4
![Page 19: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/19.jpg)
Why study CRNs?
4
![Page 20: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/20.jpg)
Why study CRNs?
4
![Page 21: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/21.jpg)
Why study CRNs?
4
![Page 22: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/22.jpg)
Why study CRNs?
4
![Page 23: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/23.jpg)
Why study CRNs?
4
![Page 24: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/24.jpg)
Why study CRNs?
4
Third party links
![Page 25: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/25.jpg)
Goal
5
![Page 26: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/26.jpg)
Goal
5
Conduct a study to observe current practices of CRNs.
![Page 27: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/27.jpg)
Goal
5
Conduct a study to observe current practices of CRNs.
• How are the widgets labeled and disclosed?
![Page 28: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/28.jpg)
Goal
5
Conduct a study to observe current practices of CRNs.
• What kind of content is being advertised?
• How are the widgets labeled and disclosed?
![Page 29: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/29.jpg)
Data Collection
6
![Page 30: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/30.jpg)
Data Collection
6
• 5 CRNs. Outbrain, Taboola, Gravity, ZergNet, Revcontent
![Page 31: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/31.jpg)
Data Collection
6
289
Alexa Media and News Category
• 5 CRNs. Outbrain, Taboola, Gravity, ZergNet, Revcontent
![Page 32: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/32.jpg)
Data Collection
6
289 211+
Alexa Media and News Category Random sample from Alexa top 1M
• 5 CRNs. Outbrain, Taboola, Gravity, ZergNet, Revcontent
![Page 33: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/33.jpg)
Data Collection
6
289 211+
Alexa Media and News Category Random sample from Alexa top 1M
500 Publishers
• 5 CRNs. Outbrain, Taboola, Gravity, ZergNet, Revcontent
![Page 34: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/34.jpg)
Data Collection
6
289 211+
Alexa Media and News Category Random sample from Alexa top 1M
500 Publishers
• For each publisher we visited 40 pages, including homepage.
• 5 CRNs. Outbrain, Taboola, Gravity, ZergNet, Revcontent
![Page 35: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/35.jpg)
Data Collection
6
289 211+
Alexa Media and News Category Random sample from Alexa top 1M
500 Publishers
• For each publisher we visited 40 pages, including homepage.
User XPath Queries
• Developed 12 queries.• Covering 5 CRNs.
Examples• Outbrain: //a[@class=‘ob-dynamic-rec-link’]• ZergNet:
//div[@class=‘zergentity’]
• 5 CRNs. Outbrain, Taboola, Gravity, ZergNet, Revcontent
![Page 36: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/36.jpg)
Data Collection
6
289 211+
Alexa Media and News Category Random sample from Alexa top 1M
500 Publishers
• For each publisher we visited 40 pages, including homepage.
User XPath Queries
• Developed 12 queries.• Covering 5 CRNs.
Examples• Outbrain: //a[@class=‘ob-dynamic-rec-link’]• ZergNet:
//div[@class=‘zergentity’]
• 5 CRNs. Outbrain, Taboola, Gravity, ZergNet, Revcontent
Extracted Data
• Widgets from 5 CRNs.• For each widget:
• Elements• Headline• Disclosure• Advertiser URLs
![Page 37: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/37.jpg)
Final Dataset
7
• 5 CRNs• 500 publishers
• 53K recommendations (links to first-party).• 131K ads (links to third-party).
• 2,689 unique advertised domains.
![Page 38: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/38.jpg)
High level Statistics
8
CRN #Publishers Total Ads Total RecsOutbrain 147 57446 35476Taboola 176 56860 15660
Revcontent 29 576 16Gravity 13 744 2054ZergNet 14 15375 0
![Page 39: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/39.jpg)
High level Statistics
8
• Outbrain and Taboola are the major players.
CRN #Publishers Total Ads Total RecsOutbrain 147 57446 35476Taboola 176 56860 15660
Revcontent 29 576 16Gravity 13 744 2054ZergNet 14 15375 0
![Page 40: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/40.jpg)
High level Statistics
8
• Outbrain and Taboola are the major players.• Outbrain (1.5x), Taboola & Revcontent (5x) display more ads on
average per page.
CRN #Publishers Total Ads Total RecsOutbrain 147 57446 35476Taboola 176 56860 15660
Revcontent 29 576 16Gravity 13 744 2054ZergNet 14 15375 0
![Page 41: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/41.jpg)
High level Statistics
8
• Outbrain and Taboola are the major players.• Outbrain (1.5x), Taboola & Revcontent (5x) display more ads on
average per page.• Gravity shows 2.7x more recommendations per page on average.
CRN #Publishers Total Ads Total RecsOutbrain 147 57446 35476Taboola 176 56860 15660
Revcontent 29 576 16Gravity 13 744 2054ZergNet 14 15375 0
![Page 42: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/42.jpg)
Headlines and Disclosures
9
Question: Are CRNs explicitly labeling sponsored links as advertisements?• We take a look at widget headlines and disclosures.
v v
v
vHeadline
Disclosure
![Page 43: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/43.jpg)
Headlines
10
![Page 44: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/44.jpg)
Headlines
10
• 88% widgets provide a headline.
![Page 45: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/45.jpg)
Headlines
10
• 88% widgets provide a headline.
Recommendations Ads
Headline % Headline %
you might also like 17 around the web 18
featured stories 12 promoted stories 15you may like 7 you may like 15
we recommend 7 you might also like 6more from this site 4 from around the wb 2
you might be interested in 2 trending today 2
trending now 1 we recommend 2
![Page 46: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/46.jpg)
Headlines
10
• 88% widgets provide a headline.
Recommendations Ads
Headline % Headline %
you might also like 17 around the web 18
featured stories 12 promoted stories 15you may like 7 you may like 15
we recommend 7 you might also like 6more from this site 4 from around the wb 2
you might be interested in 2 trending today 2
trending now 1 we recommend 2
![Page 47: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/47.jpg)
Headlines
10
• 88% widgets provide a headline.
Recommendations Ads
Headline % Headline %
you might also like 17 around the web 18
featured stories 12 promoted stories 15you may like 7 you may like 15
we recommend 7 you might also like 6more from this site 4 from around the wb 2
you might be interested in 2 trending today 2
trending now 1 we recommend 2
• Keywords used across all headlines for ad widgets.• ‘promoted’ (12 %), ‘advertiser’ (<1 %)
![Page 48: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/48.jpg)
Disclosures
11
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
![Page 49: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/49.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
![Page 50: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/50.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
![Page 51: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/51.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
✔✔
✔
![Page 52: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/52.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
✔
✗✔
✗
✔
![Page 53: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/53.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
✔
✗✔
✗
✔
![Page 54: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/54.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
✔
✗✔
✗
✔
![Page 55: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/55.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
✔
✗✔
✗
✔
![Page 56: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/56.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
✔
✗✔
✗
✔
![Page 57: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/57.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
✔
✗✔
✗
✔
![Page 58: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/58.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
✔
✗✔
✗
![Page 59: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/59.jpg)
Disclosures
11
Overall 94 % widgets had disclosures.
CRN Disclosed (%) VerdictOutbrain 90.8Taboola 97.1
Revcontent 100.0Gravity 81.6ZergNet 24.1
✔
✗✔
✗
✗
![Page 60: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/60.jpg)
Widgets with Mixed Content
12
![Page 61: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/61.jpg)
Widgets with Mixed Content
12
![Page 62: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/62.jpg)
Widgets with Mixed Content
12
![Page 63: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/63.jpg)
Widgets with Mixed Content
12
CRN Mixed (%) Verdict
Outbrain 16.9Taboola 9.0
Revcontent 0
Gravity 25.5ZergNet 0
![Page 64: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/64.jpg)
Widgets with Mixed Content
12
CRN Mixed (%) Verdict
Outbrain 16.9Taboola 9.0
Revcontent 0
Gravity 25.5ZergNet 0
![Page 65: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/65.jpg)
Widgets with Mixed Content
12
CRN Mixed (%) Verdict
Outbrain 16.9Taboola 9.0
Revcontent 0
Gravity 25.5ZergNet 0
✗✗
✗
![Page 66: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/66.jpg)
Widgets with Mixed Content
12
CRN Mixed (%) Verdict
Outbrain 16.9Taboola 9.0
Revcontent 0
Gravity 25.5ZergNet 0
✗
✔
✔
✗
✗
![Page 67: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/67.jpg)
Widgets with Mixed Content
12
Overall 11.9% widgets display mixed content.
CRN Mixed (%) Verdict
Outbrain 16.9Taboola 9.0
Revcontent 0
Gravity 25.5ZergNet 0
✗
✔
✔
✗
✗
![Page 68: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/68.jpg)
What is being Advertised?
13
![Page 69: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/69.jpg)
What is being Advertised?
13
We look at 131K third-party URLs from 2689 unique advertisers.• We did not click on any ads.
![Page 70: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/70.jpg)
What is being Advertised?
13
We look at 131K third-party URLs from 2689 unique advertisers.• We did not click on any ads.
Topic Example Keywords Landing pages (%)
Listicles improve, scams, experience 18.46
Credit Cards credit, card, interest 16.09
Celebrity Gossip Kardashians, sexiest, caught 10.94
Mortgages mortgage, HARP, loan 8.76
Movies Hollywood, Batman, Marvel 5.90
Health & Diet diabetes, fat, stomach 5.62
Investment Dow, dividend, stocks 1.57
Penny Auctions auction, bid, pennies 1.15
![Page 71: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/71.jpg)
What is being Advertised?
13
We look at 131K third-party URLs from 2689 unique advertisers.• We did not click on any ads.
Topic Example Keywords Landing pages (%)
Listicles improve, scams, experience 18.46
Credit Cards credit, card, interest 16.09
Celebrity Gossip Kardashians, sexiest, caught 10.94
Mortgages mortgage, HARP, loan 8.76
Movies Hollywood, Batman, Marvel 5.90
Health & Diet diabetes, fat, stomach 5.62
Investment Dow, dividend, stocks 1.57
Penny Auctions auction, bid, pennies 1.15
• Almost 35% articles are ‘content’.
![Page 72: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/72.jpg)
What is being Advertised?
13
We look at 131K third-party URLs from 2689 unique advertisers.• We did not click on any ads.
Topic Example Keywords Landing pages (%)
Listicles improve, scams, experience 18.46
Credit Cards credit, card, interest 16.09
Celebrity Gossip Kardashians, sexiest, caught 10.94
Mortgages mortgage, HARP, loan 8.76
Movies Hollywood, Batman, Marvel 5.90
Health & Diet diabetes, fat, stomach 5.62
Investment Dow, dividend, stocks 1.57
Penny Auctions auction, bid, pennies 1.15
• Almost 35% articles are ‘content’.• More than 20% articles about Mortgage or Credit Cards.
![Page 73: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/73.jpg)
What is being Advertised?
13
We look at 131K third-party URLs from 2689 unique advertisers.• We did not click on any ads.
Topic Example Keywords Landing pages (%)
Listicles improve, scams, experience 18.46
Credit Cards credit, card, interest 16.09
Celebrity Gossip Kardashians, sexiest, caught 10.94
Mortgages mortgage, HARP, loan 8.76
Movies Hollywood, Batman, Marvel 5.90
Health & Diet diabetes, fat, stomach 5.62
Investment Dow, dividend, stocks 1.57
Penny Auctions auction, bid, pennies 1.15
• Almost 35% articles are ‘content’.• More than 20% articles about Mortgage or Credit Cards.• Articles about financial services, miracle diets.
![Page 74: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/74.jpg)
Summary
• We present the first evaluation on Content Recommendation Networks (CRNs).
• What can be done?• Stronger regulations.• CRNs should enforce clear labels.• Effective spam filtering.
14
CRN Headlines Disclosures Mixed Content Quality
OutbrainTaboola
Revcontent
GravityZergNet
✔✔✗✗
✗✗✗✗✗
✔
✔✗
✗✗ ✗
✗✗✗-
✗
![Page 75: “Recommended For You”: A First Look at Content … · Penny Auctions auction, bid, pennies 1.15. What is being Advertised? 13 We look at 131K third-party URLs from 2689 unique](https://reader034.fdocuments.us/reader034/viewer/2022050502/5f9432b6865aaa12f857b3ce/html5/thumbnails/75.jpg)
Summary
• We present the first evaluation on Content Recommendation Networks (CRNs).
• What can be done?• Stronger regulations.• CRNs should enforce clear labels.• Effective spam filtering.
14
personalization.ccs.neu.edu
CRN Headlines Disclosures Mixed Content Quality
OutbrainTaboola
Revcontent
GravityZergNet
✔✔✗✗
✗✗✗✗✗
✔
✔✗
✗✗ ✗
✗✗✗-
✗