Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

16
Accelerating Innovation with Cloud Computing Hari Vasudev India Hadoop Summit - Bangalore February 2010

Transcript of Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Page 1: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Accelerating Innovation with

Cloud Computing

Hari Vasudev

India Hadoop Summit - Bangalore

February 2010

Page 2: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

I’m not selling anything

Page 3: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Cloud Computing is NOT

about saving money

Page 4: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source
Page 5: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Yahoo! is Perfect for Cloud Computing

HUNDREDSOF PROPERTIES / PRODUCTS

600MUNIQUE USERS / MONTH

300M+YAHOO! MAIL USERS / MONTH

HUNDREDSOF PETABYTES OF STORAGE

BILLIONSOF OBJECTS STORED

PETABYTESOF TRAFFIC DAILY

Page 6: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Yahoo! Cloud Strategy

• Creating a private Cloud for Yahoo!

• Optimizing for global Yahoo! properties

• Data processing and serving environments

• Multi-year effort

• Open Source

Page 7: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Inside Yahoo!’s Cloud

Page 8: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Yahoo!’s Open Source for Cloud

Page 9: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Cloud Solving Industry-wide Problems

• Mail abuse detection

• Dependent on globally synchronized data

• Cloud storage

• Global data replication

• Consistency

• Fast and easy to use

• Developers focus on task at hand

Page 10: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

• Organizational commitment

• Investment• Investment

• Time

Page 11: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Cloud Computing is worth it!

Page 12: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Advertising

Optimization

& Delivery

Content

Optimization

Search

Index

Yahoo!’s Cloud Use CaseCaching, Load Balancing

Machine

Learning (e.g. Spam filters)

& DeliveryOptimization

Image/Video

Storage &

Delivery

RSS

Feeds

Attachment

Storage

Page 13: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Cloud improves dynamiccontent refresh rates and content refresh rates and consumer access speed

Page 14: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Cloud abstracts away scale for processing enormous for processing enormous

data sets

Page 15: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Cloud speeds advertising optimization by improving

15

optimization by improving infrastructure utilization

Page 16: Hadoop Summit 2010 Yahoo’S Commitment To Hadoop And Open Source

Cloud Speeds Time To Market

• YQL

• SQL-like language

• Query, filter, and join data across

web services

• YQL Open Data Tables built • YQL Open Data Tables built

on Cloud storage

• Simple and fast integration and

deployment

• Immediate access to global,

replicated, fast, reliable data store

16