Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of...

21
Typische Pfade auf dem Reise zum Data Lake Andreas Leichtle Senior Account Manager, Hortonworks © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Transcript of Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of...

Page 1: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Typische Pfade auf dem Reise zum Data Lake

Andreas Leichtle Senior Account Manager, Hortonworks

© Hortonworks Inc. 2011 – 2015. All Rights Reserved

Page 2: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 2 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

This presentation contains forward-looking statements involving risks and uncertainties. Such forward-looking statements in this presentation generally relate to future events, our ability to increase the number of support subscription customers, the growth in usage of the Hadoop framework, our ability to innovate and develop the various open source projects that will enhance the capabilities of the Hortonworks Data Platform, anticipated customer benefits and general business outlook. In some cases, you can identify forward-looking statements because they contain words such as “may,” “will,” “should,” “expects,” “plans,” “anticipates,” “could,” “intends,” “target,” “projects,” “contemplates,” “believes,” “estimates,” “predicts,” “potential” or “continue” or similar terms or expressions that concern our expectations, strategy, plans or intentions. You should not rely upon forward-looking statements as predictions of future events. We have based the forward-looking statements contained in this presentation primarily on our current expectations and projections about future events and trends that we believe may affect our business, financial condition and prospects. We cannot assure you that the results, events and circumstances reflected in the forward-looking statements will be achieved or occur, and actual results, events, or circumstances could differ materially from those described in the forward-looking statements. The forward-looking statements made in this prospectus relate only to events as of the date on which the statements are made and we undertake no obligation to update any of the information in this presentation. Trademarks Hortonworks and HDP are trademarks of Hortonworks, Inc. in the United States and other jurisdictions. Other names used herein may be trademarks of their respective owners.

Page 2 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Page 3: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 3 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Who we are

2005 2011

24

1100+

100%

Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees

Open Source 530+ Customers

* The Forrester Wave Big Data Hadoop Solutions Q1 2014

Partner 600+

Page 4: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 4 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Quelle: http://homepages.inf.ed.ac.uk/miles/code.html

Page 5: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 5 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Quelle: http://developer.yahoo.com/blogs/ydn/posts/2007/07/yahoo-hadoop/

Page 6: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 6 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Page 7: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 7 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Page 8: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 8 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Page 9: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 9 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Page 10: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 10 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Dat

a Pl

atfo

rm C

apab

ilitie

s

12 months execution plan

June 2013 Begin Hadoop Execution

July 2013 Hortonworks Partnership

May ‘14 IPO

Aug 2013 Training & Dev Begins

Nov 2013 Production Cluster 60 Nodes 2 PB

Jan 2014 40% Dev Staff Perficient

Dec 2013 Three Production Apps (3 total)

Feb 2014 Three More Production Apps (6 total)

12 Month Results at TRUECar •  Six Production Hadoop Applications •  Sixty nodes/2PB data •  Storage Costs/Compute Costs

from $19/GB to $0.23/GB

“We addressed our data platform capabilities strategically as a pre-cursor to IPO.”

Page 11: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 11 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

•  “Data is our product” 8.000.000+ vehicles in inventory kept track every day 250.000.000 vehicle images under asset for live data

50 years of inventory data

•  Developers should not worry about

Deleting Data Moving Data System scale

Storage

•  Consolidation of data into a immediatiely computable, searchable infrastructure Vehicle Data VIN Decoder

Intelligent Image Processing

Page 12: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 12 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Page 13: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 13 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Page 14: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 14 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Page 15: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 15 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Summary

•  Tutorial with real world healthcare data (“Medicare Part-B”)

•  Existing Solutions: Rule based detection •  Problem: Hard to maintain, unable to detect new

patterns •  New Solutions: Personalised PageRank

•  Compute similarity between providers (CPT codes) •  Loop over all specialties (dermatologist, internal medicine, …)

Page 16: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 16 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Result

Current Procedural Terminology •  Internal eye photography •  Cmptr ophth img optic nerve •  Echo exam of eye thickness •  Revise eyelashes •  Ophthalmic biometry •  Eye exam new patient •  Eye exam established pat •  After cataract laser surgery •  Eye exam & treatment •  Eye exam with photos •  Visual field examination(s)

NPI Code •  Internal Medicine Provider

Page 17: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 17 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

SmartSense Proactively Optimizes Your HDP

Potential issues by analyzing data

Identifies

Machine data from Hadoop clusters

Specific solutions and actions

Monitors

Recommends

Page 18: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 18 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Page 18 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Payment Tracking

Call Analysis

Machine Data

Product Design

Social Mapping

Factory Yields

Defect Detection

Due Diligence

M & A Proactive Repair

Disaster Mitigation

Investment Planning

Next Product

Recs

Store Design

Risk Modeling

Ad Placement

Inventory Predictions

Sentiment Analysis

Ad Placement

Basket Analysis Segments

Customer Support

Supply Chain

Cross- Sell

Customer Retention

Vendor Scorecards

Optimize Inventories

Business executives are driving transformational outcomes with next-generation applications that empower new uses of Big Data including: data discovery, a single view of the customer and predictive analytics.

Page 19: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 19 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Page 19 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Historical Records

OPEX Reduction

Mainframe Offloads

Fraud Prevention

Data as a

Service

Public Data

Capture

IT executives are delivering substantial reductions in operating costs by modernizing their data architectures with Open Enterprise Hadoop. These cost saving innovations include active archive of cold data, offloading ETL processes and enriching existing data.

Digital Protection

Device Data

Ingest

Rapid Reporting

Page 20: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 20 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Page 20 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Hortonworks® customers leverage our technology to transform their businesses, either by achieving new business objectives or by reducing costs. The journey typically involves both of those goals in combination, across many use cases.

Social Mapping

Payment Tracking

Factory Yields

Defect Detection

Call Analysis

Machine Data

Product Design M & A

Due Diligence

Next Product

Recs

Store Design

Risk Modeling

Ad Placement

Proactive Repair

Disaster Mitigation

Investment Planning

Inventory Predictions

Customer Support

Sentiment Analysis

Supply Chain

Ad Placement

Basket Analysis Segments

Cross- Sell

Customer Retention

Vendor Scorecards

Optimize Inventories

OPEX Reduction

Mainframe Offloads

Historical Records

Data as a

Service

Public Data

Capture

Fraud Prevention

Device Data

Ingest

Rapid Reporting

Digital Protection

Page 21: Typische Pfade auf dem Reise zum Data Lake2011 24 1100+ 100% Apache Hadoop at Yahoo! Inception of Hortonworks Developers and Architects Employees Open Source ... from $19/GB to $0.23/GB

Page 21 © Hortonworks Inc. 2011 – 2015. All Rights Reserved

Questions?

Page 21 © Hortonworks Inc. 2011 – 2015. All Rights Reserved