You’ve got two weeks: Why you shouldn’t discuss cool ideas near your...
Transcript of You’ve got two weeks: Why you shouldn’t discuss cool ideas near your...
![Page 1: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/1.jpg)
You’ve got two weeks:Why you shouldn’t discuss cool ideas near your managerDan Chaffey
Itinerant Hacker and Solutions Engineer, Hortonworks
![Page 2: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/2.jpg)
2 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
About HortonworksCustomer Momentum
~1000 customers
>150 customers added per quarter
Publicly traded on NASDAQ: HDP
The Leader in Connected Data Platforms
Hortonworks DataFlow for data in motion
Hortonworks Data Platform for data at rest
Powering new modern data applications
Partner for Customer Success
Leader in open-source community, focused on innovation to meet enterprise needs
Unrivaled support subscriptions
Founded in 2011
Original 24 Architects, Developers, Operators of Hadoop from Yahoo!
1000+E M P L O Y E E S
1500+E C O S Y S T E M PA R T N E R S
![Page 3: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/3.jpg)
3 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Actionable Intelligence fromConnected Data Platforms
Capturing perishable insights from data in motion
Ensuring rich, historical insightson data at rest
Necessary for moderndata applications
DATA AT RESTDATA IN MOTION
ACTIONABLEINTELLIGENCE
Modern Data Applications
Hortonworks DataFlow
Hortonworks Data Platform
![Page 4: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/4.jpg)
4 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Modern Data AppsCustom or Off the Shelf
Real-Time Cyber Securityprotects systems with superior threat detection
Smart Manufacturingdramatically improves yields by managing more variables in greater detail
Connected, Autonomous Vehiclesdrive themselves and improve road safety
Automatic Recommendation Enginesmatch products to preferences in milliseconds
Credit Fraud Preventionpredict in real-time based on each customer’s transaction record
DATA ATREST
DATA IN MOTION
ACTIONABLEINTELLIGENCE
Modern Data Applications
Hortonworks DataFlow
Hortonworks Data Platform
![Page 5: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/5.jpg)
Credit Fraud Prevention on a Connected Data PlatformBuilt by: Kirk Haslbeck & Vadim Vaks
![Page 6: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/6.jpg)
6 © Hortonworks Inc. 2011 – 2016. All Rights ReservedPage 6
Improved Experience
/Reduced Cost
Immediate Customer Feedback
Years of Customer
Transaction Data
Fraud Detection
Complete Customer
Profile
Real time ingest of
transactions
Proactively identify potential fraudulent transactions to protect the customer and improve customer experience• Proactively monitor every credit
card transaction using machine learning to catch potential fraud
• Customer Service Analyst reviews flagged transactions in real time via a next generation application running on the connected platform
• Control real time flow of data in and out of the connected platform to the various source and destination points
Innovate
Renovate
Purchase Behavior Insight
Journey to Fraud Detection
![Page 7: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/7.jpg)
7 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
DATA ATREST
DATA IN MOTION
Actionable Intelligence fueled by Adaptive
Machine Learning
Customer Service Analyst
Customer
DATASOURCES
Fraud Detection Application Architecture
![Page 8: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/8.jpg)
8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Fraud Detection Demo Architecture
Distributed Storage: HDFS
Many Workloads: YARN
Real-time Serving
Machine Learning
UI and HTTP PubSub
Real-Time Data Movement
Data Science
Resource Allocation Interactive Query
Co
nfigu
ration
Man
agemen
t(A
mb
ari)Au
tho
rization
(Ran
ger)
Real Time Processing Inbound Messaging
D A T A I N
M O T I O N
D A T A I N
M O T I O N
D A T A I N
M O T I O N
Go
vernan
ce(A
tlas)
![Page 9: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/9.jpg)
PiWiNiFi:Bi-directional IoTsensor networksBuilt by: Simon Elliston Ball & Dan Chaffey
![Page 10: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/10.jpg)
10 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
System Architecture
HDF
HDF
HDF
BASTION
CLOUDVENUE INTERNAL
![Page 11: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/11.jpg)
11 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Demo Features
BASTION
CLOUD
VENUE
INTERNAL
Booth DRDMZ Prod
Solr / Banana Zeppelin/Spark
Raspberry Pi
Codebase
![Page 12: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/12.jpg)
12 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Data FlowData in Motion at the core and at the edge.
http://nifi.apache.org/http://hortonworks.com/hdf/http://community.hortonworks.com/
![Page 13: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/13.jpg)
13 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
ResilientClustering, Availability, Load Balanced Site-to-Site
https://www.flickr.com/photos/rosefirerising/3818133872
![Page 14: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/14.jpg)
14 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
DistributedSite-to-Site takes processing and prioritization right to the edge. Coming soon: MiNiFi!
![Page 15: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/15.jpg)
15 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
Thank You
![Page 16: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/16.jpg)
16 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
References
http://hortonworks.com/blog/credit-card-fraud-prevention-on-a-connected-data-platform/
https://community.hortonworks.com/repos/27236/credit-fraud-prevention-demo.html– https://github.com/vakshorton/CreditCardTransactionMonitor
– https://github.com/vakshorton/CreditCardTransactionMonitorMobileApp
https://github.com/simonellistonball/PiWiNiFi
![Page 17: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/17.jpg)
17 © Hortonworks Inc. 2011 – 2016. All Rights ReservedPage 17
Credit Fraud Analyst Inbox
![Page 18: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/18.jpg)
18 © Hortonworks Inc. 2011 – 2016. All Rights ReservedPage 18
Hortonworks Data Flow
![Page 19: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/19.jpg)
19 © Hortonworks Inc. 2011 – 2016. All Rights ReservedPage 19
Hortonworks Data Flow
![Page 20: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/20.jpg)
20 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
![Page 21: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/21.jpg)
21 © Hortonworks Inc. 2011 – 2016. All Rights Reserved
![Page 22: You’ve got two weeks: Why you shouldn’t discuss cool ideas near your …biconsulting.hu/letoltes/2016budapestdata/dan_chaffey... · 2016-06-29 · The Leader in Connected Data](https://reader035.fdocuments.us/reader035/viewer/2022070710/5ec72ce438d8e86675544905/html5/thumbnails/22.jpg)
22 © Hortonworks Inc. 2011 – 2016. All Rights Reserved