Big Data Berlin – Automating Decisions is the Next Frontier for Big Data
-
Upload
lars-trieloff -
Category
Software
-
view
858 -
download
0
description
Transcript of Big Data Berlin – Automating Decisions is the Next Frontier for Big Data
![Page 1: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/1.jpg)
A U T O M AT I N G D E C I S I O N S – T H E N E X T F R O N T I E R F O R B I G D ATA
L A R S T R I E L O F F – B L U E Y O N D E R – I ’ M @ T R I E L O F F O N T W I T T E R
![Page 2: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/2.jpg)
T H E F R O N T I E RW H A T N O B O D Y T E L L S Y O U A B O U T
![Page 3: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/3.jpg)
W H O I S U S I N G B I G D ATA T O D AY
![Page 4: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/4.jpg)
W H E R E B I G D ATA I S U S E D
Effective Use
Marketing
Finance
Everyone Else
S O U R C E : O W N , P O T E N T I A L LY B I A S E D O B S E R VAT I O N S
![Page 5: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/5.jpg)
W H E R E B I G D ATA I S U S E D
Potential Use
Marketing
Finance
Everyone Else
S O U R C E : O W N , P O T E N T I A L LY B I A S E D O B S E R VAT I O N S
![Page 6: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/6.jpg)
Better Decisions
Faster Data
More Data
T H R E E A P P R O A C H E S
![Page 7: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/7.jpg)
M O R E D ATAD I G I TA L M A R K E T I N G ’ S A P P R O A C H :
![Page 8: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/8.jpg)
FA S T E R D ATAF I N A N C E ’ S A P P R O A C H
![Page 9: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/9.jpg)
B U T W H AT A B O U T B E T T E R D E C I S I O N S ?
![Page 10: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/10.jpg)
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization
![Page 11: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/11.jpg)
Collection
Storage
Analysis
Prediction
Decision
![Page 12: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/12.jpg)
Collection
Storage
Analysis
Prediction
Decision
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization
![Page 13: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/13.jpg)
![Page 14: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/14.jpg)
“… all others bring data.” !
— W. E D W A R D S D E M I N G
![Page 15: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/15.jpg)
H O W D ATA - D R I V E N D E C I S I O N S S H O U L D W O R K
C O M P U T E R C O L L E C T S
C O M P U T E R S T O R E S
H U M A N A N A LY Z E S
H U M A N P R E D I C T S
H U M A N D E C I D E S
![Page 16: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/16.jpg)
H O W D ATA - D R I V E N D E C I S I O N S R E A L LY W O R K
C O M P U T E R C O L L E C T S
C O M P U T E R S T O R E S
H U M A N A N A LY Z E S
C O M M U N I C AT I O N B R E A K D O W N
H U M A N D E C I D E S
![Page 17: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/17.jpg)
C O M M U N I C AT I O N B R E A K D O W N
Communication Breakdown, It's always the same, I'm having a nervous breakdown, Drive me insane!
— L E D Z E P P E L I N
• Drill-down analysis … misunderstood or distorted• Metrics dashboards … contradictory and confusing• Monthly reports … ignored after two iterations• In-house analyst teams … overworked and powerless
![Page 18: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/18.jpg)
H O W D ATA - D R I V E N D E C I S I O N S R E A L LY W O R K
H T T P : / / D I L B E R T. C O M / S T R I P S / C O M I C / 2 0 0 7 - 0 5 - 1 6 /
![Page 19: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/19.jpg)
H O W D E C I S I O N S R E A L LY S H O U L D W O R K
C O M P U T E R C O L L E C T S
C O M P U T E R S T O R E S
C O M P U T E R A N A LY Z E S
C O M P U T E R P R E D I C T S
C O M P U T E R D E C I D E S
![Page 20: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/20.jpg)
99.9% of all business decisions can be automated
![Page 21: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/21.jpg)
H O W D E C I S I O N S A R E M A D E
Human Decisions
Business Rules
No Decision, Decision-by-default
![Page 22: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/22.jpg)
B U S I N E S S R U L E S = P R O G R A M M I N G
• Business rules are like programs – written by non-programmers
• Business rules can be contradictory, incomplete, and complex beyond comprehension
• Business rules have no built-in feedback mechanism: “It is the rule, because it is the rule”
![Page 23: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/23.jpg)
H U M A N D E C I S I O N M A K I N G
• • System 1: Fast, automatic, frequent, emotional, stereotypic, subconscious
• • System 2: Slow, effortful, infrequent, logical, calculating, conscious
D A N I E L K A H N E M A N N , T H I N K I N G FA S T A N D S L O W
![Page 24: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/24.jpg)
H O W T O D E C I D E FA S TF R E Q U E N T D E C I S I O N M A K I N G M E A N S FA S T D E C I S I O N M A K I N G , M E A N S U S I N G H E U R I S T I C S O R C O G N I T I V E B I A S E S
Anchoring effect
IKEA effect
Confirmation bias
Bandwagon effect
Substitution
Availability heuristic Texas Sharpshooter Fallacy
Rhyme as reason effect
Over-justification effect
Zero-risk bias
Framing effect
Illusory correlationSunk cost fallacy
Overconfidence
Outcome bias
Inattentional Blindness
Benjamin Franklin effect
Hindsight bias
Gambler’s fallacy
Anecdotal evidenceNegativity bias
Loss aversion
Backfire effect
![Page 25: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/25.jpg)
H O W C O M P U T E R S D E C I D E FA S TM A C H I N E L E A R N I N G O F F E R S A N A LT E R N A T I V E T O H U M A N C O G N I T I V E B I A S E S A N D C A N B E M A D E FA S T T H R O U G H B I G D A TA
K-Means Clustering
Naive BayesSupport Vector Machines Affinity Propagation
Least Angle Regression
Nearest Neighbors
Decision Trees
Markov Chain Monte Carlo
Spectral clustering
Restricted Bolzmann Machines
Logistic Regression
![Page 26: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/26.jpg)
Better decisions through predictive applications
![Page 27: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/27.jpg)
H O W P R E D I C T I V E A P P L I C AT I O N S W O R K
C O L L E C T & S T O R E
A N A LY Z E C O R R E L AT I O N S
B U I L D D E C I S I O N M O D E L
D E C I D E & T E S T O P T I M I Z E
![Page 28: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/28.jpg)
W H Y T E S T ?
– R A N D A L L M U N R O E
“Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look
over there’”
![Page 29: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/29.jpg)
S T O R Y T I M E(Not safe for vegetarians)
![Page 30: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/30.jpg)
T H E G R O U N D B E E F D I L E M M A
![Page 31: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/31.jpg)
T H E G R O U N D B E E F D I L E M M A
Yesterday Today Tomorrow Next Delivery Next Day
In Stock Demand
![Page 32: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/32.jpg)
T H E G R O U N D B E E F D I L E M M A
• Order too much and you will have to throw meat away when it goes bad. You lose money and cows die in vain
• Order too little and you won’t serve all your potential customers. You lose money and customers stay hungry.
![Page 33: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/33.jpg)
A C C U R AT E LY P R E D I C T D E M A N DC H A L L E N G E # 1
![Page 34: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/34.jpg)
A U T O M AT E R E P L E N I S H M E N TC H A L L E N G E # 2
C O L L E C T S T O C K A N D
S A L E S
P R E D I C T D E M A N D
T R A D E O F F C O S T S
C R E AT E O R D E R S I N E R P
S Y S T E M
O P T I M I Z E & R E P E AT
![Page 35: Big Data Berlin – Automating Decisions is the Next Frontier for Big Data](https://reader034.fdocuments.us/reader034/viewer/2022042606/54b3bea04a79591d438b4580/html5/thumbnails/35.jpg)
@ B YA N A LY T I C S _ E N @ T R I E L O F F