Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in...
Transcript of Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in...
![Page 1: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/1.jpg)
Ensuring big data is supporting financial analytics
• Gaining a thorough understanding of big data in order to understand analytics
• Transforming unstructured data into structured intelligence
• Using big data to predict client behaviour
Bhavani Raskutti @ Pacific Brands
![Page 2: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/2.jpg)
Ensuring big data is supporting financial analytics
• Gaining a thorough understanding of big data in order to understand analytics
• Transforming unstructured data into structured intelligence
• Using big data to predict client behaviour
Bhavani Raskutti @ Pacific Brands
![Page 3: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/3.jpg)
Agenda
• Big data
• Unstructured data
• Framework for embedding financial analytics – data analysis leading to decisions that
– impact company financials
Bhavani Raskutti @ Pacific Brands
![Page 4: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/4.jpg)
Big Data“Big data” is high-volume, -velocity and -variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.
-- Gartner Inc. 2001
• Volume: Number of entries (rows) – NOT attributes (columns)
• Velocity: Rate of change, usually rate of arrival of rows– Impacts volume
• Variety: attributes (columns) of entries– Structured:
• Transaction data: value, timestamp, type, location, …• Customer data: gender, age, occupation, …
– Unstructured: • Text: Customer interactions, blogs & twitters about the company …• Image: Customer signature, photo, ….
Big Data
![Page 5: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/5.jpg)
Predictive Analytics to Make DecisionsPredict fraudulent credit card transactions
1. Learn a model from historical data
Know
n Ca
tego
ries
NO
Kno
wn
Cate
gorie
s
Classification
Clustering Classification
2. Categorise new data to make decisions
Back-end process Time consuming Built from sampled data
Volume NOT an issue
Models simple Quick process
Velocity & Volume NOT an issueN
ew d
ata
Big Data
![Page 6: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/6.jpg)
Impact of Variety on AnalyticsIndustry Structured Data Unstructured Data Analytics
MaturityFinance • Transactions
• Customer & Product• Call centre
• Twitter/blogs/complaints
Wholesale / Retail
• Inventory, sales, POS• Customer & Product• Sourcing & supply chain
• Twitter/blogs/complaints
Utilities (gas, electricity, …)
• Usage data• Customer & Product• Sensor data
• Twitter/blogs/complaints
Telecom • Call records• Customer & Product• Call centre
• Twitter/blogs/complaints• Inbound emails/SMS• GPS data
Web services • Usage• Product & subscriber• Call centre
• Twitter/blogs/complaints• Inbound emails• Click stream transactions
Benchmark
Big Data
![Page 7: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/7.jpg)
Impact of Variety on AnalyticsIndustry Structured Data Unstructured Data Analytics
MaturityFinance • Transactions
• Customer & Product• Call centre
• Twitter/blogs/complaints
Wholesale / Retail
• Inventory, sales, POS• Customer & Product• Sourcing & supply chain
• Twitter/blogs/complaints
Utilities (gas, electricity, …)
• Usage data• Customer & Product• Sensor data
• Twitter/blogs/complaints
Telecom • Call records• Customer & Product• Call centre
• Twitter/blogs/complaints• Inbound emails/SMS• GPS data
Web services • Usage• Product & subscriber• Call centre
• Twitter/blogs/complaints• Inbound emails• Click stream transactions
Benchmark
Big Data
![Page 8: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/8.jpg)
Impact of Variety on AnalyticsIndustry Structured Data Unstructured Data Analytics
MaturityFinance • Transactions
• Customer & Product• Call centre
• Twitter/blogs/complaints
Wholesale / Retail
• Inventory, sales, POS• Customer & Product• Sourcing & supply chain
• Twitter/blogs/complaints
Utilities (gas, electricity, …)
• Usage data• Customer & Product• Sensor data
• Twitter/blogs/complaints
Telecom • Call records• Customer & Product• Call centre
• Twitter/blogs/complaints• Inbound emails/SMS• GPS data
Web services • Usage• Product & subscriber• Call centre
• Twitter/blogs/complaints• Inbound emails• Click stream transactions
Benchmark
Big Data
![Page 9: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/9.jpg)
Impact of Variety on AnalyticsIndustry Structured Data Unstructured Data Analytics
MaturityFinance • Transactions
• Customer & Product• Call centre
• Twitter/blogs/complaints
Wholesale / Retail
• Inventory, sales, POS• Customer & Product• Sourcing & supply chain
• Twitter/blogs/complaints
Utilities (gas, electricity, …)
• Usage data• Customer & Product• Sensor data
• Twitter/blogs/complaints
Telecom • Call records• Customer & Product• Call centre
• Twitter/blogs/complaints• Inbound emails/SMS• GPS data
Web services • Usage• Product & subscriber• Call centre
• Twitter/blogs/complaints• Inbound emails• Click stream transactions
Benchmark
Big Data
![Page 10: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/10.jpg)
Impact of Variety on AnalyticsIndustry Structured Data Unstructured Data Analytics
MaturityFinance • Transactions
• Customer & Product• Call centre
• Twitter/blogs/complaints
Wholesale / Retail
• Inventory, sales, POS• Customer & Product• Sourcing & supply chain
• Twitter/blogs/complaints
Utilities (gas, electricity, …)
• Usage data• Customer & Product• Sensor data
• Twitter/blogs/complaints
Telecom • Call records• Customer & Product• Call centre
• Twitter/blogs/complaints• Inbound emails/SMS• GPS data
Web services • Usage• Product & subscriber• Call centre
• Twitter/blogs/complaints• Inbound emails• Click stream transactions
Benchmark
Big Data
![Page 11: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/11.jpg)
Structuring Unstructured Data Unstructured Data
Get freeform
source text
Load sentiment word list
love, best, … hate, worst, …
Score sentiment:+1 for positive words-1 for negative words Normalise if needed
1. Sentiment Analysis 2. Topic Detection
Use in Business:For model buildingCompetitor comparisonsMood change over timeAddressing negative scores
Get freeform
source text
Load stop words list:the , on, of, a, an, by, …
Create clusters with term frequency matrix
Use in Business:For model buildingMajor customer issuesImpact of initiatives
Determine clusters of interest and label
Learn cluster models for classification
Business input
![Page 12: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/12.jpg)
Sentiment Analysis OutputUnstructured Data
No sarcasm detection
![Page 13: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/13.jpg)
Topic Detection OutputUnstructured Data
![Page 14: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/14.jpg)
Impact of Variety on AnalyticsIndustry Structured Data Unstructured Data Analytics
MaturityFinance • Transactions
• Customer & Product• Call centre
• Twitter/blogs/complaints
Wholesale / Retail
• Inventory, sales, POS• Customer & Product• Sourcing & supply chain
• Twitter/blogs/complaints
Utilities (gas, electricity, …)
• Usage data• Customer & Product• Sensor data
• Twitter/blogs/complaints
Telecom • Call records• Customer & Product• Call centre
• Twitter/blogs/complaints• Inbound emails/SMS• GPS data
Web services • Usage• Product & subscriber• Call centre
• Twitter/blogs/complaints• Inbound emails• Click stream transactions
Benchmark
Big Data
Variety NOT an issue Financial Analytics NOT impacted by Big Data
![Page 15: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/15.jpg)
Impact of Variety on AnalyticsIndustry Structured Data Unstructured Data Analytics
MaturityFinance • Transactions
• Customer & Product• Call centre
• Twitter/blogs/complaints
Wholesale / Retail
• Inventory, sales, POS• Customer & Product• Sourcing & supply chain
• Twitter/blogs/complaints
Utilities (gas, electricity, …)
• Usage data• Customer & Product• Sensor data
• Twitter/blogs/complaints
Telecom • Call records• Customer & Product• Call centre
• Twitter/blogs/complaints• Inbound emails/SMS• GPS data
Web services • Usage• Product & subscriber• Call centre
• Twitter/blogs/complaints• Inbound emails• Click stream transactions
Benchmark
Big Data
![Page 16: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/16.jpg)
Impact of Variety on AnalyticsIndustry Structured Data Unstructured Data Analytics
MaturityFinance • Transactions
• Customer & Product• Call centre
• Twitter/blogs/complaints
Wholesale / Retail
• Inventory, sales, POS• Customer & Product• Sourcing & supply chain
• Twitter/blogs/complaints
Utilities (gas, electricity, …)
• Usage data• Customer & Product• Sensor data
• Twitter/blogs/complaints
Telecom • Call records• Customer & Product• Call centre
• Twitter/blogs/complaints• Inbound emails/SMS• GPS data
Web services • Usage• Product & subscriber• Call centre
• Twitter/blogs/complaints• Inbound emails• Click stream transactions
Benchmark
Big Data
Do BIG analytics with enough data!!
![Page 17: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/17.jpg)
Decision
• Big $ impact• Many people• Timely
Framework for Embedding Analytics Initiative
Insight
Pilot
Data
Deploy
• Just the facts needed for decision making
• Prioritise entries• Support actioning
• Sell to whom, what & $$• Ordered by $$
Framework
Right People
$
Decision Support Initiative for Wholesale Sales
![Page 18: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/18.jpg)
Demand
In-s
tock
%
· R1· R2
Demand
Sell
Rate
Example of Additional Support for ActioningFramework
Sell rate vs Consumer Demand plot • Each point is a store• R1 & R2 are comparable retailers• Values for the same product
Possible reasons for difference• Competing product at R2• Pricing at R2 vs R1• Lack of stock at R2
![Page 19: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/19.jpg)
Decision
• Big $ impact• Many people• Timely
Framework for Embedding Analytics Initiative
Insight
Pilot
Data
Deploy
• Just the facts needed for decision making
• Prioritise entries• Support actioning
• Sell to whom, what & $$• Ordered by $$
Framework
• Automated feed• Objective Data• Most specific • Complete
• POS feed from retailers• SKU & store master
• Pick champions early in the process
• Develop pilot• Validate outputs from
pilot• Iterate pilot with
champions until it is accepted
• Automate • Helpdesk• Training
Right People
$
Decision Support Initiative for Wholesale Sales
![Page 20: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/20.jpg)
Conclusion• Big data is not an impediment to analytics
• Unstructured data can be structured using sentiment analysis & topic detection
• Key success factors for doing BIG analytics is to have the right people to choose–Right decisions–Right insights–Right data –Right process to pilot & deploy
![Page 21: Ensuring big data is supporting financial analytics Gaining a thorough understanding of big data in order to understand analytics Transforming unstructured.](https://reader038.fdocuments.us/reader038/viewer/2022110319/56649c7d5503460f949320f8/html5/thumbnails/21.jpg)
Questions?