How to Act on Big Data in Real Time – Part 2
Name · Title · Dunn SolutionsJanani EshwaranRenata Simanjuntak
Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next
Dunn Solutions is a Full-Service IT Consulting FirmFounded in 1988
Raleigh, NCDelivery Training
Bangalore, IndiaDelivery
MinneapolisDelivery Training Chicago
Delivery
Practice Areas
Application Development• Portals• eCommerce &
Content Managed Websites
• Mobile App Development
• Custom App Development
Training• Certified
SAP/Liferay• Classroom, On-
site, Computer Based & Virtual
• Mentoring & Custom Training
Frameworks• Accountable
Care Orgs (ACO’s)
• Corporate Legal • Higher
Education• Optical Shop
Solutions
Analytics• Analytics & BI
Platforms• Data
Warehouse & Data Integration
Big DataPredictive Analytics
Selected Clients
PartnershipsPartnerships
Analytics Practice
Business Intelligence
Big Data
Data IntegrationBusiness Analytics
Data Warehousing
• KPI’s and Metrics• Dashboards• Data Exploration
and Visualization• Ad Hoc Analysis &
Reporting
• Data Mining• Predictive
Analytics• Prescriptive
Analytics• R, AzureML
• Hadoop, MapReduce
• AWS and Azure• Hive, Sqoop, Spark• NoSQL
• Data Lake• Columnar• In-memory• EIM (Data
Integration and Data Quality
• Dimensional Modeling
Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next
Real Time Big Data Analytics
• It is not only to store and analyze streaming BIG data• It is more about making better decision
and taking meaningful action at the right time
Traditional Enterprise Data Warehouse plus analytics are no longer enough
• Fraud detection while a credit card is swiped
• Triggering an offer while a shopper is standing on a checkout line
• Placing an ad on a website while someone is reading a specific article
Real Time Business Benefit
• Service improvement• Cost savings• Fraud detection• Keep up with customer trends• Sales insights enhancement• Instantly errors detection• Immediate new strategies of your
competition notification
Power of Real Time Analytics
• Financial Loss• External Confidence• Company Morale• Increased Audit Costs
How Fraud Hurts You & Your Organization
Impact of Fraud in Telecom Industry
Top 5 Fraud Type in Telcom Industry
Communications Fraud Control Association (CFCA) Global Fraud Loss Survey
Telecom Fraud cost the industry 2015 over 38 Billion USD annually
Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next
Azure Event Hub
Customer Name / 16
• Benefits:• Stream millions of events per second• Process events with variable load profiles• Connect millions of devices across platforms
• How much data? Throughput units• To scale the traffic coming in or out• Key pricing parameter• In (Publisher): 1mb or 1000 events/sec• Out (Consumer): 2mb/sec
Azure HDInsight - Ecosystem
• Big Data with No Hassle• Open and
Flexible• Insight in MS
Excel• Build Big Data
Apps your Way
• Scalable• High-throughput• Fault-tolerant • Stream processing of live data streams• Data collected can be later post-processed • Code and business logic can be shared and reused
Spark Streaming for Real Time Analytics
SparkStreaming
SparkEngine
Input Data Stream
Batches of input data
Batches of processed data
Less time learning, implementing, and maintaining different frameworksMore focus on developing smarter applications
Hive
Customer Name / 19
• Data warehouse in Hadoop• Project structure on largely unstructured
data• Work with structured and semi-structured
data
• Hive QL
• Low cost data storage
• Empower user• Q&A function• Dashboard visualization• Innovative technology• In memory engine• Columnar database
• You own your data• Faster turn around• Lower cost
Power BI
Customer Name / 20
Enterprise-level data is yours for free or at a very low monthly cost
• Email• Link• Website• Phone: text or call• Application
Real Time Alert
Customer Name / 21
You are informed in real time when errors or frauds or anomalies existTake action in real time for real results
Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next
Big Data Real Time Use Case for Telecom (1)
• Real-time fraud prevention• Can be passed on to customer bills• Prevent revenue loss and additional expense to
correct • Visibility of service performance, costs and
discounts to the customer• Cannot monitor customer bills to provide services• Analyze and offer products and discounts
• Optimization of Least Cost Routing (LCR) • Choose lost cost network in real time • Select optimized and high performing network
quickly
Big Data Real Time Use Case for Telecom (2)
• Call performance monitoring • Cannot prevent dropped calls and issues• Can identify issues to resolve immediately
• Real-time profitability analysis • Make use of long term trend data offline• Can learn service provided to customer for
understanding gross margin
What if I wanted to…
• Capture data from any application in real time• Store the data• Perform analysis on the streamed data• Visualize the information interactively
Demonstration: Setting the Stage
Big Data, Real time project checklist• Azure Event Hub• Azure HDInsight cluster
• Spark Streaming• Hive
• Azure SQL database• Power BI• Real Time Notification (email)
What Do I Need?
Our Demo Architecture
Streaming data sources
(Call Records)
Azure Event Hub
Consume Store
Streaming data
Enrichment data
Step 1: Start The Event Hub Event To Collect The Events
Streaming data sources (Call Records)
Azure Event Hub
Step 2: Prepare Your Receiver To Receive Events
Streaming data sources (Call Records)
Azure Event Hub
Consume
Step 3: Persist The Events In Hive Table
Streaming data sources (Call Records)
Azure Event Hub
Consume Store
Step 4: Alert /Notify the anomaly
Streaming data sources (Call Records)
Azure Event Hub
Consume Store
Step 5: Visualization in Power BI
Streaming data sources
(Call Records)
Azure Event Hub
Consume Store
Streaming data
Enrichment data
Today’s Agenda
Introduction
Impact of Real Time Analytics in Telecom Industry
Microsoft Azure HDInsight
Demo –Telecom Real Time Fraud Detection
What’s Next
Recap
Streaming data sources
(Call Records)
Azure Event Hub
Consume Store
Streaming data
Enrichment data
1. Capture streaming data
2. Process and store data
4. Analyze
3. Alert by email
• What is your Big Data strategy?• Do you have a Big Data project in mind?• Are you wondering how you can use Big Data
for real time data analysis to benefit your company?• Should you do it on premise or in the cloud?
• Contact us and we’ll help you execute!• [email protected]
Let’s Get You Started with Real-Time Big Data
Thank YouJanani Eshwaran· Analytics Consultant · Dunn Solutions
Renata Simanjuntak· Analytics Manager· Dunn [email protected]
• http://cfca.org/fraudlosssurvey/2015.pdf
• http://www.cfca.org/pdf/survey/CFCA2013GlobalFraudLossSurvey-pressrelease.pdf
• Microsoft blogs and tutorials
• Other analytics webinars:• http://www.dunnsolutions.com/content/webinar-white-
paper
Reference
Top Related